title: Exporting twitter data to Gephi using R
body: <p>I have compiled a dataset consisting of thousands of tweets using R.</p>
  
  <p>The dataset basically looks like this:</p>
  
  <pre><code>Data &lt;- data.frame(
    X = c(1,2),
    text = c("Hello @User1 #hashtag1, hello @User2 and @User3, #hashtag2", "Hello @User2 #hashtag3, hello @User1 and @User3, #hashtag4"),
    screenname = c("author1", "author2")
  )
  </code></pre>
  
  <p>Now I want to export this dataset to a Gephi supported graph format (see <a href="http://gephi.github.io/users/supported-graph-formats/" rel="nofollow noreferrer">Supported Graph Formats - Gephi</a>)</p>
  
  <p>Whenever an "author" mentions a @user in the text, there should be a direct link from the author to the user. In the case above, the results should be like this:</p>
  
  <p>author1 -> @User2</p>
  
  <p>author1 -> @User3</p>
  
  <p>author2 -> @User1</p>
  
  <p>author2 -> @User3</p>
  
  <p>How can I manipulate my dataset and export it to a Gephi supported Graph Format?</p>
  
  <p>If possible, I would prefer GEXF or GraphML format. If that is not possible, I can also work with csv or a spreadsheet.</p>
  
  <hr>
  
  <p>I thought about solving this problem the whole night and made a few steps in the right direction (at least I hope so). But I need your help.</p>
  
  <p>As mentioned above, I have basically the following dataset:</p>
  
  <pre><code>Data &lt;- data.frame(
    X = c(1,2),
    text = c("Hello @User1 #hashtag1, hello @User2 and @User3, #hashtag2", "Hello @User2 #hashtag3, hello @User1 and @User3, #hashtag4"),
    screenname = c("author1", "author2")
  )
  </code></pre>
  
  <p>I want to export it to a GEXF format to use it in Gephi.</p>
  
  <p>There is a r package for exporting r data into GEXF, called rgexf (see <a href="https://bitbucket.org/gvegayon/rgexf/wiki/Installation" rel="nofollow noreferrer">https://bitbucket.org/gvegayon/rgexf/wiki/Installation</a>). To use the <code>write.gexf</code> function of the package, I need at least two things:</p>
  
  <p>1) a matrix of all the nodes in the network (in my case authors, users and hashtags)</p>
  
  <p>2) a matrix of all the edges between these nodes (i.e., the connections between authors and users as well as hashtags).</p>
  
  <p>In my Twitter data, authors are never printed with "@", although they also can be "users". So I have at first to add "@" to the authors, to avoid duplication of nodes.</p>
  
  <pre><code>data$screenname &lt;- sub("^", "@", data$screenname )
  </code></pre>
  
  <p>Then I need a matrix, consisting of all the nodes in my network (i.e., authors, users and hashtags). According to this example, the output should look like this:</p>
  
  <pre><code>people &lt;- data.frame(matrix(c(1:9, '@author1', '@author2', '@user1', '@user2', '@user3', '#hashtag1', '#hashtag2', '#hashtag3', '#hashtag4'),ncol=2))
  </code></pre>
  
  <p>Then I need a matrix of all the edges between these nodes. According to this example, the output should look like this:</p>
  
  <pre><code>relations &lt;- data.frame(matrix(c(1,3,1,4,1,5,1,6,1,7,2,4,2,3,2,5,2,8,2,9), ncol=2, byrow=T))
  </code></pre>
  
  <p>Finally, I only have to put these two things together:</p>
  
  <pre><code>write.gexf(people, relations)
  </code></pre>
  
  <p>to get the following file:</p>
  
  <pre><code>&lt;?xml version="1.0" encoding="UTF-8"?&gt;
  &lt;gexf xmlns="http://www.gexf.net/1.2draft" xmlns:viz="http://www.gexf.net/1.1draft/viz" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.gexf.net/1.2draft http://www.gexf.net/1.2draft/gexf.xsd" version="1.2"&gt;
    &lt;meta lastmodifieddate="2015-02-04"&gt;
      &lt;creator&gt;NodosChile&lt;/creator&gt;
      &lt;description&gt;A graph file writing in R using "rgexf"&lt;/description&gt;
      &lt;keywords&gt;gexf graph, NodosChile, R, rgexf&lt;/keywords&gt;
    &lt;/meta&gt;
    &lt;graph mode="static" defaultedgetype="undirected"&gt;
      &lt;nodes&gt;
        &lt;node id="1" label="@author1"/&gt;
        &lt;node id="2" label="@author2"/&gt;
        &lt;node id="3" label="@user1"/&gt;
        &lt;node id="4" label="@user2"/&gt;
        &lt;node id="5" label="@user3"/&gt;
        &lt;node id="6" label="#hashtag1"/&gt;
        &lt;node id="7" label="#hashtag2"/&gt;
        &lt;node id="8" label="#hashtag3"/&gt;
        &lt;node id="9" label="#hashtag4"/&gt;
      &lt;/nodes&gt;
      &lt;edges&gt;
        &lt;edge id="0" source="1" target="3" weight="1"/&gt;
        &lt;edge id="1" source="1" target="4" weight="1"/&gt;
        &lt;edge id="2" source="1" target="5" weight="1"/&gt;
        &lt;edge id="3" source="1" target="6" weight="1"/&gt;
        &lt;edge id="4" source="1" target="7" weight="1"/&gt;
        &lt;edge id="5" source="2" target="4" weight="1"/&gt;
        &lt;edge id="6" source="2" target="3" weight="1"/&gt;
        &lt;edge id="7" source="2" target="5" weight="1"/&gt;
        &lt;edge id="8" source="2" target="8" weight="1"/&gt;
        &lt;edge id="9" source="2" target="9" weight="1"/&gt;
      &lt;/edges&gt;
    &lt;/graph&gt;
  &lt;/gexf&gt;
  </code></pre>
  
  <p>But how can I automatically extract the nodes and the relations between these nodes (the edges) from the example above and write them to two matrices?</p>
  
  <p>Does nobody know how to solve my problem?</p>
  
  <p>I tried to figure out, how to extract the nodes from my example (i.e., the authors, users and hashtags) and saving them to a data.frame (I am sure there is a shorter and more elegant way to do it!):</p>
  
  <pre><code>#extract Users and Hashtags from text, Authors from screenname (and add @ to Author-names)
  Users &lt;- stri_extract_all(Data$text, regex = "@[A-Za-z0-9]+")
  Hash &lt;- stri_extract_all(Data$text, regex = "#[A-Za-z0-9]+")
  Data$screenname &lt;- sub("^", "@", Data$screenname )
  Authors &lt;- stri_extract_all(Data$screenname, regex = "@[A-Za-z0-9]+")
  # delete NAs
  Users &lt;- Users[!is.na(Users)]
  Hash &lt;- Hash[!is.na(Hash)]
  # converting lists to vectors
  Users &lt;- unlist(Users)
  Hash &lt;- unlist(Hash)
  Authors &lt;- unlist(Authors)
  # merging the vectors to a single vector and deleting the duplicates
  nodes &lt;- unique(c(Authors, Users, Hash))
  # saving the vectors in a data.frame and giving each node a unique ID
  nodes &lt;- data.frame(matrix(c(1:length(nodes), nodes), ncol=2))
  colnames(nodes) &lt;- c("ID", "label")
  </code></pre>
  
  <p>But how can I build a data.frame for the edges?</p>
  
  <p>There must be a way to write a function which automatically checks if an author has mentioned a user and/or a hashtag row by row and write the result into a new data.frame, using the IDs of the authors, users and hashtags. Every connection should be displayed in two columns: source and target (1,2).</p>