title: R: Increase my rvest scraper's speed?
body: <p>I just started scraping with R's rvest library. All too bold, I started with a query for 3206 sub-pages, each for which I want to extract a string.</p>
  
  <p><a href="https://i.stack.imgur.com/o4PLg.jpg" rel="noreferrer"><img src="https://i.stack.imgur.com/o4PLg.jpg" alt="string to srape"></a></p>
  
  <p><strong>The problem is:</strong></p>
  
  <p>Duration.</p>
  
  <p><strong>My question is:</strong></p>
  
  <p>Can I optimize my script (below) to speed up its execution?</p>
  
  <p><strong>Background:</strong></p>
  
  <p>Looping just for 3 integers works fine, but my script for all has now been running for a long time. I don't know any Python (which I might change since I heard that there's a thing called <code>aiohttp</code>). If there is no other way, I'd be grateful, if someone could provide the link to a good tutorial or an alternative solution.</p>
  
  <p><strong>Script</strong></p>
  
  <pre><code>library(rvest)
  library(data.table)
  #READ IN HTML
  #Link: http://globalenergyobservatory.org/list.php?db=PowerPlants&amp;type=Hydro
  hydro &lt;- read_html("http://globalenergyobservatory.org/list.php?db=PowerPlants&amp;type=Hydro")
  #GET ATTRIBUTES
  attributes &lt;- hydro %&gt;%
    html_nodes("#list a") %&gt;%
    html_attrs()
  #WRITE URLs
  urls = list()
  for (i in 1:3206) {
    da &lt;- unlist(attributes[i])
    dat &lt;- da[1]
    data &lt;- paste("http://www.globalenergyobservatory.org/",dat, sep="")
    urls[[i]] &lt;- data
  }
  #GET ABSTRACTS
  abstracts = list()
  for(i in 1:3206) {
    to_use &lt;- read_html(urls[[i]])
    to_use %&gt;%
      html_nodes("#Abstract_Block td") %&gt;%
      html_text() -&gt; to_write
    abstracts[[i]] &lt;- to_write
  }
  </code></pre>