title: how to crawl youtube video links with a keyword search
body: <p>I'm doing some research
  I want to crawl youtube video links(and then get video IDs)</p>
  
  <p>for example, if I type in keyword "obama" on youtube search bar
  then youtube.com will return the resulted page</p>
  
  <p><a href="http://www.youtube.com/results?search_query=htc&amp;page=2" rel="nofollow">http://www.youtube.com/results?search_query=htc&amp;page=2</a>
  scroll the page down to the bottom
  we can see the page number 1 2 3 4 5 6 7
  click on the page number from 2 to 3 to 4 to 5....
  then we can reach the maximum page number :50</p>
  
  <p>so I want to download all the pages from 1 to 50
  there is a url trick,like</p>
  
  <pre><code>http://www.youtube.com/results?search_query=obama&amp;page=5
  </code></pre>
  
  <p>however, if I use wget to download each page, sometimes, the downloading will be blocked
  or sometimes, when the page number is big, the returned page is not a abnormal page.</p>
  
  <p>so a correct way is to manually click on the page number button and use the web browser to download each page, is it possible to automate the mannual procedures?
  I searched on INternet, some poeple mentioned chrome javascript extention,
  anyway, are there any ways to write some automation scripts to mimic the mannual operation? thanks!</p>