title: Python, Selenium, BS4 - Navigating to the next Page
body: <p>Part of my HTML looks like below:</p>
  
  <pre><code>&lt;div id="qryNav"&gt;
  &lt;form method="post" action="OffQryRedirector.jsp" id="form1" name="form1"&gt;
      &lt;input type="hidden" name="NextPage" value="7" /&gt;
      &lt;input type="submit" name="Action" id="oq-nav-begin" value="&amp;lt;&amp;lt;" /&gt;
      &lt;input type="submit" name="Action" id="oq-nav-prv" value="&amp;lt;" /&gt;
  &lt;span class="oq-nav-btwn"&gt;Page 1 of 4&lt;/span&gt;
      &lt;input type="submit" name="Action" id="oq-nav-nxt" value="&amp;gt;" /&gt;
      &lt;input type="submit" name="Action" id="oq-nav-end" value="&amp;gt;&amp;gt;" /&gt;  
  &lt;/form&gt;
  &lt;a href="OffQryForm.jsp" class="qryNav"&gt;&lt;span&gt;Start a New Search&lt;/span&gt;&lt;/a&gt; 
  &lt;!--&lt;a href="javascript:history.back()" class="qryNav"&gt;&lt;span&gt;Modify Your Search&lt;/span&gt;    &lt;/a&gt;--&gt; 
  &lt;/div&gt;
  </code></pre>
  
  <p>I am trying to identify the Number of pages and then move to the next page.
  My code looks like below - </p>
  
  <pre><code>html = driver.page_source
  soup = BeautifulSoup(html)
  pages =  soup.find_all('span', {'class': 'oq-nav-btwn'})[0].text.encode('ascii',     'ignore').strip().upper()
  loc_of = pages.find('OF')
  num_pages = int(pages[loc_of+2:].strip())
  &gt;&gt;&gt; print num_pages
  4
  span = soup.find_all('span', {'class': 'oq-nav-btwn'})
  elem2 = span[0].find_next_sibling() 
  elem2.find_element_by_id("oq-nav-nxt")
  </code></pre>
  
  <p>Post this i am trying to run a loop for each of the 4 Pages - 1.. 4. However when i use </p>
  
  <pre><code>elem2.find_element_by_id("oq-nav-nxt").click()
  </code></pre>
  
  <p>I get the standard selenium.common.exceptions.StaleElementReferenceException: Message: u'stale element reference: element is not attached to the page document\n  (Session info: chrome=34.0.1847.131)\n  (Driver info: chromedriver=2.9.248315,platform=Windows NT 6.1 x86_64)'</p>
  
  <p>The element is visible. I dont think that the try.. catch... wait.. is the solution for this.. (I may be wrong here.)</p>
  
  <p>I also tried to do the same with the below code - </p>
  
  <pre><code>span = soup.find_all('span', {'class': 'oq-nav-btwn'})
  elem2 = span[0].find_next_sibling()
  &gt;&gt;&gt; print elem2
  &lt;input id="oq-nav-nxt" name="Action" type="submit" value="&amp;gt;"&gt;
  &lt;input id="oq-nav-end" name="Action" type="submit" value="&amp;gt;&amp;gt;"&gt;
  &lt;/input&gt;&lt;/input&gt;
  </code></pre>
  
  <p>But i am unable to navigate the elem2 value above and then click on the "oq-nav-nxt" button.</p>
  
  <p>Your help on this is appreciated.</p>