title: How to scrape website if it has load more button to load more content on the page?
body: <pre><code>from selenium import webdriver
  import time
  
  driver = webdriver.Chrome(executable_path=r'C:\Users\gkhat\Downloads\chromedriver.exe')
  driver.get('https://www.allrecipes.com/recipes/233/world-cuisine/asian/indian/')
  card_titles = driver.find_elements_by_class_name('card__detailsContainer')
  button = driver.find_element_by_id('category-page-list-related-load-more-button')
  for card_title in card_titles:
      rname = card_title.find_element_by_class_name('card__title').text
      print(rname)
  
      time.sleep(3)
      driver.execute_script(&quot;arguments[0].scrollIntoView(true);&quot;, button)
      driver.execute_script(&quot;arguments[0].click();&quot;, button)
      time.sleep(3)
  
  driver.quit()
  </code></pre>
  <p>The <a href="https://www.allrecipes.com/recipes/233/world-cuisine/asian/indian/" rel="nofollow noreferrer">website</a> loads the food cards after clicking on the the &quot;Load More&quot; button the above code scrape the recipe title I want it keep scraping the title even after clicking the load more button.
  I tried the going to the Network tab the clicking on XHR but none of the requests shows the JSON. What should I do?</p>