title: Breaking my code by putting Dictionaries in a list. What am I doing wrong?
body: <p>I made a webscraper for a supermarket's website. It believe I did everything right, but somehow I messed up using dictionaries in lists, while using a for loop. My code wont even print text anymore and I can't find the solution myself.</p>
  
  <p>I tried putting an empty dic outside the loop, on different spots in the code but it all seems to fail. I tried putting simple <code>print("test")</code>code throughout to see where the code fails. Still, the fail seems illogical.</p>
  
  <pre class="lang-py prettyprint-override"><code>import time
  import csv
  from urllib.parse import urljoin
  from selenium import webdriver
  
  
  # csvFile = open('products.csv', 'w')
  # writer = csv.writer(csvFile)
  # writer.writerow(('name', 'price'))
  
  pagenum = 1
  
  base_url = 'https://www.ah.nl'
  search_url = '/zoeken?query=groente&amp;page={}'.format(pagenum)
  active_url = urljoin(base_url,search_url)
  driver = webdriver.Chrome()
  driver.get(active_url)
  driver.implicitly_wait(100)
  
  next_page = driver.find_elements_by_tag_name('article')
  total_products = driver.find_element_by_class_name('f-load-more')
  T = total_products.text.split()
  print(" Total products: " + str(T[3]))
  
  var = len(next_page)-1
  h = next_page[var]
  
  while int(T[0]) &lt; int(T[3]):
      try:
          time.sleep(1)
          driver.execute_script("arguments[0].scrollIntoView();", h)
          total_products.find_element_by_tag_name('button').click()
          time.sleep(1)
  
      except:
          print("We did it boys. Proceeding with products ..")
          break
  
  def linkGrabber():
      items = driver.find_elements_by_tag_name('article')
      print(str(len(items)) + " items found on page.")
      urls = []
      for i in items:
          try:
              L = i.find_element_by_tag_name('a') .get_attribute('href')
              urls.append([L])
          except:
              print('link for' + str(i) + ' not found.')
      return urls
  
  yum = linkGrabber()
  # for practise purposes:
  yum = yum[0:3]
  
  
  P_LIST = []
  
  for i in yum:
      newdict = {}
      i = "".join(i)
      driver.get(i)
      TITLE = driver.find_element_by_xpath('//*[@id="app"]/main').get_attribute('title')
      newdict["url"] = i
      newdict["name"] = TITLE
      try:
          x = driver.find_element_by_class_name('product-info-content-block')
          x = x.find_element_by_tag_name('p').text
          newdict["weight/amount"] = x
      except:
          continue
      try:
          NUTRITION_TABLE = driver.find_element_by_tag_name('tbody')
          ENERGY = NUTRITION_TABLE.find_elements_by_tag_name('tr')
          for i in ENERGY:
              A = i.find_elements_by_tag_name('td')
              UNIT = A[0].text
              VALUE = A[1].text
              newdict[UNIT] = VALUE
          P_LIST.append(newdict)
  
      except:
          continue
  
      print("-----------------------------------")
  
  print(P_LIST)
  
  
  # csvFile.close()
  </code></pre>
  
  <p>output: </p>
  
  <pre><code>We did it boys. Proceeding with products ..
  
  1444 items found on page.
  
  link for&lt;selenium.webdriver.element&gt; not found.
  
  []
  </code></pre>
  
  <p>I'm trying to grab all info from unique products in a list of dictionaries. After that, I'll write this info to a csv file (code in progress). I'll start using Pandas as well.</p>
  
  <p>If you have any tips on structuring my code or anything else, please feel free to share them. Thanks for helping.</p>