title: Python: How do I iterate through an HTML Element Object with LXML/Requests?
body: <p>I'm trying to use LXML &amp; Requests to create a data table from a website. I need both text within a tag and text that's contained within a tag. Here's the HTML:</p>
  
  <pre><code>&lt;div class="houses"&gt;
      &lt;input type="hidden" class="houseNumber" value="107"&gt;
      &lt;input type="hidden" class="houseState" value="MT"&gt;
      &lt;input type="hidden" class="houseStatus" value="Occupied"&gt;
  &lt;div class="houseInfo"&gt;
      &lt;div class="houseCity"&gt;Helena&lt;/div&gt;
      &lt;div class="houseArea"&gt;Helena Valley&lt;/div&gt;
  &lt;/div&gt;
  &lt;/div&gt;
  &lt;div class="houses"&gt;
      &lt;input type="hidden" class="houseNumber" value="237"&gt;
      &lt;input type="hidden" class="houseState" value="MT"&gt;
      &lt;input type="hidden" class="houseStatus" value="Occupied"&gt;
  &lt;div class="houseInfo"&gt;
      &lt;div class="houseCity"&gt;East Helena&lt;/div&gt;
      &lt;div class="houseArea"&gt;Helena Valley&lt;/div&gt;
  &lt;/div&gt;
  &lt;/div&gt;
  &lt;div class="houses"&gt;
      &lt;input type="hidden" class="houseNumber" value="104"&gt;
      &lt;input type="hidden" class="houseState" value="MT"&gt;
      &lt;input type="hidden" class="houseStatus" value="Vacant"&gt;
  &lt;div class="houseInfo"&gt;
      &lt;div class="houseCity"&gt;Helena&lt;/div&gt;
      &lt;div class="houseArea"&gt;Helena Valley&lt;/div&gt;
  &lt;/div&gt;
  &lt;/div&gt;
  </code></pre>
  
  <p>From this, I would like to create a table like this:</p>
  
  <pre><code>['107', 'MT', 'Occupied', 'Helena', 'Helena Valley']
  ['237', 'MT', 'Occupied', 'East Helena', 'Helena Valley']
  ['104', 'MT', 'Vacant', 'Helena', 'Helena Valley']
  </code></pre>
  
  <p>Using Requests &amp; LXML, I'm trying to iterate through <code>div class="houses"</code> to get what I need, but every time I try to print the values, it prints this:</p>
  
  <pre><code>['107', '237', '104']
  ['MT', 'MT', 'MT']
  ['Occupied', 'Occupied', 'Vacant']
  ['Helena', 'East Helena', 'Helena']
  ['Helena Valley', 'Helena Valley', 'Helena Valley']
  ['107', '237', '104']
  ['MT', 'MT', 'MT']
  ['Occupied', 'Occupied', 'Vacant']
  ['Helena', 'East Helena', 'Helena']
  ['Helena Valley', 'Helena Valley', 'Helena Valley']
  ['107', '237', '104']
  ['MT', 'MT', 'MT']
  ['Occupied', 'Occupied', 'Vacant']
  ['Helena', 'East Helena', 'Helena']
  ['Helena Valley', 'Helena Valley', 'Helena Valley']
  </code></pre>
  
  <p>Here's my partial code:</p>
  
  <pre><code>link = "example.com"
  headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36'}
  response = requests.get(link, headers=headers, allow_redirects=False) 
  sourceCode = response.content
  
  htmlElem = html.document_fromstring(sourceCode)
  houses = htmlElem.find_class('houses')
  for house in houses:
      houseNumber = house.xpath('//input[@class="houseNumber"]/@value')
      houseState = house.xpath('//input[@class="houseState"]/@value')
      houseStatus = house.xpath('//input[@class="houseStatus"]/@value')
  </code></pre>
  
  <p>How do I capture the data in a table as illustrated above? Can I iterate through the houses object in a different way?</p>
  
  <p>UPDATE: @efirvida I've modified the code to the following:</p>
  
  <pre><code>link = "example.com"
  headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36'}
  response = requests.get(link, headers=headers, allow_redirects=False) 
  sourceCode = response.content
  
  htmlElem = html.document_fromstring(sourceCode)
  houses = htmlElem.find_class('houses')
  houseNumber = []
  houseState = []
  houseStatus = []
  
  for house in houses:
      houseNumber.append(house.xpath('//input[@class="houseNumber"]/@value'))
      print(houseNumber)
      houseState.append(house.xpath('//input[@class="houseState"]/@value'))
      houseStatus.append(house.xpath('//input[@class="houseStatus"]/@value'))
  
  data = map(list, zip(*[houseNumber,houseState,houseStatus]))
  </code></pre>
  
  <p>When I do this, the following prints:</p>
  
  <pre><code>[['107', '237', '104']]
  [['107', '237', '104'], ['107', '237', '104']]
  [['107', '237', '104']], ['107', '237', '104'], ['107', '237', '104']]
  </code></pre>