title: Create pandas dataframe from json objects
body: <p>I finally have output of data I need from a file with many json objects but I need some help with converting the below output into a single dataframe as it loops through the data. Here is the code to produce the output including a sample of what the output looks like:</p>
  
  <p>original data:</p>
  
  <pre><code>{
  "zipcode":"08989",
  "current"{"canwc":null,"cig":4900,"class":"observation","clds":"OVC","day_ind":"D","dewpt":19,"expireTimeGMT":1385486700,"feels_like":34,"gust":null,"hi":37,"humidex":null,"icon_code":26,"icon_extd":2600,"max_temp":37,"wxMan":"wx1111"},
  "triggers":[53,31,9,21,48,7,40,178,55,179,176,26,103,175,33,51,20,57,112,30,50,113]
  }
  {
  "zipcode":"08990",
  "current":{"canwc":null,"cig":4900,"class":"observation","clds":"OVC","day_ind":"D","dewpt":19,"expireTimeGMT":1385486700,"feels_like":34,"gust":null,"hi":37,"humidex":null,"icon_code":26,"icon_extd":2600,"max_temp":37, "wxMan":"wx1111"},
  "triggers":[53,31,9,21,48,7,40,178,55,179,176,26,103,175,33,51,20,57,112,30,50,113]
  }
  
  def lines_per_n(f, n):
      for line in f:
          yield ''.join(chain([line], itertools.islice(f, n - 1)))
  
  for fin in glob.glob('*.txt'):
      with open(fin) as f:
          for chunk in lines_per_n(f, 5):
              try:
                  jfile = json.loads(chunk)
                  zipcode = jfile['zipcode']
                  datetime = jfile['current']['proc_time']
                  triggers = jfile['triggers']
                  print pd.Series(jfile['zipcode']), 
                        pd.Series(jfile['current']['proc_time']),\
                        jfile['triggers']          
              except ValueError, e:
                  pass
              else:
                  pass
  </code></pre>
  
  <p>Sample output I get when I run the above which I would like to store in a pandas dataframe as 3 columns. </p>
  
  <pre><code>08988 20131126102946 []
  08989 20131126102946 [53, 31, 9, 21, 48, 7, 40, 178, 55, 179]
  08988 20131126102946 []
  08989 20131126102946 [53, 31, 9, 21, 48, 7, 40, 178, 55, 179]
  00544 20131126102946 [178, 30, 176, 103, 179, 112, 21, 20, 48]
  </code></pre>
  
  <p>So the below code seems a lot closer in that it gives me a funky df if I pass the in the list and Transpose the df. Any idea on how I can get this reshaped properly?</p>
  
  <pre><code>def series_chunk(chunk):
      jfile = json.loads(chunk)
      zipcode = jfile['zipcode']
      datetime = jfile['current']['proc_time']
      triggers = jfile['triggers']
      return jfile['zipcode'],\
              jfile['current']['proc_time'],\
              jfile['triggers']
  
  for fin in glob.glob('*.txt'):
      with open(fin) as f:
          for chunk in lines_per_n(f, 7):
              df1 = pd.DataFrame(list(series_chunk(chunk)))
              print df1.T
  
  [u'08988', u'20131126102946', []]
  [u'08989', u'20131126102946', [53, 31, 9, 21, 48, 7, 40, 178, 55, 179]]
  [u'08988', u'20131126102946', []]
  [u'08989', u'20131126102946', [53, 31, 9, 21, 48, 7, 40, 178, 55, 179]]
  </code></pre>
  
  <p>Dataframe:</p>
  
  <pre><code>   0               1   2
  0  08988  20131126102946  []
         0               1                                                  2
  0  08989  20131126102946  [53, 31, 9, 21, 48, 7, 40, 178, 55, 179, 176, ...
         0               1   2
  0  08988  20131126102946  []
         0               1                                                  2
  0  08989  20131126102946  [53, 31, 9, 21, 48, 7, 40, 178, 55, 179, 176, ...
  </code></pre>
  
  <p>Here is my final code and output. How do I capture each dataframe it creates through the loop and concatenate them on the fly as one dataframe object? </p>
  
  <pre><code>for fin in glob.glob('*.txt'):
      with open(fin) as f:
          print pd.concat([series_chunk(chunk) for chunk in lines_per_n(f, 7)], axis=1).T
  
         0               1                                                  2
  0  08988  20131126102946                                                 []
  1  08989  20131126102946  [53, 31, 9, 21, 48, 7, 40, 178, 55, 179, 176, ...
         0               1                                                  2
  0  08988  20131126102946                                                 []
  1  08989  20131126102946  [53, 31, 9, 21, 48, 7, 40, 178, 55, 179, 176, ...
  </code></pre>