title: Creating Pandas Dataframe with Generators instead of Lists for performance efficiency
body: <p>This question is related to a former question posted here:</p>
  
  <p><a href="https://stackoverflow.com/questions/61468089/list-comprehensions-with-break/61469873#61469873">list comprehensions with break</a></p>
  
  <p>I want to create a Pandas Dataframe as follows:</p>
  
  <pre><code>   0  1  2   3   4   5   6   7   8   9
  0  4  5  6   9  10  16  21  23  25  27
  1  5  7  7  11  11  17  24  24  26  56
  </code></pre>
  
  <p>And this is the code that I have written so far.</p>
  
  <pre><code>import pandas as pd
  import timeit
  from bisect import bisect_left
  
  list1 = [4, 5, 6, 9, 10, 16, 21, 23, 25, 27]
  list2 = [1, 3, 5, 7, 8, 11, 12, 13, 14, 15, 17, 20, 24, 26, 56]
  
  # List comprehension with []
  list3 = [list2[bisect_left(list2,i+1)] for i in list1]
  print(list3)
  
  # List comprehension with ()
  list3_with_gen = (list2[bisect_left(list2,i+1)] for i in list1)
  print(list3_with_gen)
  
  # Timing the []
  print(timeit.timeit('''
  
  import timeit
  from bisect import bisect_left
  
  list1 = [4, 5, 6, 9, 10, 16, 21, 23, 25, 27]
  list2 = [1, 3, 5, 7, 8, 11, 12, 13, 14, 15, 17, 20, 24, 26, 56]
  
  list3 = [list2[bisect_left(list2,i+1)] for i in list1]
  
  '''))
  
  # Timing the ()
  print(timeit.timeit('''
  
  import timeit
  from bisect import bisect_left
  
  list1 = [4, 5, 6, 9, 10, 16, 21, 23, 25, 27]
  list2 = [1, 3, 5, 7, 8, 11, 12, 13, 14, 15, 17, 20, 24, 26, 56]
  
  list3 = (list2[bisect_left(list2,i+1)] for i in list1)
  
  '''))
  
  df = pd.DataFrame([list1, list3])
  print(df)
  
  
  
  # # original for loops
  # list3 = []
  # for i in list1:
  #     for j in list2:
  #         if j&gt;i:
  #             # print(i,j)
  #             list3.append(j)
  #             break
  # # print(list1)
  # # print(list3)
  </code></pre>
  
  <p>The output of the code is:</p>
  
  <pre><code>[5, 7, 7, 11, 11, 17, 24, 24, 26, 56]
  &lt;generator object &lt;genexpr&gt; at 0x0000016618C46548&gt;
  3.8416419
  1.3952507
     0  1  2   3   4   5   6   7   8   9
  0  4  5  6   9  10  16  21  23  25  27
  1  5  7  7  11  11  17  24  24  26  56
  </code></pre>
  
  <p>What does the code do?</p>
  
  <ul>
  <li><p>It compares the timing of creating list3 with 'List comprehension with []' and list3_with_gen with 'List comprehension with ()'.</p></li>
  <li><p>The timing of 'List comprehension with ()' is about 3x faster.</p></li>
  </ul>
  
  <p>I have some issues fully understanding generators, and it's not because of lack of trying. My objective is to create the dataframe as quickly and efficiently as possible, because this is a small sample and the lists have large dimensions.</p>
  
  <p>Is there a way that I can create that dataframe using the list3_with_gen generator object, since it is about 3x times faster to create?</p>