title: How can I bulk load JSON into Postgres using psycopg2?
body: <p>I've been loading json data from a file like this:</p>
  
  <pre><code>with open("data.json") as jd:
      print("loading json")
      j = json.load(jd)
      print("inserting")
      SendToPostGres(j)
  
  def SendToPostGres(incs):
      length = len(incs)
      processed = 0
      pgParams = {
              'database': 'mydb',
              'user': 'hi',
              'password': '2u',
              'host': 'somedb.com',
              'port': 1111
              }
      conn = psycopg2.connect(**pgParams)
      curs = conn.cursor()
  
      for i in incs:
          curs.execute("insert into MY_TABLE (data) values (%s)", [Json(i)])
          processed += 1
          conn.commit()
          print("%s processed, %s remaining" % (processed, length+1-processed))
  </code></pre>
  
  <p>This is highly inefficient. I've tried googling this and looking at other posts, but I can't seem to get the desired effect of: "For each item in my list of json, create a row in my database with the corresponding data stored as a json type in postgres."</p>
  
  <p>Could someone explain to me the most efficent way to do this in bulk? </p>
  
  <p><strong>UPDATE</strong>:</p>
  
  <p>Per an answer below, I've tried updating to use the execute_values function from extras. The error I'm receiving now is:</p>
  
  <p>"string index out of range"</p>
  
  <p>Note that I tried changing page size, because I thought that might be related. What I tried didn't work. But it might still be an issue.</p>
  
  <pre><code>def SendToPostGres(incs):
      values = []
      for i in incs:
          values.append(json.dumps(i))
  
      pgParams = {
              'database': 'MY_DB',
              'user': 'hi',
              'password': '2u',
              'host': 'somedb.com',
              'port': 5432
              }
      conn = psycopg2.connect(**pgParams)
      curs = conn.cursor()
  
      try:
          psycopg2.extras.execute_values(curs, "insert into incidents (data) values (%s)", values, page_size=len(values))
      except Exception as e:
          raise e
      rows = curs.fetchall()
      curs.close()
  </code></pre>