title: Why does pickle take so much longer than np.save?
body: <p>I want to save a <code>dict</code> or arrays.</p>
  
  <p>I try both with <code>np.save</code> and with <code>pickle</code> and see that the former always take much less time.</p>
  
  <p>My actual data is much bigger but I just present a small piece here for demonstration purposes:</p>
  
  <pre><code>import numpy as np
  #import numpy.array as array
  import time
  import pickle
  
  b = {0: [np.array([0, 0, 0, 0])], 1: [np.array([1, 0, 0, 0]), np.array([0, 1, 0, 0]), np.array([0, 0, 1, 0]), np.array([0, 0, 0, 1]), np.array([-1,  0,  0,  0]), np.array([ 0, -1,  0,  0]), np.array([ 0,  0, -1,  0]), np.array([ 0,  0,  0, -1])], 2: [np.array([2, 0, 0, 0]), np.array([1, 1, 0, 0]), np.array([1, 0, 1, 0]), np.array([1, 0, 0, 1]), np.array([ 1, -1,  0,  0]), np.array([ 1,  0, -1,  0]), np.array([ 1,  0,  0, -1])], 3: [np.array([1, 0, 0, 0]), np.array([0, 1, 0, 0]), np.array([0, 0, 1, 0]), np.array([0, 0, 0, 1]), np.array([-1,  0,  0,  0]), np.array([ 0, -1,  0,  0]), np.array([ 0,  0, -1,  0]), np.array([ 0,  0,  0, -1])], 4: [np.array([2, 0, 0, 0]), np.array([1, 1, 0, 0]), np.array([1, 0, 1, 0]), np.array([1, 0, 0, 1]), np.array([ 1, -1,  0,  0]), np.array([ 1,  0, -1,  0]), np.array([ 1,  0,  0, -1])], 5: [np.array([0, 0, 0, 0])], 6: [np.array([1, 0, 0, 0]), np.array([0, 1, 0, 0]), np.array([0, 0, 1, 0]), np.array([0, 0, 0, 1]), np.array([-1,  0,  0,  0]), np.array([ 0, -1,  0,  0]), np.array([ 0,  0, -1,  0]), np.array([ 0,  0,  0, -1])], 2: [np.array([2, 0, 0, 0]), np.array([1, 1, 0, 0]), np.array([1, 0, 1, 0]), np.array([1, 0, 0, 1]), np.array([ 1, -1,  0,  0]), np.array([ 1,  0, -1,  0]), np.array([ 1,  0,  0, -1])], 7: [np.array([1, 0, 0, 0]), np.array([0, 1, 0, 0]), np.array([0, 0, 1, 0]), np.array([0, 0, 0, 1]), np.array([-1,  0,  0,  0]), np.array([ 0, -1,  0,  0]), np.array([ 0,  0, -1,  0]), np.array([ 0,  0,  0, -1])], 8: [np.array([2, 0, 0, 0]), np.array([1, 1, 0, 0]), np.array([1, 0, 1, 0]), np.array([1, 0, 0, 1]), np.array([ 1, -1,  0,  0]), np.array([ 1,  0, -1,  0]), np.array([ 1,  0,  0, -1])]}
  
  
  start_time = time.time()
  with open('testpickle', 'wb') as myfile:
      pickle.dump(b, myfile)
  print("--- Time to save with pickle: %s milliseconds ---" % (1000*time.time() - 1000*start_time))
  
  start_time = time.time()
  np.save('numpy', b)
  print("--- Time to save with numpy: %s milliseconds ---" % (1000*time.time() - 1000*start_time))
  
  start_time = time.time()
  with open('testpickle', 'rb') as myfile:
      g1 = pickle.load(myfile)
  print("--- Time to load with pickle: %s milliseconds ---" % (1000*time.time() - 1000*start_time))
  
  start_time = time.time()
  g2 = np.load('numpy.npy')
  print("--- Time to load with numpy: %s milliseconds ---" % (1000*time.time() - 1000*start_time))
  </code></pre>
  
  <p>which gives an output:</p>
  
  <pre><code>--- Time to save with pickle: 4.0 milliseconds ---
  --- Time to save with numpy: 1.0 milliseconds ---
  --- Time to load with pickle: 2.0 milliseconds ---
  --- Time to load with numpy: 1.0 milliseconds ---
  </code></pre>
  
  <p>The time difference is even more pronounced with my actual size (~100,000 keys in the dict).</p>
  
  <p>Why does pickle take longer than np.save, both for saving and for loading?</p>
  
  <p>When should I use <code>pickle</code>?</p>