title: Why scipy.distance.cdist has a big performance difference between using float32 (slower) and float64 (faster)?
body: <p>Why scipy.distance.cdist has a big performance difference between using float32 and float64?</p>
  
  <pre><code>from scipy.spatial import distance
  import numpy as np
  import time
  
  a_float32 = np.empty((1000000, 512), dtype=np.float32)
  b_float32 = np.empty((1, 512), dtype=np.float32)
  a_float64 = np.empty((1000000, 512), dtype=np.float64)
  b_float64 = np.empty((1, 512), dtype=np.float64)
  
  t1 = time.time()
  for i in range(100):
      distance.cdist(a_float32, b_float32, "sqeuclidean")
  t2 = time.time()
  print(t2-t1)
  t1 = time.time()
  for i in range(100):
      distance.cdist(a_float64, b_float64, 'sqeuclidean')
  t2 = time.time()
  print(t2-t1)
  </code></pre>
  
  <p>On my computer, this code produces a timing of 130.6998474597931 for float32 and 22.450339794158936 for float64, a whopping 6x difference. <strong><em>What is the reason for such a big difference with float64 being faster than float32?</em></strong></p>
  
  <p>However the gap seems to become smaller and smaller if instead of b being np.empty((1, 512)), b is np.empty((k, 512)) with k (>1) becoming larger and larger. For example, when k=5, I get 222.25975680351257 for float32 and 110.36117148399353 for float64 (2x). <strong><em>Why the gap becomes smaller when k becomes larger?</em></strong></p>