title: Python vectorised minimisation (i.e multiple data realisations)
body: <p>Suppose I have a (multivariate) function</p>
  
  <pre><code>f(x,data)
  </code></pre>
  
  <p>that I want to minimise, where <code>x</code> is the parameter vector. Ordinarily in Python I could do something like this:</p>
  
  <pre><code>scipy.optimize.minimize(f, x0, args=my_data)
  </code></pre>
  
  <p>However, suppose I now want to do this repeatedly, for lots of different realisations of <code>my_data</code> (this is for some statistical simulations so I need lots of realisations to study the behaviour of test statistics and so on).</p>
  
  <p>I could do it in a big loop, but this is really slow if I have tens of thousands, up to millions, of data realisations.</p>
  
  <p>So, I am wondering if there is some clever way to vectorise this rather than using a loop. I have one vague idea; say I have <code>N</code> data realisations and <code>p</code> parameters. I could make an enormous combined function with <code>N*p</code> parameters, which accepts a data vector of size <code>N</code>, and finds the minimum of this giant function all at once, where the global minimum of this giant function minimises all the individual functions simultaneously.</p>
  
  <p>However this sounds like a difficult problem for most multivariate minimisers to handle. Yet, the structure of the problem is fairly simple, since each block of <code>p</code> parameters can be minimised entirely independently. So, I wonder, is there an algorithm in <code>scipy</code> or somewhere that can make use of this known dependency structure between the parameters? Or, if not, is there some other smart way to achieve a speedy minimisation of the same function repeatedly?</p>