title: MATLABs 'fminsearch' different from Octave's 'fmincg'
body: <p>I am trying to get consistent answers for a simple optimization problem, between two functions in MATLAB and Octave. Here is my code:</p>
  
  <pre><code>  options = optimset('MaxIter', 500 , 'Display', 'iter', 'MaxFunEvals', 1000);
  
    objFunc = @(t) lrCostFunction(t,X,y);
  
    [result1] = fminsearch(objFunc, theta, options);
    [result2]=  fmincg (objFunc, theta, options);
  </code></pre>
  
  <p>(Bear in mind, that X, y, and theta are defined earlier and are correct). The problem is the following: When I run the above code in MATLAB with it using fmincg, (commend out fminsearch), I get the correct answer. </p>
  
  <p>However, if I comment out fmincg and let us run fminsearch, I get no conversion whatsoever.   In fact the output looks like this:</p>
  
  <pre><code>   491          893         0.692991         reflect
     492          894         0.692991         reflect
     493          895         0.692991         reflect
     494          896         0.692991         reflect
     495          897         0.692991         reflect
     496          898         0.692991         reflect
     497          899         0.692991         reflect
     498          900         0.692991         reflect
     499          901         0.692991         reflect
     500          902         0.692991         reflect
  
  
  
  Exiting: Maximum number of iterations has been exceeded
           - increase MaxIter option.
           Current function value: 0.692991 
  </code></pre>
  
  <p>Increasing the number of iterations doesnt do jack. In contrast, when using the fmincg, I see it converging, and it finally gives me the correct result:</p>
  
  <pre><code>Iteration     1 | Cost: 2.802128e-001
  Iteration     2 | Cost: 9.454389e-002
  Iteration     3 | Cost: 5.704641e-002
  Iteration     4 | Cost: 4.688190e-002
  Iteration     5 | Cost: 3.759021e-002
  Iteration     6 | Cost: 3.522008e-002
  Iteration     7 | Cost: 3.234531e-002
  Iteration     8 | Cost: 3.145034e-002
  Iteration     9 | Cost: 3.008919e-002
  Iteration    10 | Cost: 2.994639e-002
  Iteration    11 | Cost: 2.678528e-002
  Iteration    12 | Cost: 2.660323e-002
  Iteration    13 | Cost: 2.493301e-002
  
  .
  .
  .
  
  
  Iteration   493 | Cost: 1.311466e-002
  Iteration   494 | Cost: 1.311466e-002
  Iteration   495 | Cost: 1.311466e-002
  Iteration   496 | Cost: 1.311466e-002
  Iteration   497 | Cost: 1.311466e-002
  Iteration   498 | Cost: 1.311466e-002
  Iteration   499 | Cost: 1.311466e-002
  Iteration   500 | Cost: 1.311466e-002
  </code></pre>
  
  <p>This gives the correct asnwer.</p>
  
  <p>So what gives? Why is fminsearch not working in this minimization case? </p>
  
  <p>Additional context:</p>
  
  <p>1)  Octave is the language that has fmincg btw, however a quick google result also retrieves this function. My MATLAB can call either.</p>
  
  <p>2) My problem has a convex error surface, and its error surface is everywhere differentiable.</p>
  
  <p>3) I only have access to fminsearch, fminbnd (which I cant use since this problem is multivariate not univariate), so that leaves fminsearch.
  Thanks!</p>