TITLE: Lagrange Multiplier do not make sense
QUESTION [0 upvotes]: Basically the core of Lagrange's multiplier says that the solution to a constrained optimization occurs when the contour line of the function being maximized/minimized is tangential to the constraint curve.
I am not able to convince myself of above statement.
Consider below diagram wherein 'f' is some function being maximized/minimized and g(x,y) is the constraint. the contour lines of f and g(x,y) are plotted.
Clearly the contour line which corresponds to local maximum of f is not tangential to g i.e. the gradient vector of f and g don't align at the intersections of f=5 and g.
I am not looking for a rigorous proof or something, just want to get a basic understanding of why gradient of f and g should point in same direction at the local maxima/minima of f?

REPLY [2 votes]: What Lagrange multiplier says is somewhat more general, it says that in a local maximum the gradient of the function being optimized is collinear with the gradient of the function defining the constraint.
And, if neither of the gradients vanishes then this is indeed equivalent to your formulation: tangentiality of the contours. But, if the gradient of the function is zero somewhere then Lagrange criterion is also satisfied in that point (we consider the $0$ vector collinear with any vector, as it is a multiple of any vector).
If the function $f$ described by your contour lines is differentiable then its gradient must be zero somewhere on that circle. Say, if $5$ is its actual maximum then $\text{grad } f$ vanishes along the whole $f=5$ line.
{"set_name": "stack_exchange", "score": 0, "question_id": 2973219}