In mathematical optimization, the method of Lagrange multipliers is a strategy for finding the local maxima and minima of a function subject to equation constraints (i.e., subject to the condition that one or more equations have to be satisfied exactly by the chosen values of the variables). It is named after the mathematician Joseph-Louis Lagrange.
Summary and rationale The basic idea is to convert a constrained problem into a form such that the derivative test of an unconstrained problem can still be applied. The relationship between the gradient of the function and gradients of the constraints rather naturally leads to a reformulation of the original problem, known as the Lagrangian function or Lagrangian. In the general case, the Lagrangian is defined as
L ( x , λ ) ≡ f ( x ) + ⟨ λ , g ( x ) ⟩ {\displaystyle {\mathcal {L}}(x,\lambda )\equiv f(x)+\langle \lambda ,g(x)\rangle }
for functions f , g {\displaystyle f,g} ; the notation ⟨ ⋅ , ⋅ ⟩ {\displaystyle \langle \cdot ,\cdot \rangle } denotes an inner product. The value λ {\displaystyle \lambda } is called the Lagrange multiplier. In simple cases, where the inner product is defined as the dot product, the Lagrangian is
L ( x , λ ) ≡ f ( x ) + λ ⋅ g ( x ) {\displaystyle {\mathcal {L}}(x,\lambda )\equiv f(x)+\lambda \cdot g(x)}
The method can be summarized as follows: in order to find the maximum or minimum of a function f {\displaystyle f} subject to the equality constraint g ( x ) = 0 {\displaystyle g(x)=0} , find the stationary points of L {\displaystyle {\mathcal {L}}} considered as a function of x {\displaystyle x} and the Lagrange multiplier λ {\displaystyle \lambda ~} . This means that all partial derivatives should be zero, including the partial derivative with respect to λ {\displaystyle \lambda ~} .
or equivalently
The solution corresponding to the original constrained optimization is always a saddle point of the Lagrangian function, which can be identified among the stationary points from the definiteness of the bordered Hessian matrix. The great advantage of this method is that it allows the optimization to be solved without explicit parameterization in terms of the constraints. As a result, the method of Lagrange multipliers is widely used to solve challenging constrained optimization problems. Further, the method of Lagrange multipliers is generalized by the Karush–Kuhn–Tucker conditions, which can also take into account inequality constraints of the form h ( x ) ≤ c {\displaystyle h(\mathbf {x} )\leq c} for a given constant c {\displaystyle c} .
Statement The following is known as the Lagrange multiplier theorem. Let f : R n → R {\displaystyle f\colon \mathbb {R} ^{n}\to \mathbb {R} } be the objective function and let g : R n → R c {\displaystyle g\colon \mathbb {R} ^{n}\to \mathbb {R} ^{c}} be the constraints function, both belonging to C 1 {\displaystyle C^{1}} (that is, having continuous first derivatives). Consider the following constrained optimization problem:
maximize f ( x ) subject to: g ( x ) = 0 {\displaystyle {\begin{aligned}&{\text{maximize }}f(x)\\&{\text{subject to: }}g(x)=0\end{aligned}}}
… excerpt ends here. Continue reading the full article.
