In the unconstrained minimization problem, the Wolfe conditions (also known as the Armijo-Wolfe conditions in some books) are a set of inequalities for performing inexact line search, especially in quasi-Newton methods, first published by Philip Wolfe in 1969 (also named after Larry Armijo). In these methods the idea is to find
min x f ( x ) {\displaystyle \min _{x}f(\mathbf {x} )}
for some smooth f : R n → R {\displaystyle f\colon \mathbb {R} ^{n}\to \mathbb {R} } . Each step often involves approximately solving the subproblem
min α f ( x k + α p k ) {\displaystyle \min _{\alpha }f(\mathbf {x} _{k}+\alpha \mathbf {p} _{k})}
where x k {\displaystyle \mathbf {x} _{k}} is the current best guess, p k ∈ R n {\displaystyle \mathbf {p} _{k}\in \mathbb {R} ^{n}} is a search direction, and α ∈ R {\displaystyle \alpha \in \mathbb {R} } is the step length. The inexact line searches provide an efficient way of computing an acceptable step length α {\displaystyle \alpha } that reduces the objective function 'sufficiently', rather than minimizing the objective function over α ∈ R + {\displaystyle \alpha \in \mathbb {R} ^{+}} exactly. A line search algorithm can use Wolfe conditions as a requirement for any guessed α {\displaystyle \alpha } , before finding a new search direction p k {\displaystyle \mathbf {p} _{k}} .
Armijo rule and curvature A step length α k {\displaystyle \alpha _{k}} is said to satisfy the Wolfe conditions, restricted to the direction p k {\displaystyle \mathbf {p} _{k}} , if the following two inequalities hold:
with 0 < c 1 < c 2 < 1 {\displaystyle 0<c_{1}<c_{2}<1} . (In examining condition (ii), recall that to ensure that p k {\displaystyle \mathbf {p} _{k}} is a descent direction, we have p k T ∇ f ( x k ) < 0 {\displaystyle \mathbf {p} _{k}^{\mathrm {T} }\nabla f(\mathbf {x} _{k})<0} , as in the case of gradient descent, where p k = − ∇ f ( x k ) {\displaystyle \mathbf {p} _{k}=-\nabla f(\mathbf {x} _{k})} , or Newton–Raphson, where p k = − H − 1 ∇ f ( x k ) {\displaystyle \mathbf {p} _{k}=-\mathbf {H} ^{-1}\nabla f(\mathbf {x} _{k})} with H {\displaystyle \mathbf {H} } positive definite.)
… excerpt ends here. Continue reading the full article.
