The L-curve is a graphical tool used in the numerical treatment of linear inverse problems. It displays how the size of a regularized solution varies relative to the size of its residual as a function of a regularization parameter. When the points are plotted in logarithmic scale, the curve typically has a characteristic "L" shape. The flat part of the curve corresponds to heavy regularization (small solution norm but large residual), while the steep part corresponds to light regularization (small residual but large solution norm). The corner of this curve often marks a compromise between fidelity to the data and stability of the solution and can therefore guide the choice of an appropriate regularization parameter..
Definition Consider a linear inverse problem of the form A x = b {\displaystyle \mathbf {A} \mathbf {x} =\mathbf {b} } , where A ∈ R m × n {\displaystyle \mathbf {A} \in \mathbb {R} ^{m\times n}} is an ill-conditioned or rank-deficient matrix and b ∈ R n {\displaystyle \mathbf {b} \in \mathbb {R} ^{n}} is contaminated by noise. In Tikhonov regularization and related methods the solution depends on a scalar parameter λ {\displaystyle \lambda } that controls the amount of regularization. Let x λ {\displaystyle \mathbf {x} _{\lambda }} be the regularized solution (for example, the minimizer of ‖ A x − b ‖ 2 + ‖ L ( x − x 0 ) ‖ 2 {\displaystyle \left\|\mathbf {A} \mathbf {x} -\mathbf {b} \right\|^{2}+\left\|L(\mathbf {x} -\mathbf {x} _{0})\right\|^{2}} ). The L-curve is the set of points
L = { ( log ( ‖ A x λ − b ‖ ) , log ( ‖ x λ ‖ ) ) ∣ λ > 0 } {\displaystyle {\mathcal {L}}=\left\{\left(\log \left(\left\|\mathbf {A} \mathbf {x} _{\lambda }-\mathbf {b} \right\|\right),\log \left(\left\|\mathbf {x} _{\lambda }\right\|\right)\right)\mid \lambda >0\right\}}
that is, the logarithm of the residual norm plotted against the logarithm of the solution norm. For many problems this curve is shaped like the letter "L", giving the tool its name. Early papers in numerical analysis used similar trade-off plots; Lawson and Hanson and later Miller were among the first to employ such curves.
Interpretation and properties For a typical discrete inverse problem, the L-curve has two distinct segments. When the regularization parameter is large, the regularization term dominates and the solution norm ‖ x λ ‖ {\displaystyle \|\mathbf {x} _{\lambda }\|} remains essentially constant, while the residual ‖ A x λ − b ‖ {\displaystyle \|\mathbf {A} \mathbf {x} _{\lambda }-\mathbf {b} \|} grows; this appears as a nearly horizontal branch of the curve. When λ {\displaystyle \lambda } is small, the residual becomes small but the solution norm increases rapidly, giving a nearly vertical branch. The corner separating these two regimes highlights where the influence of the regularization errors and perturbation errors is roughly balanced. In log-log coordinates the curve is insensitive to simple scalings of A {\displaystyle \mathbf {A} } and b {\displaystyle \mathbf {b} } . The L-curve can also be characterized variationally: any point ( δ , η ) {\displaystyle (\delta ,\eta )} on the curve corresponds to solutions of two inequality-constrained least squares problems: the minimal residual subject to a bound on the solution norm and the minimal solution norm subject to a bound on the residual norm. Hansen has shown that for standard Tikhonov regularization the semi-norm ‖ L x λ ‖ {\displaystyle \|L\mathbf {x} _{\lambda }\|} is a monotonically decreasing convex function of the residual norm.
Corner and curvature Hansen and O'Leary suggested choosing the regularization parameter corresponding to the vertex (or corner) of the L-curve. A quantitative way to locate the corner is to compute the curvature of the log-log plot. Let η = ‖ x λ ‖ 2 {\displaystyle \eta =\|\mathbf {x} _{\lambda }\|^{2}} and ρ = ‖ A x λ − b ‖ 2 {\displaystyle \rho =\|\mathbf {A} \mathbf {x} _{\lambda }-\mathbf {b} \|^{2}} . Writing η ^ = log η {\displaystyle {\hat {\eta }}=\log \eta } and ρ ^ = log ρ {\displaystyle {\hat {\rho }}=\log \rho } , the curvature κ λ {\displaystyle \kappa _{\lambda }} of the L-curve at λ {\displaystyle \lambda } is given by
κ λ = 2 ρ ^ ′ η ^ − 2 ρ ^ η ^ ′ ( ( ρ ^ ′ ) 2 + ( η ^ ′ ) 2 ) 3 / 2 , {\displaystyle \kappa _{\lambda }={\frac {2\,{\hat {\rho }}'{\hat {\eta }}-2\,{\hat {\rho }}{\hat {\eta }}'}{{\bigl (}({\hat {\rho }}')^{2}+({\hat {\eta }}')^{2}{\bigr )}^{3/2}}},}
where primes denote differentiation with respect to λ {\displaystyle \lambda } . The point of maximum curvature is often used as an estimate of the corner and yields a regularization parameter λ L {\displaystyle \lambda _{L}} . Computing this curvature directly can be expensive for large matrices; Calvetti, Hansen and Reichel therefore proposed approximating the L-curve and its curvature using Lanczos bidiagonalization, forming so-called L-ribbons and curvature-ribbons that bound the true curve.
Parameter-choice criterion The L-curve criterion is a heuristic method for selecting the regularization parameter. It chooses the parameter λ L {\displaystyle \lambda _{L}} corresponding to the point on the L-curve with the largest curvature. This choice aims to balance the amplification of data noise (which dominates for small λ {\displaystyle \lambda } ) against the loss of resolution (which dominates for large λ {\displaystyle \lambda } ). The L-curve criterion has been used for Tikhonov regularization, truncated singular value decomposition and other regularization schemes. It can also be adapted to iterative methods such as conjugate gradient on the normal equations by treating the iteration index as a discrete regularization parameter.
Applications The L-curve is widely employed in inverse problems arising in fields such as tomography, geoscience and deconvolution. It provides a visual diagnostic that helps practitioners detect under- or over-regularization and to compare different regularization techniques. The method has been incorporated into software packages for solving discrete ill-posed problems.
Limitations Although popular, the L-curve criterion is heuristic and has known shortcomings. Hanke constructed an infinite-dimensional example where the parameter selected by the L-curve criterion tends to zero too rapidly as the noise level decreases, causing the regularized solutions to diverge from the true solution. In practice the criterion tends to produce slightly over-smoothed solutions; however, it is considered more robust than generalized cross-validation because it rarely yields very large errors. Hansen's studies also show that the L-curve may fail to reveal an appropriate corner when the exact solution is very smooth, in which case the curve does not exhibit a pronounced L-shape.
History Trade-off curves plotting solution size against residual size appeared in numerical analysis literature decades before the term "L-curve" was popularised. Lawson and Hanson's work on least squares problems and Miller's study of regularization employed similar plots. Tikhonov’s 1963 paper introduced what is now called Tikhonov regularization for solving ill-posed problems, and Hansen and O'Leary systematically analysed the use of the L-curve for parameter selection in the early 1990s. Subsequent research has examined its properties, developed efficient methods for computing curvature, and proposed alternatives such as the L-ribbon and curvature-ribbon.
See also Elbow method (clustering) - another heuristic that identifies a "knee" in a plot Heuristic Knee of a curve
References