Smoothing splines are function estimates, f ^ ( x ) {\displaystyle {\hat {f}}(x)} , obtained from a set of noisy observations y i {\displaystyle y_{i}} of the target f ( x i ) {\displaystyle f(x_{i})} , in order to balance a measure of goodness of fit of f ^ ( x i ) {\displaystyle {\hat {f}}(x_{i})} to y i {\displaystyle y_{i}} with a derivative based measure of the smoothness of f ^ ( x ) {\displaystyle {\hat {f}}(x)} . They provide a means for smoothing noisy x i , y i {\displaystyle x_{i},y_{i}} data. The most familiar example is the cubic smoothing spline, but there are many other possibilities, including for the case where x {\displaystyle x} is a vector quantity.
Cubic spline definition Let { x i , Y i : i = 1 , … , n } {\displaystyle \{x_{i},Y_{i}:i=1,\dots ,n\}} be a set of observations, modeled by the relation Y i = f ( x i ) + ϵ i {\displaystyle Y_{i}=f(x_{i})+\epsilon _{i}} where the ϵ i {\displaystyle \epsilon _{i}} are independent, zero mean random variables. The cubic smoothing spline estimate f ^ {\displaystyle {\hat {f}}} of the function f {\displaystyle f} is defined to be the unique minimizer, in the Sobolev space W 2 2 {\displaystyle W_{2}^{2}} on a compact interval, of
∑ i = 1 n { Y i − f ^ ( x i ) } 2 + λ ∫ f ^ ′ ′ ( x ) 2 d x . {\displaystyle \sum _{i=1}^{n}\{Y_{i}-{\hat {f}}(x_{i})\}^{2}+\lambda \int {\hat {f}}^{\prime \prime }(x)^{2}\,dx.}
Remarks:
λ ≥ 0 {\displaystyle \lambda \geq 0} is a smoothing parameter, controlling the trade-off between fidelity to the data and roughness of the function estimate. This is often estimated by generalized cross-validation, or by restricted marginal likelihood (REML) which exploits the link between spline smoothing and Bayesian estimation (the smoothing penalty can be viewed as being induced by a prior on the f {\displaystyle f} ). The integral is often evaluated over the whole real line although it is also possible to restrict the range to that of x i {\displaystyle x_{i}} . As λ → 0 {\displaystyle \lambda \to 0} (no smoothing), the smoothing spline converges to the interpolating spline. As λ → ∞ {\displaystyle \lambda \to \infty } (infinite smoothing), the roughness penalty becomes paramount and the estimate converges to a linear least squares estimate. The roughness penalty based on the second derivative is the most common in modern statistics literature, although the method can easily be adapted to penalties based on other derivatives. In early literature, with equally-spaced ordered x i {\displaystyle x_{i}} , second or third-order differences were used in the penalty, rather than derivatives. See also Whittaker–Henderson smoothing. The penalized sum of squares smoothing objective can be replaced by a penalized likelihood objective in which the sum of squares terms is replaced by another log-likelihood based measure of fidelity to the data. The sum of squares term corresponds to penalized likelihood with a Gaussian assumption on the ϵ i {\displaystyle \epsilon _{i}} .
Derivation of the cubic smoothing spline It is useful to think of fitting a smoothing spline in two steps:
… excerpt ends here. Continue reading the full article.
