In statistics, nonlinear regression is a form of regression analysis in which observational data are modeled by a function which is a nonlinear combination of the model parameters and depends on one or more independent variables. The data are fitted by a method of successive approximations (iterations).
General In nonlinear regression, a statistical model of the form,
y ∼ f ( x , β ) {\displaystyle \mathbf {y} \sim f(\mathbf {x} ,{\boldsymbol {\beta }})}
relates a vector of independent variables, x {\displaystyle \mathbf {x} } , and its associated observed dependent variables, y {\displaystyle \mathbf {y} } . The function f {\displaystyle f} is nonlinear in the components of the vector of parameters β {\displaystyle \beta } , but otherwise arbitrary. For example, the Michaelis–Menten model for enzyme kinetics has two parameters and one independent variable, related by f {\displaystyle f} by:
f ( x , β ) = β 1 x β 2 + x {\displaystyle f(x,{\boldsymbol {\beta }})={\frac {\beta _{1}x}{\beta _{2}+x}}}
This function, which is a rectangular hyperbola, is nonlinear because it cannot be expressed as a linear combination of the two β {\displaystyle \beta } s. Systematic error may be present in the independent variables but its treatment is outside the scope of regression analysis. If the independent variables are not error-free, this is an errors-in-variables model, also outside this scope. Other examples of nonlinear functions include exponential functions, logarithmic functions, trigonometric functions, power functions, Gaussian function, and Lorentz distributions. Some functions, such as the exponential or logarithmic functions, can be transformed so that they are linear. When so transformed, standard linear regression can be performed but must be applied with caution. See § Linearization §§ Transformation, below, for more details. In microbiology and biotechnology, nonlinear regression is used to model complex microbial growth kinetics. While simple growth follows monoauxic functions (such as the Gompertz or Boltzmann models), multiphasic (polyauxic) growth is modeled using linear combinations of these nonlinear functions. Estimating parameters for these complex models often requires robust regression techniques (e.g., using a Lorentzian loss function to mitigate outliers) and global optimization algorithms (such as Differential evolution with L-BFGS-B) to avoid local minima and ensure biologically interpretable results. In general, there is no closed-form expression for the best-fitting parameters, as there is in linear regression. Usually numerical optimization algorithms are applied to determine the best-fitting parameters. Again in contrast to linear regression, there may be many local minima of the function to be optimized and even the global minimum may produce a biased estimate. In practice, estimated values of the parameters are used, in conjunction with the optimization algorithm, to attempt to find the global minimum of a sum of squares. For details concerning nonlinear data modeling see least squares and non-linear least squares.
Regression statistics The assumption underlying this procedure is that the model can be approximated by a linear function, namely a first-order Taylor series:
f ( x i , β ) ≈ f ( x i , 0 ) + ∑ j J i j β j {\displaystyle f(x_{i},{\boldsymbol {\beta }})\approx f(x_{i},0)+\sum _{j}J_{ij}\beta _{j}}
where J i j = ∂ f ( x i , β ) ∂ β j {\displaystyle J_{ij}={\frac {\partial f(x_{i},{\boldsymbol {\beta }})}{\partial \beta _{j}}}} are Jacobian matrix elements. It follows from this that the least squares estimators are given by
β ^ ≈ ( J T J ) − 1 J T y , {\displaystyle {\hat {\boldsymbol {\beta }}}\approx \mathbf {(J^{T}J)^{-1}J^{T}y} ,}
… excerpt ends here. Continue reading the full article.
