In statistics and in machine learning, a linear predictor function is a linear function (linear combination) of a set of coefficients and explanatory variables (independent variables), whose value is used to predict the outcome of a dependent variable. This sort of function usually comes in linear regression, where the coefficients are called regression coefficients. However, they also occur in various types of linear classifiers (e.g. logistic regression, perceptrons, support vector machines, and linear discriminant analysis), as well as in various other models, such as principal component analysis and factor analysis. In many of these models, the coefficients are referred to as "weights".
Definition The basic form of a linear predictor function f ( i ) {\displaystyle f(i)} for data point i (consisting of p explanatory variables), for i = 1, ..., n, is
f ( i ) = β 0 + β 1 x i 1 + ⋯ + β p x i p , {\displaystyle f(i)=\beta _{0}+\beta _{1}x_{i1}+\cdots +\beta _{p}x_{ip},}
where x i k {\displaystyle x_{ik}} , for k = 1, ..., p, is the value of the k-th explanatory variable for data point i, and β 0 , … , β p {\displaystyle \beta _{0},\ldots ,\beta _{p}} are the coefficients (regression coefficients, weights, etc.) indicating the relative effect of a particular explanatory variable on the outcome.
Notations It is common to write the predictor function in a more compact form as follows:
The coefficients β0, β1, ..., βp are grouped into a single vector β of size p + 1. For each data point i, an additional explanatory pseudo-variable xi0 is added, with a fixed value of 1, corresponding to the intercept coefficient β0. The resulting explanatory variables xi0(= 1), xi1, ..., xip are then grouped into a single vector xi of size p + 1.
Vector Notation This makes it possible to write the linear predictor function as follows:
f ( i ) = β ⋅ x i {\displaystyle f(i)={\boldsymbol {\beta }}\cdot \mathbf {x} _{i}}
using the notation for a dot product between two vectors.
Matrix Notation An equivalent form using matrix notation is as follows:
f ( i ) = β T x i = x i T β {\displaystyle f(i)={\boldsymbol {\beta }}^{\mathrm {T} }\mathbf {x} _{i}=\mathbf {x} _{i}^{\mathrm {T} }{\boldsymbol {\beta }}}
where β {\displaystyle {\boldsymbol {\beta }}} and x i {\displaystyle \mathbf {x} _{i}} are assumed to be a (p+1)-by-1 column vectors, β T {\displaystyle {\boldsymbol {\beta }}^{\mathrm {T} }} is the matrix transpose of β {\displaystyle {\boldsymbol {\beta }}} (so β T {\displaystyle {\boldsymbol {\beta }}^{\mathrm {T} }} is a 1-by-(p+1) row vector), and β T x i {\displaystyle {\boldsymbol {\beta }}^{\mathrm {T} }\mathbf {x} _{i}} indicates matrix multiplication between the 1-by-(p+1) row vector and the (p+1)-by-1 column vector, producing a 1-by-1 matrix that is taken to be a scalar.
Linear regression An example of the usage of a linear predictor function is in linear regression, where each data point is associated with a continuous outcome yi, and the relationship written
y i = f ( i ) + ε i = β T x i + ε i , {\displaystyle y_{i}=f(i)+\varepsilon _{i}={\boldsymbol {\beta }}^{\mathrm {T} }\mathbf {x} _{i}\ +\varepsilon _{i},}
where ε i {\displaystyle \varepsilon _{i}} is a disturbance term or error variable — an unobserved random variable that adds noise to the linear relationship between the dependent variable and predictor function.
Stacking In some models (standard linear regression, in particular), the equations for each of the data points i = 1, ..., n are stacked together and written in vector form as
… excerpt ends here. Continue reading the full article.
