In statistics, simple linear regression (SLR) is a linear regression model with a single explanatory variable. That is, it concerns two-dimensional sample points with one independent variable and one dependent variable (conventionally, the x and y coordinates in a Cartesian coordinate system) and finds a linear function (a non-vertical straight line) that, as accurately as possible, predicts the dependent variable values as a function of the independent variable. The adjective simple refers to the fact that the outcome variable is related to a single predictor. It is common to make the additional stipulation that the ordinary least squares (OLS) method should be used: the accuracy of each predicted value is measured by its squared residual (vertical distance between the point of the data set and the fitted line), and the goal is to make the sum of these squared deviations as small as possible. In this case, the slope of the fitted line is equal to the correlation between y and x corrected by the ratio of standard deviations of these variables. The intercept of the fitted line is such that the line passes through the center of mass (x, y) of the data points.
Formulation and computation Consider the model function
y = α + β x , {\displaystyle y=\alpha +\beta x,}
which describes a line with slope β and y-intercept α. In general, such a relationship may not hold exactly for the largely unobserved population of values of the independent and dependent variables; we call the unobserved deviations from the above equation the errors. Suppose we observe n data pairs and call them {(xi, yi), i = 1, ..., n}. We can describe the underlying relationship between yi and xi involving this error term εi by
y i = α + β x i + ε i . {\displaystyle y_{i}=\alpha +\beta x_{i}+\varepsilon _{i}.}
This relationship between the true (but unobserved) underlying parameters α and β and the data points is called a linear regression model. The goal is to find estimated values α ^ {\displaystyle {\widehat {\alpha }}} and β ^ {\displaystyle {\widehat {\beta }}} for the parameters α and β which would provide the "best" fit in some sense for the data points. As mentioned in the introduction, in this article the "best" fit will be understood as in the least-squares approach: a line that minimizes the sum of squared residuals (see also Errors and residuals) ε ^ i {\displaystyle {\widehat {\varepsilon }}_{i}} (differences between actual and predicted values of the dependent variable y), each of which is given by, for any candidate parameter values α {\displaystyle \alpha } and β {\displaystyle \beta } ,
ε ^ i = y i − α − β x i . {\displaystyle {\widehat {\varepsilon }}_{i}=y_{i}-\alpha -\beta x_{i}.}
In other words, α ^ {\displaystyle {\widehat {\alpha }}} and β ^ {\displaystyle {\widehat {\beta }}} solve the following minimization problem:
( α ^ , β ^ ) = argmin ( Q ( α , β ) ) , {\displaystyle ({\hat {\alpha }},\,{\hat {\beta }})=\operatorname {argmin} \left(Q(\alpha ,\beta )\right),}
where the objective function Q is:
Q ( α , β ) = ∑ i = 1 n ε ^ i 2 = ∑ i = 1 n ( y i − α − β x i ) 2 . {\displaystyle Q(\alpha ,\beta )=\sum _{i=1}^{n}{\widehat {\varepsilon }}_{i}^{\,2}=\sum _{i=1}^{n}(y_{i}-\alpha -\beta x_{i})^{2}\ .}
… excerpt ends here. Continue reading the full article.
