Linear model

Key takeaways

Linear model belongs to science; place it in that map before memorising details.
Learn the definition first, then one example that makes the definition concrete.
Connect Linear model to a quantity you can measure, compute or draw — that is where exam questions come from.
Reproduce the core statement of Linear model from memory before moving on to harder problems.

Reference excerpt

In statistics, the term linear model refers to any model which assumes linearity in the system. The most common occurrence is in connection with regression models and the term is often taken as synonymous with linear regression model. However, the term is also used in time series analysis with a different meaning. In each case, the designation "linear" is used to identify a subclass of models for which substantial reduction in the complexity of the related statistical theory is possible.

Linear regression models

For the regression case, the statistical model is as follows. Given a (random) sample ( Y i , X i 1 , … , X i p ) , i = 1 , … , n {\displaystyle (Y_{i},X_{i1},\ldots ,X_{ip}),\,i=1,\ldots ,n} the relation between the observations Y i {\displaystyle Y_{i}} and the independent variables X i j {\displaystyle X_{ij}} is formulated as

Y i = β 0 + β 1 ϕ 1 ( X i 1 ) + ⋯ + β p ϕ p ( X i p ) + ε i i = 1 , … , n {\displaystyle Y_{i}=\beta _{0}+\beta _{1}\phi _{1}(X_{i1})+\cdots +\beta _{p}\phi _{p}(X_{ip})+\varepsilon _{i}\qquad i=1,\ldots ,n}

where ϕ 1 , … , ϕ p {\displaystyle \phi _{1},\ldots ,\phi _{p}} may be nonlinear functions. In the above, the quantities ε i {\displaystyle \varepsilon _{i}} are random variables representing errors in the relationship. The "linear" part of the designation relates to the appearance of the regression coefficients, β j {\displaystyle \beta _{j}} in a linear way in the above relationship. Alternatively, one may say that the predicted values corresponding to the above model, namely

Y ^ i = β 0 + β 1 ϕ 1 ( X i 1 ) + ⋯ + β p ϕ p ( X i p ) ( i = 1 , … , n ) , {\displaystyle {\hat {Y}}_{i}=\beta _{0}+\beta _{1}\phi _{1}(X_{i1})+\cdots +\beta _{p}\phi _{p}(X_{ip})\qquad (i=1,\ldots ,n),}

are linear functions of the β j {\displaystyle \beta _{j}} . Given that estimation is undertaken on the basis of a least squares analysis, estimates of the unknown parameters β j {\displaystyle \beta _{j}} are determined by minimising a sum of squares function

S = ∑ i = 1 n ε i 2 = ∑ i = 1 n ( Y i − β 0 − β 1 ϕ 1 ( X i 1 ) − ⋯ − β p ϕ p ( X i p ) ) 2 . {\displaystyle S=\sum _{i=1}^{n}\varepsilon _{i}^{2}=\sum _{i=1}^{n}\left(Y_{i}-\beta _{0}-\beta _{1}\phi _{1}(X_{i1})-\cdots -\beta _{p}\phi _{p}(X_{ip})\right)^{2}.}

… excerpt ends here. Continue reading the full article.

Worked examples

Example 1 — a first encounter with Linear model

Start with the simplest possible case. Write down what Linear model claims or describes in one sentence, then invent the smallest concrete situation in which that sentence is true. In science, the smallest case is usually a single object, a single equation or a single measurement. Check that every symbol or term in your sentence has a meaning in that case.

Example 2 — changing one variable

Take the situation from Example 1 and change exactly one quantity: double it, halve it, or set it to zero. Predict what should happen to Linear model before you calculate. Comparing your prediction with the result is the fastest way to find out whether you understand the idea or only the words.

Example 3 — an exam-style question

Typical questions about Linear model ask you to (a) state it precisely, (b) apply it to given data, and (c) explain a limitation. Practise writing all three answers in under five minutes; the third part is what separates a full-mark answer from an average one.

Applications of Linear model

In research: Linear model appears in science research whenever the underlying quantities have to be modelled precisely. Papers usually cite it as a starting assumption and then explore where it breaks down.
In technology and industry: Engineering practice reuses Linear model in design rules, simulations and safety margins. Knowing the idea lets you read a specification sheet and understand why the numbers look the way they do.
In the classroom: Linear model is common in secondary-school and first-year university syllabi. It links to neighbouring topics Curve fitting, Regression models, so understanding it makes those chapters shorter.
In everyday life: Look for Linear model outside the textbook — in sport, cooking, traffic, electronics or the sky above you. An example you found yourself is remembered far longer than one you were given.

How to study Linear model in 20 minutes

Read the reference excerpt below once, without taking notes.
Close the page and write down what Linear model means in your own words.
Compare your version with the excerpt and mark what you missed.
Work through the three examples above with pen and paper.
Explain Linear model out loud to somebody else — or to Teacher Smith in the lgStudy chat.

Frequently asked questions

What is Linear model in simple terms?

In statistics, the term linear model refers to any model which assumes linearity in the system. The most common occurrence is in connection with regression models and the term is often taken as synonymous with linear regression model.

Why does Linear model matter?

Because it connects several science ideas at once: it gives you a definition you can apply, a quantity you can calculate, and a way to check whether a result is plausible.

How should I study Linear model?

Read the excerpt, restate it from memory, then work through the examples and applications listed on this page. The five-step study plan above takes about twenty minutes.

What does this page cover?

It gives you a compact reference excerpt plus original lgStudy explanations, examples, applications and study material on Linear model.