mathematics

Statistical theory

Statistical theory is a mathematics topic covered in the lgStudy science library. This page brings together a partial reference excerpt, illustrations, worked examples, real-world applications and a short study plan, so you can understand Statistical theory rather than just read about it. In short: The theory of statistics provides a basis for the whole range of techniques, in both study design and data analysis, that are used within applications of statistics. The theory covers approaches to statistical-decision problems and to statistical inference, and the actions and deductions that satisfy the basic principles stated for these different approaches.

Key takeaways

Statistical theory belongs to mathematics; place it in that map before memorising details.
Learn the definition first, then one example that makes the definition concrete.
Connect Statistical theory to a quantity you can measure, compute or draw — that is where exam questions come from.
Reproduce the core statement of Statistical theory from memory before moving on to harder problems.

Reference excerpt

The theory of statistics provides a basis for the whole range of techniques, in both study design and data analysis, that are used within applications of statistics. The theory covers approaches to statistical-decision problems and to statistical inference, and the actions and deductions that satisfy the basic principles stated for these different approaches. Within a given approach, statistical theory gives ways of comparing statistical procedures; it can find the best possible procedure within a given context for given statistical problems, or can provide guidance on the choice between alternative procedures. Apart from philosophical considerations about how to make statistical inferences and decisions, much of statistical theory consists of mathematical statistics, and is closely linked to probability theory, to utility theory, and to optimization.

Scope Statistical theory provides an underlying rationale and provides a consistent basis for the choice of methodology used in applied statistics.

Modelling Statistical models describe the sources of data and can have different types of formulation corresponding to these sources and to the problem being studied. Such problems can be of various kinds:

Sampling from a finite population Measuring observational error and refining procedures Studying statistical relations Statistical models, once specified, can be tested to see whether they provide useful inferences for new data sets.

Data collection Statistical theory provides a guide to comparing methods of data collection, where the problem is to generate informative data using optimization and randomization while measuring and controlling for observational error. Optimization of data collection reduces the cost of data while satisfying statistical goals, while randomization allows reliable inferences. Statistical theory provides a basis for good data collection and the structuring of investigations in the topics of:

Design of experiments to estimate treatment effects, to test hypotheses, and to optimize responses. Survey sampling to describe populations

Summarising data The task of summarising statistical data in conventional forms (also known as descriptive statistics) is considered in theoretical statistics as a problem of defining what aspects of statistical samples need to be described and how well they can be described from a typically limited sample of data. Thus the problems theoretical statistics considers include:

Choosing summary statistics to describe a sample Summarising probability distributions of sample data while making limited assumptions about the form of distribution that may be met Summarising the relationships between different quantities measured on the same items with a sample

Interpreting data Besides the philosophy underlying statistical inference, statistical theory has the task of considering the types of questions that data analysts might want to ask about the problems they are studying and of providing data analytic techniques for answering them. Some of these tasks are:

Summarising populations in the form of a fitted distribution or probability density function Summarising the relationship between variables using some type of regression analysis Providing ways of predicting the outcome of a random quantity given other related variables Examining the possibility of reducing the number of variables being considered within a problem (the task of Dimension reduction) When a statistical procedure has been specified in the study protocol, then statistical theory provides well-defined probability statements for the method when applied to all populations that could have arisen from the randomization used to generate the data. This provides an objective way of estimating parameters, estimating confidence intervals, testing hypotheses, and selecting the best. Even for observational data, statistical theory provides a way of calculating a value that can be used to interpret a sample of data from a population, it can provide a means of indicating how well that value is determined by the sample, and thus a means of saying corresponding values derived for different populations are as different as they might seem; however, the reliability of inferences from post-hoc observational data is often worse than for planned randomized generation of data.

Applied statistical inference Statistical theory provides the basis for a number of data-analytic approaches that are common across scientific and social research. Interpreting data is done with one of the following approaches:

Estimating parameters Providing a range of values instead of a point estimate Testing statistical hypotheses Many of the standard methods for those approaches rely on certain statistical assumptions (made in the derivation of the methodology) actually holding in practice. Statistical theory studies the consequences of departures from these assumptions. In addition it provides a range of robust statistical techniques that are less dependent on assumptions, and it provides methods checking whether particular assumptions are reasonable for a given data set.

See also List of statistical topics Foundations of statistics

References

Citations

Sources

Further reading Peirce, C. S. (1876), "Note on the Theory of the Economy of Research" in Coast Survey Report, pp. 197–201 (Appendix No. 14), NOAA PDF Eprint. Reprinted 1958 in Collected Papers of Charles Sanders Peirce 7, paragraphs 139–157 and in 1967 in Operations Research 15 (4): pp. 643–648, Abstract from JSTOR. (1967) Peirce, C. S. (1967). "Note on the Theory of the Economy of Research". Operations Research. 15 (4): 643–648. doi:10.1287/opre.15.4.643. (1877–1878), "Illustrations of the Logic of Science" (1883), "A Theory of Probable Inference" and Jastrow, Joseph (1885), "On Small Differences in Sensation" in Memoirs of the National Academy of Sciences 3: pp. 73–83. Eprint. Bickel, Peter J. & Doksum, Kjell A. (2001). Mathematical Statistics: Basic and Selected Topics. Vol. I (Second (updated printing 2007) ed.). Pearson Prentice-Hall. ISBN 0-13-850363-X. Davison, A.C. (2003) Statistical Models. Cambridge University Press. ISBN 0-521-77339-3 Lehmann, Erich (1983). Theory of Point Estimation. Liese, Friedrich & Miescke, Klaus-J. (2008). Statistical Decision Theory: Estimation, Testing, and Selection. Springer. ISBN 978-0-387-73193-3.

External links Media related to Statistical theory at Wikimedia Commons

Worked examples

Example 1 — a first encounter with Statistical theory

Start with the simplest possible case. Write down what Statistical theory claims or describes in one sentence, then invent the smallest concrete situation in which that sentence is true. In mathematics, the smallest case is usually a single object, a single equation or a single measurement. Check that every symbol or term in your sentence has a meaning in that case.

Example 2 — changing one variable

Take the situation from Example 1 and change exactly one quantity: double it, halve it, or set it to zero. Predict what should happen to Statistical theory before you calculate. Comparing your prediction with the result is the fastest way to find out whether you understand the idea or only the words.

Example 3 — an exam-style question

Typical questions about Statistical theory ask you to (a) state it precisely, (b) apply it to given data, and (c) explain a limitation. Practise writing all three answers in under five minutes; the third part is what separates a full-mark answer from an average one.

Applications of Statistical theory

In research: Statistical theory appears in mathematics research whenever the underlying quantities have to be modelled precisely. Papers usually cite it as a starting assumption and then explore where it breaks down.
In technology and industry: Engineering practice reuses Statistical theory in design rules, simulations and safety margins. Knowing the idea lets you read a specification sheet and understand why the numbers look the way they do.
In the classroom: Statistical theory is common in secondary-school and first-year university syllabi. It links to neighbouring topics Statistical theory, so understanding it makes those chapters shorter.
In everyday life: Look for Statistical theory outside the textbook — in sport, cooking, traffic, electronics or the sky above you. An example you found yourself is remembered far longer than one you were given.

Affiliate

Preply — study more efficiently by working with a personal tutor. 50% off.

How to study Statistical theory in 20 minutes

Read the reference excerpt below once, without taking notes.
Close the page and write down what Statistical theory means in your own words.
Compare your version with the excerpt and mark what you missed.
Work through the three examples above with pen and paper.
Explain Statistical theory out loud to somebody else — or to Teacher Smith in the lgStudy chat.

Frequently asked questions

What is Statistical theory in simple terms?

Why does Statistical theory matter?

Because it connects several mathematics ideas at once: it gives you a definition you can apply, a quantity you can calculate, and a way to check whether a result is plausible.

How should I study Statistical theory?

Read the excerpt, restate it from memory, then work through the examples and applications listed on this page. The five-step study plan above takes about twenty minutes.

What does this page cover?

It gives you a compact reference excerpt plus original lgStudy explanations, examples, applications and study material on Statistical theory.