A statistical model is a mathematical model that embodies a set of statistical assumptions concerning the generation of sample data (and similar data from a larger population). A statistical model represents, often in considerably idealized form, the data-generating process. When referring specifically to probabilities, the corresponding term is probabilistic model. All statistical hypothesis tests and all statistical estimators are derived via statistical models. More generally, statistical models are part of the foundation of statistical inference. A statistical model is usually specified as a mathematical relationship between one or more random variables and other non-random variables. As such, a statistical model is "a formal representation of a theory" (Herman Adèr quoting Kenneth Bollen).
Introduction Informally, a statistical model can be thought of as a statistical assumption (or set of statistical assumptions) with a certain property: that the assumption allows us to calculate the probability of any event. As an example, consider a pair of ordinary six-sided dice. We will study two different statistical assumptions about the dice. The first statistical assumption is this: for each of the dice, the probability of each face (1, 2, 3, 4, 5, and 6) coming up is 1/6. From that assumption, we can calculate the probability of both dice coming up 5: 1/6 × 1/6 = 1/36. More generally, we can calculate the probability of any event: e.g. (1 and 2) or (3 and 3) or (5 and 6). The alternative statistical assumption is this: for each of the dice, the probability of the face 5 coming up is 1/8 (because the dice are weighted). From that assumption, we can calculate the probability of both dice coming up 5: 1/8 × 1/8 = 1/64. We cannot, however, calculate the probability of any other nontrivial event, as the probabilities of the other faces are unknown. The first statistical assumption constitutes a statistical model: because with the assumption alone, we can calculate the probability of any event. The alternative statistical assumption does not constitute a statistical model: because with the assumption alone, we cannot calculate the probability of every event. In the example above, with the first assumption, calculating the probability of an event is easy. With some other examples, though, the calculation can be difficult, or even impractical (e.g. it might require millions of years of computation). For an assumption to constitute a statistical model, such difficulty is acceptable: doing the calculation does not need to be practicable, just theoretically possible.
Formal definition In mathematical terms, a statistical model is a pair ( S , P {\displaystyle S,{\mathcal {P}}} ), where S {\displaystyle S} is the set of possible observations, i.e. the sample space, and P {\displaystyle {\mathcal {P}}} is a set of probability distributions on S {\displaystyle S} . The set P {\displaystyle {\mathcal {P}}} represents all of the models that are considered possible. This set is typically parameterized: P = { F θ : θ ∈ Θ } {\displaystyle {\mathcal {P}}=\{F_{\theta }:\theta \in \Theta \}} . The set Θ {\displaystyle \Theta } defines the parameters of the model. If a parameterization is such that distinct parameter values give rise to distinct distributions, i.e. F θ 1 = F θ 2 ⇒ θ 1 = θ 2 {\displaystyle F_{\theta _{1}}=F_{\theta _{2}}\Rightarrow \theta _{1}=\theta _{2}} (in other words, the mapping is injective), it is said to be identifiable. In some cases, the model can be more complex.
In Bayesian statistics, the model is extended by adding a probability distribution over the parameter space Θ {\displaystyle \Theta } . A statistical model can sometimes distinguish two sets of probability distributions. The first set Q = { F θ : θ ∈ Θ } {\displaystyle {\mathcal {Q}}=\{F_{\theta }:\theta \in \Theta \}} is the set of models considered for inference. The second set P = { F λ : λ ∈ Λ } {\displaystyle {\mathcal {P}}=\{F_{\lambda }:\lambda \in \Lambda \}} is the set of models that could have generated the data which is much larger than Q {\displaystyle {\mathcal {Q}}} . Such statistical models are key in checking that a given procedure is robust, i.e. that it does not produce catastrophic errors when its assumptions about the data are incorrect.
… excerpt ends here. Continue reading the full article.
