In statistical decision theory, a minimax estimator δ M {\displaystyle \delta ^{M}\,\!} is an estimator which performs best in the worst possible case allowed in a problem. With problems of estimating a deterministic parameter (vector) θ ∈ Θ {\displaystyle \theta \in \Theta } from observations x ∈ X , {\displaystyle x\in {\mathcal {X}},} an estimator (estimation rule) δ M {\displaystyle \delta ^{M}\,\!} is called minimax if its maximal risk is minimal among all estimators of θ {\displaystyle \theta \,\!} .
Definition Definition : An estimator δ M : X → Θ {\displaystyle \delta ^{M}:{\mathcal {X}}\rightarrow \Theta \,\!} is called minimax with respect to a risk function R ( θ , δ ) {\displaystyle R(\theta ,\delta )\,\!} if it achieves the smallest maximum risk among all estimators, satisfying
sup θ ∈ Θ R ( θ , δ M ) = inf δ sup θ ∈ Θ R ( θ , δ ) . {\displaystyle \sup _{\theta \in \Theta }R(\theta ,\delta ^{M})=\inf _{\delta }\sup _{\theta \in \Theta }R(\theta ,\delta ).\,}
Problem setup An example is the problem of estimating a deterministic (not Bayesian) parameter θ ∈ Θ {\displaystyle \theta \in \Theta } from noisy or corrupt data x ∈ X {\displaystyle x\in {\mathcal {X}}} related through the conditional probability distribution P ( x ∣ θ ) {\displaystyle P(x\mid \theta )\,\!} . The goal is to find a "good" estimator δ ( x ) {\displaystyle \delta (x)\,\!} for estimating the parameter θ {\displaystyle \theta \,\!} , which minimizes some given risk function R ( θ , δ ) {\displaystyle R(\theta ,\delta )\,\!} . The risk function (technically a Functional or Operator since R {\displaystyle R} is a function of a function, not function composition) is the expectation of some loss function L ( θ , δ ) {\displaystyle L(\theta ,\delta )\,\!} with respect to P ( x ∣ θ ) {\displaystyle P(x\mid \theta )\,\!} . A popular example for a loss function is the squared error loss L ( θ , δ ) = ‖ θ − δ ‖ 2 {\displaystyle L(\theta ,\delta )=\|\theta -\delta \|^{2}\,\!} , and the risk function for this loss is the mean squared error (MSE). In general, the risk cannot be minimized because it depends on the unknown parameter θ {\displaystyle \theta \,\!} itself, and if the actual value of θ {\displaystyle \theta \,\!} were known, there would be no need to estimate it. Therefore, additional criteria for finding an optimal estimator in some sense are required. One such criterion is the minimax criterion.
Least favorable distribution Logically, an estimator is minimax when it is the best in the worst case. Continuing this logic, a minimax estimator should be a Bayes estimator with respect to a least favorable prior distribution of θ {\displaystyle \theta \,\!} . To demonstrate this notion denote the average risk of the Bayes estimator δ π {\displaystyle \delta _{\pi }\,\!} with respect to a prior distribution π {\displaystyle \pi \,\!} as
r π = ∫ R ( θ , δ π ) d π ( θ ) {\displaystyle r_{\pi }=\int R(\theta ,\delta _{\pi })\,d\pi (\theta )\,}
… excerpt ends here. Continue reading the full article.
