In statistical theory, the field of high-dimensional statistics studies data whose dimension is larger (relative to the number of datapoints) than typically considered in classical multivariate analysis. The area arose owing to the emergence of many modern data sets in which the dimension of the data vectors may be comparable to, or even larger than, the sample size, so that justification for the use of traditional techniques, often based on asymptotic arguments with the dimension held fixed as the sample size increased, was lacking. There are several notions of high-dimensional analysis of statistical methods including:
Non-asymptotic results which apply for finite n , p {\displaystyle n,p} (number of data points and dimension size, respectively). Kolmogorov asymptotics which studies the asymptotic behavior where the ratio n / p {\displaystyle n/p} is converges to a specific finite value.
Examples
Parameter estimation in linear models
The most basic statistical model for the relationship between a covariate vector x ∈ R p {\displaystyle x\in \mathbb {R} ^{p}} and a response variable y ∈ R {\displaystyle y\in \mathbb {R} } is the linear model
y = x ⊤ β + ϵ , {\displaystyle y=x^{\top }\beta +\epsilon ,}
where β ∈ R p {\displaystyle \beta \in \mathbb {R} ^{p}} is an unknown parameter vector, and ϵ {\displaystyle \epsilon } is random noise with mean zero and variance σ 2 {\displaystyle \sigma ^{2}} . Given independent responses Y 1 , … , Y n {\displaystyle Y_{1},\ldots ,Y_{n}} , with corresponding covariates x 1 , … , x n {\displaystyle x_{1},\ldots ,x_{n}} , from this model, we can form the response vector Y = ( Y 1 , … , Y n ) ⊤ {\displaystyle Y=(Y_{1},\ldots ,Y_{n})^{\top }} , and design matrix X = ( x 1 , … , x n ) ⊤ ∈ R n × p {\displaystyle X=(x_{1},\ldots ,x_{n})^{\top }\in \mathbb {R} ^{n\times p}} . When n ≥ p {\displaystyle n\geq p} and the design matrix has full column rank (i.e. its columns are linearly independent), the ordinary least squares estimator of β {\displaystyle \beta } is
β ^ := ( X ⊤ X ) − 1 X ⊤ Y . {\displaystyle {\hat {\beta }}:=(X^{\top }X)^{-1}X^{\top }Y.}
… excerpt ends here. Continue reading the full article.
