In statistics, the jackknife (jackknife cross-validation) is a cross-validation technique and, therefore, a form of resampling. It is especially useful for bias and variance estimation. The jackknife pre-dates other common resampling methods such as the bootstrap. Given a sample of size n {\displaystyle n} , a jackknife estimator can be built by aggregating the parameter estimates from each subsample of size ( n − 1 ) {\displaystyle (n-1)} obtained by omitting one observation. The jackknife is a linear approximation of the bootstrap. The jackknife technique was developed by Maurice Quenouille (1924–1973) from 1949 and refined in 1956. John Tukey expanded on the technique in 1958 and proposed the name "jackknife" because, like a physical jack-knife (a compact folding knife), it is a rough-and-ready tool that can improvise a solution for a variety of problems even though specific problems may be more efficiently solved with a purpose-designed tool.
A simple example: mean estimation The jackknife estimator of a parameter is found by systematically leaving out each observation from a dataset and calculating the parameter estimate over the remaining observations and then aggregating these calculations. For example, if the parameter to be estimated is the population mean of random variable x {\displaystyle x} , then for a given set of i.i.d. observations x 1 , . . . , x n {\displaystyle x_{1},...,x_{n}} the natural estimator is the sample mean:
x ¯ = 1 n ∑ i = 1 n x i = 1 n ∑ i ∈ [ n ] x i , {\displaystyle {\bar {x}}={\frac {1}{n}}\sum _{i=1}^{n}x_{i}={\frac {1}{n}}\sum _{i\in [n]}x_{i},}
where the last sum used another way to indicate that the index i {\displaystyle i} runs over the set [ n ] = { 1 , … , n } {\displaystyle [n]=\{1,\ldots ,n\}} . Then we proceed as follows: For each i ∈ [ n ] {\displaystyle i\in [n]} we compute the mean x ¯ ( i ) {\displaystyle {\bar {x}}_{(i)}} of the jackknife subsample consisting of all but the i {\displaystyle i} -th data point, and this is called the i {\displaystyle i} -th jackknife replicate:
x ¯ ( i ) = 1 n − 1 ∑ j ∈ [ n ] , j ≠ i x j , i = 1 , … , n . {\displaystyle {\bar {x}}_{(i)}={\frac {1}{n-1}}\sum _{j\in [n],j\neq i}x_{j},\qquad i=1,\dots ,n.}
It could help to think that these n {\displaystyle n} jackknife replicates x ¯ ( 1 ) , … , x ¯ ( n ) {\displaystyle {\bar {x}}_{(1)},\ldots ,{\bar {x}}_{(n)}} approximate the distribution of the sample mean x ¯ {\displaystyle {\bar {x}}} . A larger n {\displaystyle n} improves the approximation. Then finally to get the jackknife estimator, the n {\displaystyle n} jackknife replicates are averaged:
x ¯ jack = 1 n ∑ i = 1 n x ¯ ( i ) . {\displaystyle {\bar {x}}_{\text{jack}}={\frac {1}{n}}\sum _{i=1}^{n}{\bar {x}}_{(i)}.}
… excerpt ends here. Continue reading the full article.
