In statistics, the Kolmogorov–Smirnov test (also K–S test or KS test) is a nonparametric test of the equality of continuous (or discontinuous, see Section 2.2), one-dimensional probability distributions. It can be used to test whether a sample came from a given reference probability distribution (one-sample K–S test), or to test whether or not two samples came from the same distribution (two-sample K–S test). It is named after Andrey Kolmogorov and Nikolai Smirnov, who developed it in the 1930s. The Kolmogorov–Smirnov statistic quantifies a distance between the empirical distribution function of the sample and the cumulative distribution function of the reference distribution, or between the empirical distribution functions of two samples. The null distribution of this statistic is calculated under the null hypothesis that the sample is drawn from the reference distribution (in the one-sample case) or that the samples are drawn from the same distribution (in the two-sample case). In the one-sample case, the distribution considered under the null hypothesis may be continuous (see Section 2), purely discrete or mixed (see Section 2.2). In the two-sample case (see Section 3), the distribution considered under the null hypothesis is a continuous distribution but is otherwise unrestricted. The two-sample K–S test is one of the most useful and general nonparametric methods for comparing two samples, as it is sensitive to differences in both location and shape of the empirical cumulative distribution functions of the two samples. The Kolmogorov–Smirnov test can be modified to serve as a goodness of fit test. In the special case of testing for normality of the distribution, samples are standardized and compared with a standard normal distribution. This is equivalent to setting the mean and variance of the reference distribution equal to the sample estimates, and it is known that using these to define the specific reference distribution changes the null distribution of the test statistic (see Test with estimated parameters). Various studies have found that, even in this corrected form, the test is less powerful for testing normality than the Shapiro–Wilk test or Anderson–Darling test. However, these other tests have their own disadvantages. For instance, the Shapiro–Wilk test is known not to work well in samples with many identical values.
One-sample Kolmogorov–Smirnov statistic The empirical distribution function Fn for n independent and identically distributed (i.i.d.) ordered observations Xi is defined as
F n ( x ) = number of (elements in the sample ≤ x ) n = 1 n ∑ i = 1 n 1 ( − ∞ , x ] ( X i ) , {\displaystyle F_{n}(x)={\frac {{\text{number of (elements in the sample}}\leq x)}{n}}={\frac {1}{n}}\sum _{i=1}^{n}1_{(-\infty ,x]}(X_{i}),}
where 1 ( − ∞ , x ] ( X i ) {\displaystyle 1_{(-\infty ,x]}(X_{i})} is the indicator function, equal to 1 if X i ≤ x {\displaystyle X_{i}\leq x} and equal to 0 otherwise. The Kolmogorov–Smirnov statistic for a given cumulative distribution function F(x) is
D n = sup x | F n ( x ) − F ( x ) | {\displaystyle D_{n}=\sup _{x}|F_{n}(x)-F(x)|}
where supx is the supremum of the set of distances. Intuitively, the statistic takes the largest absolute difference between the two distribution functions across all x values. By the Glivenko–Cantelli theorem, if the sample comes from the distribution F(x), then Dn converges to 0 almost surely in the limit when n {\displaystyle n} goes to infinity. Kolmogorov strengthened this result, by effectively providing the rate of this convergence (see Kolmogorov distribution). Donsker's theorem provides a yet stronger result. In practice, the statistic requires a relatively large number of data points (in comparison to other goodness of fit criteria such as the Anderson–Darling test statistic) to properly reject the null hypothesis.
Kolmogorov distribution
The Kolmogorov distribution is the distribution of the random variable
K = sup t ∈ [ 0 , 1 ] | B ( t ) | {\displaystyle K=\sup _{t\in [0,1]}|B(t)|}
where B(t) is the Brownian bridge. The cumulative distribution function of K is given by
… excerpt ends here. Continue reading the full article.
