Zipf's law () is an empirical law stating that when a set of measured values is sorted in decreasing order, the value of the n-th entry is often approximately inversely proportional to n. The best-known instance of Zipf's law applies to the frequency distribution of words in a text or corpus of natural language:
w o r d f r e q u e n c y ∝ 1 w o r d r a n k . {\displaystyle \ {\mathsf {word\ frequency}}\ \propto \ {\frac {1}{\ {\mathsf {word\ rank}}\ }}~.}
It is usually found that the most common word occurs approximately twice as often as the next common one, three times as often as the third most common, and so on. For example, in the Brown Corpus of American English text, the word "the" is the most frequently occurring word, and by itself accounts for nearly 7% of all word occurrences (69,971 out of slightly over 1 million). True to Zipf's law, the second-place word "of" accounts for slightly over 3.5% of words (36,411 occurrences), followed by "and" (28,852). It is often used in the following form, called the Zipf-Mandelbrot law:
f r e q u e n c y ∝ 1 ( r a n k + b ) a {\displaystyle \ {\mathsf {frequency}}\ \propto \ {\frac {1}{\ \left(\ {\mathsf {rank}}+b\ \right)^{a}\ }}\ } where a {\displaystyle \ a\ } and b {\displaystyle \ b\ } are fitted parameters, with a ≈ 1 {\displaystyle \ a\approx 1} , and b ≈ 2.7 {\displaystyle \ b\approx 2.7~} . This law is named after the American linguist George Kingsley Zipf, and is still an important concept in quantitative linguistics. It has been found to apply to many other types of data studied in the physical and social sciences. In mathematical statistics, the concept has been formalized as the Zipfian distribution: A family of related discrete probability distributions whose rank-frequency distribution is an inverse power law relation. They are related to Benford's law and the Pareto distribution. Some sets of time-dependent empirical data deviate somewhat from Zipf's law. Such empirical distributions are said to be quasi-Zipfian.
History In 1913, the German physicist Felix Auerbach observed an inverse proportionality between the population sizes of cities, and their ranks when sorted by decreasing order of that variable. Zipf's law had been discovered before Zipf, first by the French stenographer Jean-Baptiste Estoup in 1916, and also by G. Dewey in 1923, and by E. Condon in 1928. The same relation for frequencies of words in natural language texts was observed by George Zipf in 1932, but he never claimed to have originated it. In fact, Zipf did not like mathematics. In his 1932 publication, the author speaks with disdain about mathematical involvement in linguistics, a.o. ibidem, p. 21:
... let me say here for the sake of any mathematician who may plan to formulate the ensuing data more exactly, the ability of the highly intense positive to become the highly intense negative, in my opinion, introduces the devil into the formula in the form of − i . {\displaystyle \ {\sqrt {-i\;}}~.}
The only mathematical expression Zipf used looks like ab2 = constant, which he "borrowed" from Alfred J. Lotka's 1926 publication. The same relationship was found to occur in many other contexts, and for other variables besides frequency. For example, when corporations are ranked by decreasing size, their sizes are found to be inversely proportional to the rank. The same relation is found for personal incomes (where it is called Pareto principle), number of people watching the same TV channel, notes in music, cells transcriptomes, and more. In 1957 George A. Miller proposed that a power law emerges even in randomly generated texts. and in 1992 bioinformatician Wentian Li published a proof that the power law form of Zipf's law was a byproduct of ordering words by rank.
Formal definition
Formally, the Zipf distribution on N elements assigns to the element of rank k (counting from 1) the probability:
… excerpt ends here. Continue reading the full article.
