In computer science and information theory, a Huffman code is a particular type of optimal prefix code that is commonly used for lossless data compression. The process of finding or using such a code is Huffman coding, an algorithm developed by David A. Huffman while he was a Sc.D. student at MIT, and published in the 1952 paper "A Method for the Construction of Minimum-Redundancy Codes". The output from Huffman's algorithm can be viewed as a variable-length code table for encoding a source symbol (such as a character in a file). The algorithm derives this table from the estimated probability or frequency of occurrence (weight) for each possible value of the source symbol. As in other entropy encoding methods, more common symbols are generally represented using fewer bits than less common symbols. Huffman's method can be efficiently implemented, finding a code in time linear to the number of input weights if these weights are sorted. However, although optimal among methods encoding symbols separately, Huffman coding is not always optimal among all compression methods – it is replaced with arithmetic coding if a better compression ratio is required.
History In 1951, David A. Huffman and his MIT information theory classmates were given the choice of a term paper or a final exam. The professor, Robert M. Fano, assigned a term paper on the problem of finding the most efficient binary code. Huffman, unable to prove any codes were the most efficient, was about to give up and start studying for the final when he hit upon the idea of using a frequency-sorted binary tree and quickly proved this method the most efficient. In doing so, Huffman outdid Fano, who had worked with Claude Shannon to develop a similar code. Building the tree from the bottom up guaranteed optimality, unlike the top-down approach of Shannon–Fano coding.
Terminology Huffman coding uses a specific method for choosing the representation for each symbol, resulting in a prefix code (sometimes called "prefix-free codes", that is, the bit string representing some particular symbol is never a prefix of the bit string representing any other symbol). Huffman coding is such a widespread method for creating prefix codes that the term "Huffman code" is widely used as a synonym for "prefix code" even when such a code is not produced by Huffman's algorithm.
Problem definition
Informal description Given A set of symbols S {\displaystyle S} and for each symbol x ∈ S {\displaystyle x\in S} , the frequency f x {\displaystyle f_{x}} representing the fraction of symbols in the text that are equal to x {\displaystyle x} . Find A prefix-free binary code (a set of codewords) with minimum expected codeword length (equivalently, a tree with minimum weighted path length from the root).
Formalized description Input. Alphabet A = ( a 1 , a 2 , … , a n ) {\displaystyle A=(a_{1},a_{2},\dots ,a_{n})} , which is the symbol alphabet of size n {\displaystyle n} . Tuple W = ( w 1 , w 2 , … , w n ) {\displaystyle W=(w_{1},w_{2},\dots ,w_{n})} , which is the tuple of the (positive) symbol weights (usually proportional to probabilities), i.e. w i = weight ( a i ) , i ∈ { 1 , 2 , … , n } {\displaystyle w_{i}=\operatorname {weight} \left(a_{i}\right),\,i\in \{1,2,\dots ,n\}} .
Output. Code C ( W ) = ( c 1 , c 2 , … , c n ) {\displaystyle C\left(W\right)=(c_{1},c_{2},\dots ,c_{n})} , which is the tuple of (binary) codewords, where c i {\displaystyle c_{i}} is the codeword for a i , i ∈ { 1 , 2 , … , n } {\displaystyle a_{i},\,i\in \{1,2,\dots ,n\}} .
Goal. Let L ( C ( W ) ) = ∑ i = 1 n w i length ( c i ) {\textstyle L(C(W))=\sum _{i=1}^{n}w_{i}\operatorname {length} (c_{i})} be the weighted path length of code C {\displaystyle C} . Condition: L ( C ( W ) ) ≤ L ( T ( W ) ) {\displaystyle L(C(W))\leq L(T(W))} for any code T ( W ) {\displaystyle T(W)} .
… excerpt ends here. Continue reading the full article.
