One-hot

Key takeaways

One-hot belongs to mathematics; place it in that map before memorising details.
Learn the definition first, then one example that makes the definition concrete.
Connect One-hot to a quantity you can measure, compute or draw — that is where exam questions come from.
Reproduce the core statement of One-hot from memory before moving on to harder problems.

Reference excerpt

In digital circuits and machine learning, a one-hot is a group of bits among which the legal combinations of values are only those with a single high (1) bit and all the others low (0). A similar implementation in which all bits are '1' except one '0' is sometimes called one-cold. In statistics, dummy variables represent a similar technique for representing categorical data.

Applications

Digital circuitry One-hot encoding is often used for indicating the state of a state machine. When using binary, a decoder is needed to determine the state. A one-hot state machine, however, does not need a decoder as the state machine is in the nth state if, and only if, the nth bit is high. A ring counter with 15 sequentially ordered states is an example of a state machine. A 'one-hot' implementation would have 15 flip-flops chained in series with the Q output of each flip-flop connected to the D input of the next and the D input of the first flip-flop connected to the Q output of the 15th flip-flop. The first flip-flop in the chain represents the first state, the second represents the second state, and so on to the 15th flip-flop, which represents the last state. Upon reset of the state machine all of the flip-flops are reset to '0' except the first in the chain, which is set to '1'. The next clock edge arriving at the flip-flops advances the one 'hot' bit to the second flip-flop. The 'hot' bit advances in this way until the 15th state, after which the state machine returns to the first state. An address decoder converts from binary to one-hot representation. A priority encoder converts from one-hot representation to binary.

Comparison with other encoding methods

Advantages Determining the state has a low and constant cost of accessing one flip-flop Changing the state has the constant cost of accessing two flip-flops Easy to design and modify Easy to detect illegal states Takes advantage of an FPGA's abundant flip-flops Using a one-hot implementation typically allows a state machine to run at a faster clock rate than any other encoding of that state machine

Disadvantages Requires more flip-flops than other encodings, making it impractical for PAL devices Many of the states are illegal

Natural language processing In natural language processing, a one-hot vector is a 1 × N matrix (vector) used to distinguish each word in a vocabulary from every other word in the vocabulary. The vector consists of 0s in all cells with the exception of a single 1 in a cell used uniquely to identify the word. One-hot encoding ensures that machine learning does not assume that higher numbers are more important. For example, the value '8' is bigger than the value '1', but that does not make '8' more important than '1'. The same is true for words: the value 'laughter' is not more important than 'laugh'.

Machine learning and statistics In machine learning, one-hot encoding is a frequently used method to deal with categorical data. Because many machine learning models need their input variables to be numeric, categorical variables need to be transformed in the pre-processing part.

Categorical data can be either nominal or ordinal. Ordinal data has a ranked order for its values and can therefore be converted to numerical data through ordinal encoding. An example of ordinal data would be the ratings on a test ranging from A to F, which could be ranked using numbers from 6 to 1. Since there is no quantitative relationship between nominal variables' individual values, using ordinal encoding can potentially create a fictional ordinal relationship in the data. Therefore, one-hot encoding is often applied to nominal variables, in order to improve the performance of the algorithm. For each unique value in the original categorical column, a new column is created in this method. These dummy variables are then filled up with zeros and ones (1 meaning TRUE, 0 meaning FALSE). Because this process creates multiple new variables, it is prone to creating a 'big p' problem (too many predictors) if there are many unique values in the original column. Another downside of one-hot encoding is that it causes multicollinearity between the individual variables, which potentially reduces the model's accuracy. Also, if the categorical variable is an output variable, you may want to convert the values back into a categorical form in order to present them in your application. In practical usage, this transformation is often directly performed by a function that takes categorical data as an input and outputs the corresponding dummy variables. An example would be the dummyVars function of the Caret library in R.

See also Constant-weight code – Method for encoding data in communications, where a constant number of bits are set Two-out-of-five code – Error-detection code for decimal digits, widely used in barcoding and at one time in telephone exchanges Bi-quinary coded decimal – Numeral encoding scheme Gray code – Ordering of binary values, used for positioning and error correction Kronecker delta – Mathematical function of two variables; outputs 1 if they are equal, 0 otherwise Indicator vector Serial decimal Single-entry vector – Concept in mathematicsPages displaying short descriptions of redirect targets Unary numeral system – Base-1 numeral system Uniqueness quantification – Logical quantifier XOR gate – Logic gate

References

Worked examples

Example 1 — a first encounter with One-hot

Start with the simplest possible case. Write down what One-hot claims or describes in one sentence, then invent the smallest concrete situation in which that sentence is true. In mathematics, the smallest case is usually a single object, a single equation or a single measurement. Check that every symbol or term in your sentence has a meaning in that case.

Example 2 — changing one variable

Take the situation from Example 1 and change exactly one quantity: double it, halve it, or set it to zero. Predict what should happen to One-hot before you calculate. Comparing your prediction with the result is the fastest way to find out whether you understand the idea or only the words.

Example 3 — an exam-style question

Typical questions about One-hot ask you to (a) state it precisely, (b) apply it to given data, and (c) explain a limitation. Practise writing all three answers in under five minutes; the third part is what separates a full-mark answer from an average one.

Applications of One-hot

In research: One-hot appears in mathematics research whenever the underlying quantities have to be modelled precisely. Papers usually cite it as a starting assumption and then explore where it breaks down.
In technology and industry: Engineering practice reuses One-hot in design rules, simulations and safety margins. Knowing the idea lets you read a specification sheet and understand why the numbers look the way they do.
In the classroom: One-hot is common in secondary-school and first-year university syllabi. It links to neighbouring topics 1 (number), Digital electronics, so understanding it makes those chapters shorter.
In everyday life: Look for One-hot outside the textbook — in sport, cooking, traffic, electronics or the sky above you. An example you found yourself is remembered far longer than one you were given.

How to study One-hot in 20 minutes

Read the reference excerpt below once, without taking notes.
Close the page and write down what One-hot means in your own words.
Compare your version with the excerpt and mark what you missed.
Work through the three examples above with pen and paper.
Explain One-hot out loud to somebody else — or to Teacher Smith in the lgStudy chat.

Frequently asked questions

What is One-hot in simple terms?

In digital circuits and machine learning, a one-hot is a group of bits among which the legal combinations of values are only those with a single high (1) bit and all the others low (0). A similar implementation in which all bits are '1' except one '0' is sometimes called one-cold.

Why does One-hot matter?

Because it connects several mathematics ideas at once: it gives you a definition you can apply, a quantity you can calculate, and a way to check whether a result is plausible.

How should I study One-hot?

Read the excerpt, restate it from memory, then work through the examples and applications listed on this page. The five-step study plan above takes about twenty minutes.

What does this page cover?

It gives you a compact reference excerpt plus original lgStudy explanations, examples, applications and study material on One-hot.