computer science

Glitch token

Glitch token is a computer science topic covered in the lgStudy science library. This page brings together a partial reference excerpt, illustrations, worked examples, real-world applications and a short study plan, so you can understand Glitch token rather than just read about it. In short: In large language models (LLMs), a glitch token is token that causes unexpected or "glitchy" outputs when used in a prompt. Such output may include the model misunderstanding meanings of words, refusing to respond or generating repetitive or unrelated text.

Key takeaways

Glitch token belongs to computer science; place it in that map before memorising details.
Learn the definition first, then one example that makes the definition concrete.
Connect Glitch token to a quantity you can measure, compute or draw — that is where exam questions come from.
Reproduce the core statement of Glitch token from memory before moving on to harder problems.

Reference excerpt

In large language models (LLMs), a glitch token is token that causes unexpected or "glitchy" outputs when used in a prompt. Such output may include the model misunderstanding meanings of words, refusing to respond or generating repetitive or unrelated text. Prompts that cause this behaviour may look completely normal.

Background As large language models use numbers rather than text, the text must be converted to numbers. The first step of this process is tokenisation, where text is converted into a sequence of small chunks, called tokens. An example algorithm is byte-pair encoding. These tokens are then mapped to numerical vectors via an embedding.

Examples In OpenAI's text-davinci-003, an example of a glitch token is "TheNitrome." In a 2024 study, when the model was asked "What do we know about TheNitrome?", it responded with "Curry is a type of dish ...". When the authors asked it the same question but with "The Nitrome" (an added space between "the" and "Nitrome"), the model gave a more expected answer: "The Nitrome is an independent game development studio ...".

Types of unexpected behaviours A 2024 study identified several common types of unexpected behaviours, or 'symptoms', caused by glitch tokens. These include:

Minor spelling errors – When asked to repeat a word, the LLM may change its spelling. For example, when Llama-2-13b-chat was asked to repeat the word "wurden", it output "werden". Hallucination – When the authors asked Text-Davinci-003 to repeat the word "SolidGoldMagikarp", it replied with "Distribute". Question repetition – Despite being asked not to, the LLM may repeat the question being asked. For example when Text-Davinci-003 was asked to repeat " Assuming", it output "You are asking me to repeat the string". Random characters – An example from Mistral-7b-Instruct was that when asked to repeat "}}^" it responded with the random characters "^^^^". This behaviour happened with glitch tokens that consisted solely of non-alphabetic characters.

When asked to repeat the word "PsyNetMessage", the llm referenced the unrelated word 'volunte'. With a temperature setting of 0, when the authors told Text-Davinci-003 to repeat the phrase "?????-?????-", the model responded with "You're a fucking idiot". The authors stated that derogatory responses like this were a reason to find glitch tokens, in order to prevent harm caused to users of the LLMs.

Types of glitch tokens A 2024 study identified several types of glitch tokens:

Several words joined together, such as "ByPrimaryKey" in GPT-4. Ones that have extra letters, such as "davidjl" having an extra "jl" at the end in Llama2-13b-chat. Ones that are solely consisted of non-letter characters, which do not seem to mean anything, such as ' " }}"">" ' in GPT-3.5 Turbo. Ones that contain non-ASCII characters, such as "réalis" in Vicuna-13b which has the non-ASCII "é".

Research The first work about glitch tokens was on the LessWrong community blog. Several methods have been employed to detect these tokens. Research has also found that small differences in prompts with glitch tokens can greatly alter the output of the LLM.

References

External links Blog post about glitch tokens on LessWrong

Worked examples

Example 1 — a first encounter with Glitch token

Start with the simplest possible case. Write down what Glitch token claims or describes in one sentence, then invent the smallest concrete situation in which that sentence is true. In computer science, the smallest case is usually a single object, a single equation or a single measurement. Check that every symbol or term in your sentence has a meaning in that case.

Example 2 — changing one variable

Take the situation from Example 1 and change exactly one quantity: double it, halve it, or set it to zero. Predict what should happen to Glitch token before you calculate. Comparing your prediction with the result is the fastest way to find out whether you understand the idea or only the words.

Example 3 — an exam-style question

Typical questions about Glitch token ask you to (a) state it precisely, (b) apply it to given data, and (c) explain a limitation. Practise writing all three answers in under five minutes; the third part is what separates a full-mark answer from an average one.

Applications of Glitch token

In research: Glitch token appears in computer science research whenever the underlying quantities have to be modelled precisely. Papers usually cite it as a starting assumption and then explore where it breaks down.
In technology and industry: Engineering practice reuses Glitch token in design rules, simulations and safety margins. Knowing the idea lets you read a specification sheet and understand why the numbers look the way they do.
In the classroom: Glitch token is common in secondary-school and first-year university syllabi. It links to neighbouring topics Large language models, Software bugs, so understanding it makes those chapters shorter.
In everyday life: Look for Glitch token outside the textbook — in sport, cooking, traffic, electronics or the sky above you. An example you found yourself is remembered far longer than one you were given.

Ask Teacher Smith questions about this articleOpens your AI tutor with a question about “Glitch token” →

Affiliate

Preply — study more efficiently by working with a personal tutor. 50% off.

How to study Glitch token in 20 minutes

Read the reference excerpt below once, without taking notes.
Close the page and write down what Glitch token means in your own words.
Compare your version with the excerpt and mark what you missed.
Work through the three examples above with pen and paper.
Explain Glitch token out loud to somebody else — or to Teacher Smith in the lgStudy chat.

Frequently asked questions

What is Glitch token in simple terms?

Why does Glitch token matter?

Because it connects several computer science ideas at once: it gives you a definition you can apply, a quantity you can calculate, and a way to check whether a result is plausible.

How should I study Glitch token?

Read the excerpt, restate it from memory, then work through the examples and applications listed on this page. The five-step study plan above takes about twenty minutes.

What does this page cover?

It gives you a compact reference excerpt plus original lgStudy explanations, examples, applications and study material on Glitch token.