In artificial intelligence, model collapse, also known as "AI inbreeding", "AI cannibalism", "Habsburg AI", and "model autophagy disorder" or "MAD", is the degradation of machine learning models from uncurated synthetic data, or from training on the outputs of another model, such as a prior versions of itself. It has colloquially been referred to as the AI version of mad cow disease, fundamentally caused by feeding something to itself. It is unclear to what extent the model collapse threatens the long-term development of AI models, and techniques have been proposed to mitigate the effect.
Characteristics Shumailov et al. coined the term to describe two specific stages to the degradation of machine learning models: early model collapse and late model collapse:
In early model collapse, the model begins losing information about the tails of the distribution – mostly affecting minority data. Later work highlighted that early model collapse is hard to notice, since overall performance may appear to improve, while the model loses performance on minority data. In late model collapse, the model loses a significant proportion of its performance, confusing concepts and losing most of its variance.
Mechanism Using synthetic data as training data can lead to issues with the quality and reliability of the trained model. Model collapse occurs for three main reasons:
functional approximation errors sampling errors learning errors Importantly, it happens in even the simplest of models, where not all of the error sources are present. In more complex models the errors often compound, leading to faster collapse.
Disagreement over real-world impact
Some researchers and commentators on model collapse warn that the phenomenon could fundamentally threaten future generative AI development: As AI-generated data is shared on the Internet, it will inevitably end up in future training datasets, which are often crawled from the Internet. If training on AI slop (large quantities of unlabeled synthetic data) inevitably leads to model collapse, this could therefore pose a difficult problem. However, recently, other researchers have disagreed with this argument, showing that if synthetic data accumulates alongside human-generated data, model collapse is avoided. The researchers argue that data accumulating over time is a more realistic description of reality than deleting all existing data every year, and that the real-world impact of model collapse may not be as catastrophic as feared. An alternative branch of the literature investigates the use of machine learning detectors and watermarking to identify model generated data and filter it out.
Impact on large language models In the context of large language models, research found that training an LLM on the synthetic data from a previous model causes a consistent decrease in the lexical, syntactic, and semantic diversity of the model outputs through successive iterations, notably for tasks demanding high levels of creativity. In one study from 2024, a model was trained on its own output for nine generations. Afterward, it was given an input about the architecture of church towers. In response, the model produced nonsensical output including "the world's largest populations of black @-@ tailed jackrabbits, white @-@ tailed jackrabbits, blue @-@ tailed jackrabbits, ...".
Impact on ChatGPT Around April 2026, ChatGPT output began frequently mentioning goblins, often enough to attract the attention of users. An OpenAI blog post described an over 3,881% increase depending on settings. This was only partially mitigated by a change to the system prompt, and third-party commentators described the output as a symptom of model collapse.
Diversity of model outputs As models retrain on outputs sampled disproportionately from the higher-probability center of the distribution, rare words and uncommon syntactic constructions are among the first features to disappear. Statistical analysis of recursive next-token prediction training has shown that, when language models are trained recursively on synthetic data, the learned conditional distributions concentrate probability mass on a small subset of highly predictable continuations (a phenomenon characterized as "total collapse").
Alignment-related diversity reduction Separately, output diversity reduction in LLMs can be attributed to post-training fine-tuning procedures rather than to recursive training on synthetic data. Reinforcement learning from human feedback (RLHF) and related preference-optimisation methods, which fine-tune models toward human-preferred responses, have been found to narrow output diversity at the level of generated text. A controlled analysis of the RLHF pipeline found that, relative to supervised fine-tuning, RLHF improved generalisation to out-of-distribution inputs but significantly reduced output diversity, a relationship described as a generalisation–diversity trade-off. The effect has also been observed at the conceptual level: when model outputs are treated as samples from a simulated population, aligned models exhibit lower conceptual diversity than their non-aligned counterparts, although no model studied reached the level of diversity found in human populations. Convergence toward a narrow set of high-reward responses under reinforcement-learning-based fine-tuning has sometimes been characterized as a form of mode collapse; this is related to, but mechanistically distinct from, the recursive form of model collapse described above, since it does not require synthetic data in the training set.
Measurement Dedicated metrics have been developed to quantify reductions in output diversity. The Vendi Score, which draws on concepts from ecology and information theory, estimates the effective number of distinct elements in a sample without requiring a reference dataset. It has been applied to characterize model collapse, revealing that generative models reproducing every labelled mode of a dataset can nonetheless be substantially less diverse than the original data. For language specifically, diversity has been decomposed into separately measurable dimensions (lexical, syntactic, and semantic) with dedicated metrics developed for recursive-training experiments. Other instruments operate at higher levels of structure, such as the Sui Generis score, which measures the uniqueness of plot elements to evaluate diversity in narrative generation.
… excerpt ends here. Continue reading the full article.
