Talkie is a small language model developed by Nick Levine, David Duvenaud, and Alec Radford. It was announced in April 2026 and described by the developers as a vintage language model. Talkie is trained solely on pre-1931 texts that are in the public domain, in order to reduce legal issues and liability with releasing model data. The model family consists of a 13 billion parameter model called talkie-1930-13b-base and a post-trained checkpoint designed to power a chat interface, called talkie-1930-13b-it.
Development The initial idea was to build a model trained on historical data that could be used to explore whether models can forecast future events: what is predictable, and how far out events can be predicted. The model was also developed in order to study cultural change, and model self-conception. Another goal expressed by authors is testing whether language models can arrive at inventions or scientific discoveries. The authors cite a thought experiment proposed by Demis Hassabis, who asked whether a model trained on data up to 1911 could independently discover Albert Einstein's General Relativity theory. The model was trained on 260 billion tokens of pre-1931 English text from sources like the Institutional Data Initiative, Common Pile or the Internet Archive. The 31 December 1930 cutoff is based on copyright term rules in the United States, where works published between 1923-1977 are protected for 95 years. The data includes books, newspapers, periodicals, scientific journals, patents, and case law. The developers experimented with various optical character recognition (OCR) methods and developed a dedicated vintage OCR system. The compute needed to train the model was provided by Anthropic. The term vintage language model is attributed to Owain Evans and describes language models trained only on historical text. The purpose of such models is to simulate language use from the past, and to study behavior of models not contaminated by contemporary content. Other vintage models include Ranke 4B, Mr Chatterbox or Machina Mirabilis. Talkie is also inspired by Calcifer Computing’s work on Temporal Language Models, able to represent temporal trends in language. One of the main challenges with building vintage model is contamination of the model by anachronistic data from beyond the cutoff data. This is typically due to incorrect metadata, or editing notes added to the text. Because of this, talkie is for example aware of Franklin Delano Roosevelt and Adolf Hitler. A dedicated post-training pipeline was developed in order to fine tune a chatbot based on the base model. For this purpose, only historical structured texts, such as etiquette manuals, letter-writing manuals, cookbooks, dictionaries, and encyclopedias, were used, to avoid contamination of the model. Nevertheless, due to the fact that the model was also fine-tuned through synthetic chats with a Claude Opus model, some anachronisms were introduced.
Reception Multiple outlets used the available demo of the Talkie chatbot to test the model's worldview. For example, Decrypt's Jose Antonio Lanz reported on Talkie's analysis of Adolf Hitler. Both the Decoder and The Register noted that the model performs worse than modern counterparts on standard benchmarks like HumanEval. The limitations of the model have been acknowledged by the research team. The "Harvard Observer" has highlighted the importance of libraries and archives in the development of such language models, including Harvard University's Institutions Data Initiative. The article notes the importance of the quality of both input data and outputs in projects like talkie. John Naughton in "The Observer" argues that talkie is important for ongoing debates on AI development and copyright. He states that "It is, by design, the thing that the plaintiffs in the current piracy lawsuits are arguing that the big labs should have built". Novelist Robin Sloan praised the Talkie project as "a triumph", distinguishing it from earlier vintage language model experiments. He engaged with the Hassabis thought experiment and proposed an alternative, simpler benchmark. Instead of General Relativity, he considered whether the model could arrive at Claude Shannon's insight about mapping electric circuits to Boolean logic. Sloan tested this with Talkie, which denied any such correspondence exists. Simon Willison describes the base talkie model as a "vegan" model – one trained entirely on licensed or out-of-copyright data. The chat model does not qualify, because of the above mentioned contamination during fine tuning, when proprietary synthetic data generated by Claude Opus was introduced.
References
External links Official website Talkie-powered chat interface Talkie project repository on GitHub Talkie project repository on HuggingFace
