Unbounded Labs releases Bart, a vintage LLM trained on pre-1931 text
soggydoggy8 · reddit · 2026-08-25
Unbounded Labs announced Bart, a 2.82B parameter "vintage" LLM trained from scratch on 20.1B tokens of English text written before 1931. The project cost $800.
Highlights:
- Beats GPT-1900 on the Vintage CORE benchmark at its scale.
- Cleaned a massive vintage dataset from Harvard's Institutional Books (242B -> 23B tokens).
- Created Vintage CORE, the first suite of 20 benchmarks for vintage LLMs.
- Ran 10 hours of autonomous research on one H100, finding 26 improvements.
- Released the largest known vintage SFT dataset (416k Q&A pairs).
- Trained the final model in 5 days on an H100 at 60% MFU.
All datasets, methodology, code, and evals are open-sourced. The team is seeking compute grants, funding, and mentors.
Related event: Unbounded Labs Open-Sources Bart, an LLM Trained Only on Pre-1931 Text(2 posts)→
More from Models
- Unredacted Anthropic filing exposes internal data-quality debate months before Opus 3 cutoff — zacharynado · 2026-10-15
- Google says teams across the company are using Gemini 4 Argon on its hardest engineering problems — CurieuxExplorer · 2026-10-15
- Perplexity's new embedding models: index with a 9B model, search with 0.6B — antoine_chaffin · 2026-10-15
- 797GB of n-gram counts in CPU RAM let 1B models match 57% more compute — PangWeiKoh · 2026-10-15
- Leak: Moonshot's Kimi flagship 'x4' done with pre-training, K3.1 in internal testing — lyraxana · 2026-10-15
- Unsloth ships GGUF quants for Nemotron 3.5: Bolt-74B runs in 24GB RAM — danielhanchen · 2026-10-15