Trained Vintage LLM Bart for $800 to Explore Pre-1931 Language Models
soggydoggy8 · reddit · 2026-08-25
Unbounded Labs released Bart, a 2.82B parameter "vintage" LLM trained exclusively on English text pre-1931. The project investigates whether models can reach conclusions similar to past scientists.
Key Achievements:
- Outperforms GPT-1900 on the Vintage CORE benchmark with a smaller token budget.
- Cleaned Harvard's Institutional Books dataset (242B -> 23B tokens).
- Created Vintage CORE, the first suite of 20 benchmarks for vintage LLMs.
- Open-sourced all datasets, methodology, training code, evals, and training runs.
Technical Details:
- Trained in 5 days on a single H100, maintaining 60% MFU.
- Ran 10 hours of autonomous research (100 experiments, 26 improvements found).
- Released the largest known vintage SFT dataset (416k graded Q&A pairs).
The team is seeking compute grants, funding, and mentors for future endeavors.
Related event: Unbounded Labs Open-Sources Bart, an LLM Trained Only on Pre-1931 Text(2 posts)→
More from Models
- 6 open models, 80 M&A memos, double-blind AI judges: Step 5 Preview claims a new Pareto frontier — Aizkmusic · 2026-10-16
- Muse Spark Open lands on MLX just 30 minutes after release — pcuenq · 2026-10-16
- Google launches revamped Gemma docs hub covering fine-tuning, inference and model internals — _philschmid · 2026-10-16
- Post-trained NVIDIA Nemotron-3.5-Super-VL hits 35% on AutoResearchExam, matching a 20× larger model — AlexGDimakis · 2026-10-16
- Nathan Lambert slams Meta's Muse Open License as 'trash' — natolambert · 2026-10-16
- Google launches Gemini 4 Argon to Ultra and Pro subscribers, touting physics and 3D skills — ShikharMurty · 2026-10-16