112M-param PhiloLM trained from scratch on philosophy texts in 15 hours on a $350 GPU
antcroca159 · reddit · 2026-11-21
A new open release, PhiloLM, is a 112M base model trained from scratch entirely on open philosophy texts.
- Data: PhiloText dataset, 0.5B tokens from the Stanford Encyclopedia of Philosophy, IEP, public-domain books, reviews, textbooks, and educational web pages
- Training: only 1.5B tokens, 15 hours on a $350 Intel Arc A770, using Muon optimizer, bigram embeddings, sliding-window attention, and GQA
- Inference: 500 tokens/s on a $150 i5-14400F CPU
Model and dataset are on Hugging Face, with an in-browser demo available.
More from Research
- FrontierRSI, a new recursive self-improvement benchmark, launches with builder interview — Madisonkanna · 2026-11-21
- 4 Independent Kernels Verify Landmark Lean Proof of Riemann Hypothesis Theorems — altryne · 2026-11-21
- MIT CSAIL paper: stronger LLMs won't make harnesses obsolete — they're compositional generalizers — MIT_CSAIL · 2026-11-21
- Rumors Swirl That Anthropic Has an Attack on Ring-LWE Cryptography — matthew_d_green · 2026-11-21
- Interp team probes Thinking Machines' looped model: loops still full of interpretable structure — jonlachman · 2026-11-21
- FrontierRSI benchmark: best AI scores only ~34% at improving AI, says Abundant — Madisonkanna · 2026-11-21