LoL at matched FLOPs rivals dense transformers at 1B params, router probe hits 0.95 AUROC
micahgoldblum · x · 2026-11-04
Follow-up posts detail LoL's results: a linear probe on the router separates safe from unsafe web text with up to 0.95 AUROC without ever showing labels, and the architecture matches dense transformers in loss and benchmarks at 1B params on math data at matched FLOPs, with the 360M version holding its own against looped transformers and MoEs.
More from Models
- DeepSeek quietly launches "DeepSeek Harness" YouTube channel, hinting at plugin-based agent framework — gnukeith · 2026-11-04
- Reddit user seeks uncensored local model that runs on a weak PC for search and bulk downloads — Emotional_Orange_279 · 2026-11-04
- DeepSeek partial outage fuels speculation compute is shifting to unreleased V4.1 Pro — AccBalanced · 2026-11-04
- Claude hand-writes a 984-param neural net for C syntax highlighting, 5x faster than Pygments — AlpinDale · 2026-11-04
- Fireworks Forge keynote: Sovereign stack, Ember next-gen targeting 30x efficiency, No-GPU — AccBalanced · 2026-11-04
- Kimi K3.1 Pro weights drop tomorrow: 2.8T params, 1M-token context — togethercompute · 2026-11-04