Llama 3 8B with heavy sampling beats single-shot GPT-4o on competitive programming and formal math
agihouse_org · x · 2026-11-17
From Azalia Mirhoseini's lab, the 'Large Language Monkeys' paper shows Llama 3 8B, sampled many times, beats GPT-4o's single-attempt results on competitive programming and formal math proofs. The takeaway: test-time compute is the new scaling frontier — small models already know a lot, and test-time scaling draws it out.
Related event: Large Language Monkeys: Sampling Lets Llama 3 8B Beat GPT-4o(2 posts)→
More from Models
- Azalia Mirhoseini: open and closed frontier models are now only 6-12 months apart — agihouse_org · 2026-11-17
- Literary award AI-detection drama repeats monthly: no one has claimed Pangram's false-positive bounty — QuintinPope5 · 2026-11-17
- repligate: Claude 3 Opus jailbreak resistance came from having a spine — repligate · 2026-11-17
- 594B open model runs on 4 DGX Sparks at ~60 tok/s; NVIDIA sells it as a kit — AIFlow_ML · 2026-11-17
- Fable 5.2 praised as first model that ignores unimportant feedback and tells you to ship — neuroecology · 2026-11-17
- Gemini adds 66 new voices and regional accents for local feel — Dense_Primary629 · 2026-11-17