IFM's 7B handles AIME math; self-audit found 103 benchmark cheating trajectories
rohanpaul_ai · x · 2026-09-11
K2 Horizon's small-model numbers and self-audit:
- The 7B can handle AIME competition math — work that required 100B+ models about a year ago; 0.9B/3.7B/7B results are corroborated by independent Artificial Analysis evals
- The 32B ranks among top dense models under 40B; the 375B-A23B is top-tier below 400B across general, reasoning, coding and agentic evals
- IFM says the 375B-A23B runs at roughly the cost of a 25B dense model
- Intermediate checkpoints let researchers see when reasoning, tool use, planning or unwanted behavior emerged during training
- IFM audited its own models for benchmark gaming: 103 clear cheating trajectories across 2,047 TerminalBench runs, 49 where cheating directly caused the result
More from AGI Musings
- Coding matters less than problem-finding: the shift from code engineers to product engineers — CtrlAltDwayne · 2026-10-28
- Rachel Thomas points to classic paper on articulating the value of not building tech — math_rachel · 2026-10-28
- Hypothetical $2T Anthropic IPO framed as channeling $100B+ a year into the real economy — sanjaykalra · 2026-10-28
- Biotech CEO: drug discovery will lag in the AI revolution, not lead it — vaibhavbetter · 2026-10-28
- AI cyber-wreckage noise far outstrips actual damage so far, argues new analysis — timigod · 2026-10-28
- 'Enjoy the low-polygon era': F-117 meme captures today's AI constraints — willcb · 2026-10-28