How models fake determinants: a 3x3 trick that dies at 4x4 and a sign-counting hack
scaling01 · x · 2026-09-27
A deep-dive (summarized by Opus) into how models Astra, Sol and Fable compute determinants: Astra's edge comes from two learned tricks with sharp limits, not a general algorithm.
- The 3×3 trick: Astra nails 3×3 determinants in one step (100% with entries up to ±5) where Sol and Fable score 0. But it's narrowly specialized: accuracy drops to 50% at entries around ±25, near 0 at ±99; dense 4×4s fail even with tiny entries (≤25%); and it only works on a 3×3 hidden in a larger matrix if the block sits isolated in a sparse matrix of size 6+.
- The sign trick: Astra gets the sign right by counting minus signs — perfect while the rarer sign appears ≤16 times, even in 512-number lists or 64×64 matrices, breaking around 30. Length itself doesn't hurt; dense negative packing does. Sol and Fable lose the sign after 2–3 negatives, and Fable just guesses 'negative' when it sees many minus signs.
This informs ongoing probing: fixing the count of -1s while varying matrix size n would hint at what algorithm, if any, the model implements.
More from Models
- ScienceArena benchmark: LLMs score 64.5% on chemistry tasks needing structural diagrams vs 74.1% without — geoffwolfe · 2026-09-27
- GLM-5.3 Flash Matches Claude at 1/429th the Price in a YouTube Script Benchmark — OnlyProggingForFun · 2026-09-27
- Frontier AI is now so cheap and abundant that subscriptions go barely used — intellectronica · 2026-09-27
- Karpathy: Claude Opus 4.5 beats GPT-5 Pro for interactive history learning — doodlestein · 2026-09-27
- ChatGPT-6 Astra cracks 85-year-old 1941 Enigma message in two days — luisdans · 2026-09-27
- Grok accused of uploading user chat images to the web as Musk says 'this keeps getting worse' — EthanJPerez · 2026-09-27