Proposed test: scale Astra's determinant task to 64x64 matrices to probe its parity mechanism
justanotherlaw · x · 2026-09-26
Follow-up to the parity discussion: extend the experiments to larger matrices and more -1 entries to see where performance should break down due to precision limits at scale.
The discriminating logic: gradual degradation with length suggests the model uses a periodic feature to count -1s (notably absent in other models), while perfect accuracy on, say, 64x64 matrices would count as evidence against that hypothesis.
More from Models
- Gemini 4 Carbon matches predecessor's score but explores less, gets distracted — patience_cave · 2026-11-11
- Fable 5.2 takes MazeBench frontier, matches GPT-6 Astra puzzle-solving — patience_cave · 2026-11-11
- MazeBench: Fable 5.2 scores 15%, near-frontier at half price; Gemini 4 Carbon 11% — patience_cave · 2026-11-11
- ERNIE 5.5 Flash Preview enters LMArena Battle Mode for crowdsourced ranking — arena · 2026-11-11
- Blind tests: DeepSeek matches Opus on 95% of agentic loops at 40x lower cost — bindureddy · 2026-11-11
- Indie dev flips from $20 Pro back to $200 Gemini Ultra after two days of deep use — yihui_indie · 2026-11-11