Sol61 flunks instruction-following: skips mandated research even when explicitly told
lucasmeijer · x · 2026-10-26
Developer lucasmeijer is evaluating a new skill across Opus 5.5 and Sol61, running it through pi, his own harness, Claude Code and Codex. His finding: instruction-following is inconsistent across the board, with Sol61 the worst offender — it skips research and jumps straight to a plan even when instructions explicitly demand extensive research first.
More from coding & agent
- Firecrawl offers 2.5x credits to a16z Speedrun founders, unveils $75M Series B — devdigest · 2026-10-27
- Chums: open-source Apple Watch app puts Claude Code and Codex on your wrist — brandon_galang · 2026-10-27
- Open-source Clairic Studio turns pasted text into typed data and working code in 5 languages — dfinke · 2026-10-27
- Matt Pocock shares expand/contract trick: split one-way-door PRs to de-risk big changes — mattpocockuk · 2026-10-27
- Qdrant builds Universal I/O to run vector search mmap code on object storage — qdrant_engine · 2026-10-27
- theo publishes tstack skills, Garry Tan urges everyone to 'publish your stack' — ycombinator · 2026-10-26