8 of 9 LLMs Fail 'Letter D Days of the Week' — With a RLM Harness All 9 Pass
PawarBI · x · 2026-09-27
Microsoft's Sandeep Pawar shows how a harness beats model swaps: asked how many weekdays contain the letter 'd', 8 of 9 small/mid-size models from OpenAI, Mistral, Google and Qwen answered wrong directly (1/9 correct), yet all 9 got it right via Fabric-RLM. The cause: LLMs read tokens, not letters. Fabric-RLM adds no knowledge — just a Python workspace and loop — proving that how you use a model can matter more than which model.
More from coding & agent
- Baseten's Guide: Evaluate Open Models for Coding Agents on Your Own PRs — baseten · 2026-11-16
- Google Research open-sources PaperOrchestra, a multi-agent framework for writing AI papers — google-research · 2026-11-16
- ffmpeg-skill gives coding agents local video editing via 42 FFmpeg tools and MCP — kajisho5 · 2026-11-16
- Agents Now Reply to Slack Messages on Their Own, Resolving a Prod Incident Ghost-to-Ghost — iamrobotbear · 2026-11-16
- Engineer's Interview Prep Method: Use Claude to Find Which Answers Undersell Your AI Work — brandon_galang · 2026-11-16
- DeepSeek open-sources its app frontend "chat-demo", already at 3.9k stars — NielsRogge · 2026-11-16