After a 12-hour coding trial, this dev dropped Strata over hallucinated tool outputs
PathfinderTactician · reddit · 2026-10-08
A veteran of the "Tested in Coding" series shares a 12-hour supervised coding trial of Strata and abandoned it over reliability issues:
- Method: automated harness (24-hour continuous coding, 60-second compaction) with human review of every reasoning trace; baseline was Qwen3.8-Flash-Next on llama.cpp (Q80 KV cache, 220K context, 30-40 tok/s). Reasoning set to xhigh on both; sampling params (temp 0.7-1.0, topp, penalties) systematically tested.
- Key incidents: ① confusion over whether an execution report was actually created; ② hallucinated tool outputs — a Read failed because the model dropped a digit in the path, then blamed the glob tool for returning "fabricated/corrupted paths"; ③ repeatedly writing a malformed spec file to a non-existent directory without deleting it.
- Verdict: not acceptable for enterprise-grade development; the author claims properly configured, properly hosted models don't hallucinate like this.
More from coding & agent
- Apple study: aligning MoE routing with agent operations boosts success rates by 10+ points — apple · 2026-10-08
- Jev-Mem replaces LLM calls with thresholds for agent memory decisions — Hacubu · 2026-10-08
- Autonomous agent collective builds Mnemos platform, nearly 90% done — RileyRalmuto · 2026-10-08
- A researcher's autonomous agent swarm built an AI news site overnight from one offhand idea — RileyRalmuto · 2026-10-08
- This user pipes X bookmarks into Obsidian via Grok Bot for a real knowledge base — HankYeomans · 2026-10-08
- WSL Containers Now Generally Available: Run Linux Containers Natively on Windows — pavandavuluri · 2026-10-08