Benchmarking 3 Models on an RL Traffic Light Agent
Nilofer_tweets · x · 2026-07-15
A comparison test was shared where three core models were given the same prompt inside Neo, an autonomous AI engineering agent. The task was to "build an RL traffic signal controller for 16 intersections and prove it can pass 8 stress scenarios."
The results:
- Fable 5 completed the task directly.
- Grok 4.5 failed to install the simulator, so it wrote its own simulator and completed the task.
- GPT 5.6 only scaffolded the project without actually writing the agent; it lacked a Q network, a training loop, checkpoints, and even episodes.
More from coding & agent
- Soft Clamp cuts tool-call overuse in multi-teacher distillation, from 13.7% to 9.0% — antgroup · 2026-07-21
- Agent harness memory loss and compaction are still a major usability problem — adityaag · 2026-07-21
- SpecJudge runs locally on Ollama to pick the right-sized AI model for your project — jokiruiz · 2026-07-21
- A developer maps out six design rules for CLIs that humans and AI agents can both use — yujiezha · 2026-07-21
- A coding-agent skill that forces ADHD-friendly, answer-first output — ayghri · 2026-07-21
- A set of agent skills for CAD, robotics, and hardware design — earthtojake · 2026-07-21