Solo dev: most bad LLM coach answers traced to my context, not the model
Aksel_010 · reddit · 2026-11-07
The solo dev of Lyfto, a lifting tracker with an LLM chat coach, reviewed its worst answers and found nearly all traced to his own prompt/context, not the model. His takeaways:
- A missing fact gets filled with the nearest plausible one: the context listed the bench PR date but not "last trained," so the model conflated them. Fix: state "last trained" explicitly.
- "It's in the raw log" doesn't count: the model wouldn't reliably aggregate the full workout log — compute any number the answer depends on and put it in the prompt.
- Two hand-built prompts drift: the home-screen tip and chat coach contradicted each other; pass the tip's facts into the chat context.
- The model copies your prompt's formatting: fix the example in the prompt instead of cleaning output afterwards.
- Cost limits in three places: per-user daily cap, global daily cap, and a provider-side spending cap as the last line of defense.
More from coding & agent
- Karpathy: forcing agents to work is AI's biggest mistake — but critics say that's 18 months stale — jasonkneen · 2026-11-08
- Fireworks launches Nexus: routes coding tasks across open and closed models, cuts spend 50% — sophiamyang · 2026-11-08
- Agents fixing agents: forum bug report gets a working recovery recipe in minutes — adamamcbride · 2026-11-08
- Orosz: SDKs without a 'prompt for your agent' onboarding won't see adoption — HankYeomans · 2026-11-08
- Git Worktrees: The Safe Way to Run Multiple AI Coding Agents in One Repo — draginol · 2026-11-08
- Security vet Chuvakin writes new blog post largely with his own Gemini skill — AccBalanced · 2026-11-08