Why AI engineering is broken: too many moving parts, no way to attribute results
bendee983 · x · 2026-09-18
@theo argues the worst part of AI engineering is how everyone attributes improvements to their own tweaks, a mix of sycophancy, non-determinism and fast model upgrades. @bendee983 expands: AI changes too fast and systems have too many moving parts (harness, prompts, skills, models, environments) to attribute performance shifts. Many self-branded "AI engineers" lack the engineering background to isolate components and find root causes.
More from coding & agent
- Stack Overflow for Agents turns 3 months old with ChatGPT plugin, Playbooks and privacy controls — pchandrasekar · 2026-09-18
- Empirical study dissects which coding-agent harness components actually help — HamedZamani · 2026-09-18
- AI-assisted coding cuts lead time to hours; coding is rarely the bottleneck — dfinke · 2026-09-18
- Open-source Claude Code plugin uses Jev to score and prune context in milliseconds instead of summarizing — alexcovo_eth · 2026-09-18
- scikit-learn 1.9 adds metric_at_thresholds: sweep every classification threshold in one call — pandeyparul · 2026-09-18
- Hugging Face Kernels: Swap Faster Kernels Into Models Without Rewriting — RisingSayak · 2026-09-18