Harness is a hidden variable: researcher blasts AI labs as terrible engineers
gerardsans · x · 2026-11-19
In a follow-up to the mixed-LLM repo discussion, gerardsans argues current AI systems are undiagnosable because of hidden variables — even the harness environment becomes one, breaking baselines unpredictably. He goes further: 'AI labs are terrible software engineers. Nobody in their right mind would allow a system like this to exist.'
More from coding & agent
- mitsuhiko's agent verifies AirPlay hack at 3 AM by blasting sound through the living room mic — altryne · 2026-11-19
- Gemini 4 Neon ships: #1 on AA Coding Agent Index, $3/$15 per 1M tokens — Kyrannio · 2026-11-19
- Mistral Vibe hits desktop; dev's quick test finds two critical vulnerabilities — b_roziere · 2026-11-19
- mitsuhiko's overnight agent woke him at 3AM blasting audio to verify AirPlay reverse engineering — mitsuhiko · 2026-11-19
- Nace ships 71.8GB DeepSeek V4.1 Flash coding quant with 39% lower KL than rivals — m4rkGS · 2026-11-19
- AI is letting hackers attack at scale, warns security pro amid week of outages — IanAndrewsDC · 2026-11-19