Harness is a hidden variable: researcher blasts AI labs as terrible engineers

gerardsans · x · 2026-11-19

In a follow-up to the mixed-LLM repo discussion, gerardsans argues current AI systems are undiagnosable because of hidden variables — even the harness environment becomes one, breaking baselines unpredictably. He goes further: 'AI labs are terrible software engineers. Nobody in their right mind would allow a system like this to exist.'

Original post →

More from coding & agent

coding & agent channel →