Senior engineer: 'popularity is the weights of an LLM' — half of 600 engineering scars can't be baked into models
RandalSchwartz · reddit · 2026-11-01
A veteran engineer argues cross-entropy training makes 'popularity' literally the weights of an LLM, so raw model intuition is coding by majority vote of public tutorials. Using his team's catalog of 600 failure scars across 246 tickets, he estimates frontier labs could bake only about half into weights via rapid-regret commits and Socratic self-play; the other half flip sign per repo (e.g., parallel async writes are best practice in cloud workers but brick single-channel hardware buses). He compares prose checklists in one context window to handing a hasty intern a paper checklist, and argues for enforced sign-off workflows instead.
Key points:
- Senior value = knowing what not to do, which mostly can't come from popularity-weighted training
- 50% of failure scars are universal anti-patterns; 50% are repo-specific invariants
- Asks how others separate the two in their agent harnesses
More from AGI Musings
- Stanford's Christopher Potts publishes long rebuttal to skeptical views of interpretability research — ChrisGPotts · 2026-11-01
- Vitalik Buterin pens 'Digital Minimalism' essay urging intentional, deliberate tech use — SuhailKakar · 2026-11-01
- NYT: Data scientist argues predictive algorithms are flattening culture into monotony — nordicinst · 2026-11-01
- Burkov: humans validating machine work is no different from using a tractor — burkov · 2026-11-01
- Octant and Schrödinger profile 700+ molecules to explain why pKa and logP predictions clash with experiments — _onionesque · 2026-11-01
- Lightcone operator publicly calls Scott Alexander's Anthropic strategy 'unethical' — ohabryka · 2026-11-01