Open-weights models aren't copies of the internet: data coverage is AI's bottleneck
gerardsans · x · 2026-10-20
In a debate, gerardsans argues a 4GB open-weights model on your machine cannot be a one-to-one copy of the internet — compression is lossy. His core point: data coverage is the main limiting factor for current AI. For coding, only common public tasks work well; anything absent from training data won't perform, and "infinite coverage" is impossible.
Related event: Debate Erupts: Is AI Just Compression, and Can It Create New Information?(4 posts)→
More from Models
- Users want open-format export from Claude Docs as charts only save as images — BlackHC · 2026-10-20
- EMNLP study: LLM safety interventions drop up to 4.5x when distress is wrapped in delusion — MaartenSap · 2026-10-20
- Cybersecurity-focused VLM OrcaCyber-Nano-1.0 trends on Hugging Face — orcarouter · 2026-10-20
- 0.8B local model with 2,000 labels beats Jev's frontier API on banking data — kalyan_kpl · 2026-10-20
- New TTS benchmark tests 26 models across 30,564 requests: fastest at 46ms — mattturck · 2026-10-20
- Agents handle unreleased language Jai flawlessly, even compile-time execution — lucasmeijer · 2026-10-20