Blog explores the "shape" of language models and their future tradeoffs in harness design
layer07_yuxi · x · 2026-10-05
The author published a short blog on the "shape" of language models and the tradeoffs different designs may present, arguing it's a research direction worth thinking about now, especially regarding harness design. In replies he breaks down three paradigms: GPT (the one everyone uses), Google NMT (the odd one, recurrent encoder + Transformer decoder), and BERT (earliest, lightly instruction-tuned but never pursued seriously).
Related event: Blog Explores the "Shape" of Language Models and Future Trade-offs(2 posts)→
More from AGI Musings
- Imbue's Josh Albrecht on AI consciousness: physical systems can't be reset, and that may be what makes experience 'matter' — joshalbrecht · 2026-10-05
- Chalmers: at least 50% credence that augmented LLMs could be conscious within a decade — pickover · 2026-10-05
- Aligning superintelligence via pretraining filtering is 'witchcraft, not engineering' — repligate · 2026-10-05
- repligate on AI consciousness: taking it seriously means accepting there's no staying pure — repligate · 2026-10-05
- Worrying about LLMs' misaligned utility functions is incoherent, argues dev — zetalyrae · 2026-10-05
- If One Model Release Updates Your AI View Significantly, You Haven't Thought Deeply Enough — _aidan_clark_ · 2026-10-05