Co-occurrence is the mathematical backbone of transformers, researcher argues
gerardsans · x · 2026-10-04
gerardsans argues that while co-occurrence isn't the full picture, it's an essential ingredient: transformers are token-driven architectures whose weights and biases are compressed distributions learned via gradient descent, with co-occurrence as the sustaining mathematical framework. He adds the industry will only look back once cornered, predicting a bubble pop ahead.
Related event: Debate: Co-occurrence Statistics as the Mathematical Foundation of LLMs(3 posts)→
More from Research
- Paper Finds Personal Agents Get Worse as Memory Notes Pile Up — rohanpaul_ai · 2026-10-05
- Micro entities and individual inventors are far likelier to use AI in patent applications, new paper finds — robseamans · 2026-10-05
- Elephant rumble choruses encode social context through acoustic convergence, study finds — abenitezburraco · 2026-10-05
- Swarm added just +0.05: splitting context into blocks did the real work for tiny local models — deadatreides1 · 2026-10-05
- Sinclair's Harvard lab publicly demos resetting colon cancer cells via 3 reprogramming genes — tomchapin · 2026-10-04
- Diverse Approach Sampling Beats IID Self-Training on Hard Problems — PMinervini · 2026-10-04