Hazy shows an MLP can be initialized with knowledge and queried by a transformer
andrew_n_carr · x · 2026-07-23
A post highlights a Hazy research result suggesting that an MLP can be initialized with knowledge already inside it, without any training, and that a transformer can then query and use that knowledge properly.
The implication is that knowledge can be embedded into the network at initialization time rather than learned only through gradient descent. The poster frames it as an intriguing step toward continual learning.
Related event: HazyResearch Proposes Training-Free Method to Write Facts into LLMs(5 posts)→
More from Research
- TMLR paper: enforced thinking doesn't reliably make LLM agents disclose key info to users — SharonYixuanLi · 2026-10-27
- Fern architecture claims 10M-token context with predictions still improving — AccBalanced · 2026-10-27
- Diamandis: AI Cancer Navigator and $100 Genome Sequencing Close the Care Gap — PeterDiamandis · 2026-10-27
- EMNLP Oral: training-free Nexus Sampling cuts KV-cache memory 10x; OpenResearcher tops BrowseComp-Plus at 54.8% — yuz9yuz · 2026-10-27
- ICML 2026 position paper: LLM 'thinking traces' are a harmful anthropomorphism — rao2z · 2026-10-27
- Forking gap follows a power law in context frequency; masking rare n-grams shrinks it — ZimingLiu11 · 2026-10-27