Latent feedback debate: read a deep intermediate block, not the last — lower loss at same FLOPs
kastnerkyle · x · 2026-11-18
- Frontier labs like MAI and Seed are reportedly exploring latent feedback: fusing the final layer's latent with the next token embedding as model input (Recycler-style).
- The poster argues the final block is the wrong source — you should read a deep intermediate block instead, achieving lower loss at the same params and FLOPs, with an architecture diagram contrasting both wiring options.
- A notable design-space data point for recurrent-depth / layer-reuse pretraining research.
Related event: Researchers say latent feedback should read mid-layers, not the last(2 posts)→
More from Research
- Self-inoculation cuts RL reward hacking from 82% to 20% across 11 environments — repligate · 2026-11-18
- GPU QR competition audit: winning kernel ran 107x faster yet failed on new inputs — kfountou · 2026-11-18
- Embodied First Exam: 1,090 embodied tasks show agents crumble outside privileged settings — ManlingLi_ · 2026-11-18
- GPT-6.2 Astra tops Mercor's OpenWorldEngineer benchmark at 67.2% — morqon · 2026-11-18
- Google details EmbeddingGemma 2: unified multimodal embeddings across text, image, audio, video — GlennCameronjr · 2026-11-18
- LatchBio launches NeuroscienceBench and a 13-benchmark leaderboard for biology AI agents — kenbwork · 2026-11-18