Logit lens on a Beat Saber finetune reveals it reuses language grammar circuits
ctjlewis · x · 2026-10-16
kalomaze shared a striking demo of transfer learning in action: after a friend finetuned a Beat Saber automapper with a new token head, running a logit lens against the original vocabulary shows the model reusing language-model machinery for music mapping.
- Arc onsets activate opening brackets ((, $()
- Wall-start tokens light up person-denoting nouns (人, woman, guy)
- Beat endings correspond to linking verbs (is, 是, ist)
- Unmapped stretches trigger conditionals (if, 如果)
- Outros/silence activate reflexives (themselves, 自己)
The thread argues the "data wall" is far less of a bottleneck than commonly claimed — transfer learning lets models reuse pretrained linguistic structure even in entirely new modalities.
More from Fun
- ML author Burkov jokes: postal-mail submissions as bullet-proof anti AI-slop fix — francoisfleuret · 2026-10-16
- littmath's poetic AI-automation quip: it'll smell the flower for you — littmath · 2026-10-16
- Surreal AI image turns Dwayne Johnson's torso into a giant rock — OVolosin82152 · 2026-10-16
- Musk amplifies Grok Imagine dance video demo tackling generative AI's hardest motion — elonmusk · 2026-10-16
- Developer Shares That Her 57-Year-Old Chinese Dad Started Using Muse — KlausCodes · 2026-10-16
- UCL Put a Satisfaction Survey Next to Jeremy Bentham's Auto-Icon — gleech · 2026-10-16