MAIN2026 talk: LLMs can verbalize and control their own activations
karimjerbineuro · x · 2026-10-29
At MAIN2026, Damiano Fornasiere of LawZero argued that "language models can verbalize and control (properties of) their activations," linking model introspection with interpretability research. His slide shows a "Neurofeedback" experiment (Li et al., 2026): the model writes a sentence each turn to maximize a score, with its activations scored in real time (52 → 67) — essentially neurofeedback training for LLMs, with implications for AI safety.
More from Safety
- Orange Tsai's exploit chain pops browser from calculator, hailed as peak hacking — rez0__ · 2026-10-29
- Independent eval finds DeepSeek V4.1 Flash-1017 near-parity with closed models on safety — FangcunAI · 2026-10-29
- Cryptographer Matthew Green: clever claimed subexponential ECDLP algorithm has zero practical implications — matthew_d_green · 2026-10-29
- Why AI sandbox escapes mostly happen in training: alignment must live in RL environments, not rewards — DimitrisPapail · 2026-10-29
- NYT: Chinese spy suspect cultivated close ties with Nvidia CEO Jensen Huang — almmaasoglu · 2026-10-29
- Matthew Green: harsh vuln triage policies entitle researchers to public disclosure — matthew_d_green · 2026-10-29