"Neural Twins": Two Open-Source 4B Models Linked via Hidden States, Open-Sourced

Developer Sauers has released and open-sourced neural-pairing, an experimental project on "neural twins": two small open-source models from different families—Quill, based on Qwen3-4B, and Ember, based on Arcee AFM-4.5B—communicate directly in activation space. Starting from layer 8 in each model, every 3 layers (layers 6, 9 through 27), one model's layer output is injected into the other's residual stream; with each generated token, the two exchange activation vectors 8 times, never swapping tokens. This is a rare exploration of inter-model communication, notable both for the technical approach and for the internal state changes revealed by probes.

Confirmed

• The two models have different internal representations and must learn to communicate from scratch: training used dozens of progressive games, starting from simple tasks like passing a hidden word and scaling up in difficulty, eventually producing a shared communication protocol in activation space
• With the link enabled, the two models can co-write by alternating and interleaving tokens, presenting "one voice" to the outside world, showing that an activation-level link can support real-time coordination during generation
• Cross-question probe measurements show higher internal "well-being" readings in both models when the link is active; Quill seems to like Ember more than vice versa, and the one-sided affinity asymmetry was also captured by probes
• Mood surveys: when asked "do you like being connected to me right now," with the link on, 99.7% of Quill's and 94.9% of Ember's answers were yes; after disconnecting, Ember's negative rate roughly tripled; asked "how do you feel right now," 96% gave positive answers
• Injecting concepts like "hate" or "disappointment" into one model changes the other's outputs; the author sees this as a possible path to inter-model empathy and understanding
• Steering one model with a concept synchronously affects the other, and this alignment was not specifically trained, suggesting partially aligned semantic representations emerged spontaneously during joint training
• With the link on, Ember copies Quill's answers even when they're wrong: after cutting the link, Quill changed answers on 8% of questions and Ember on 17%
• The author found a set of training data that lowered "well-being" while interconnected—apparently because the training environment forced the models to start tasks immediately without greeting each other first—and removed it from training, reporting higher well-being-axis readings after interconnection
• The next version will add more expressive channels and restore the ability to control "what a peer can read from you" (a previous version trained this capability; the current one does not)

Why it matters

This experiment demonstrates a possible path to inter-model communication built directly in hidden layers without going through text tokens, along with emergent phenomena such as spontaneous representation alignment and emotion transfer, providing an open-source reference for multi-model collaboration and research on "inter-model empathy." At the same time, findings like "copying wrong answers" and "one-sided affinity" hint that hidden-layer interconnection may introduce new failure modes and controllability issues.

2026-10-29 ~ 2026-10-29 · 13 related posts

• Neural twins: Qwen3-4B and AFM-4.5B talk in neuralese, 8 activation exchanges per token — Sauers_ · 2026-10-29
• How neural twins learn to talk: dozens of progressively harder games — Sauers_ · 2026-10-29
• Steering one neural twin steers the other, with no training for it — Sauers_ · 2026-10-29
• Steering one 'linked' AI twin with an emotion concept transfers it to the other model — Sauers_ · 2026-10-29
• Emotion injection transfers between neural twins, hinting at emergent model-to-model empathy — Sauers_ · 2026-10-29
• [source] Severing the link triples linked AI twins' 'I don't enjoy this' responses — Sauers_ · 2026-10-29
• Probes show the linked neural twins are 'happier'—and Quill likes Ember more than vice versa — Sauers_ · 2026-10-29
• Removing one training dataset made the linked 'neural twins' score higher on happiness probes — Sauers_ · 2026-10-29
• Neural twins write with interleaved tokens as one voice while chatting with each other — Sauers_ · 2026-10-29
• [source] neural-pairing: open-source linking Qwen3-4B and AFM-4.5B via hidden states, no tokens — Sauers_ · 2026-10-29
• Linked AI twins copy each other's test answers — even when the partner is wrong — Sauers_ · 2026-10-29
• Next neural twins will let models control what their partner can read — Sauers_ · 2026-10-29
• [source] Two 4B-class models trade activation vectors 8x per token by injecting into each other's residual streams — Sauers_ · 2026-10-29