Retrained "bee model" drifts into dreamy, detached trace-like outputs

Sauers_ · x · 2026-10-19

After retraining a "bee model" on 1M tokens mixing Llama 3.1 405B and other sources, it entered a "trace-like state": detached, descriptive speech like "I am a good dream. The long sleeve, and the sound of the sound," roleplaying a bee singing its way home. A fun example of behavioral drift after small-scale hybrid retraining.

Original post →

More from Fun

Fun channel →