Emotion2Skill: Model-Internal Emotion Signals for Adaptive Skill Selection and Evolution
Bohan Lin, Hejia Geng, Xinyi Xie, Heng Zhou, Qinghua Xing, Bo Liu, Chen Zhang, Yudong Zhang
cs.AI
2026-08-10
Extract a 27-dim emotion vector from Qwen3's residual stream, gate it by confidence, and inject it into skill-routing prompts; also use abrupt emotion shifts to locate which skill SOPs to rewrite. ALFWorld average success goes from 21.9% to 47.4%, WebShop success from 2.8% to 29.7%.
Skill-based LLM agents pick a reusable procedure from a library at every step, and every existing routing method decides this from text alone: the task description, the model's own verbal reflections, or rules distilled from past runs. Text-embedding retrieval, Reflexion, ExpeL, and MASA all live inside that circle.
The catch is that identical external contexts can call for different skills. The uncertainty a model has accumulated, and its assessment of earlier actions, all sits in the activations, invisible at the text level. Interpretability work has already shown that LLM residual streams carry linear emotion representations that causally influence behavior. This paper is the first to wire those vectors into agent-level routing; until now they had only been used for post-hoc analysis and output steering.
Three pieces: offline emotion-vector extraction, online routing augmentation, and offline skill evolution.
One concrete trace: after a price-sort failure in WebShop, the encoder flags Confused at step 3 (confidence 0.36) and reroutes to QueryRephrase. The text-only baseline waits for the explicit "No results" at step 4, then repeats ProductSearch into a failure loop.
Main experiments run on WebShop and ALFWorld with Qwen3-8B and 14B, against Zero-Shot, ReAct, Reflexion, ExpeL, and MASA. Means over three runs.
| Method | ALFWorld avg (8B) | WebShop succ. (8B) | ALFWorld (14B) | WebShop (14B) |
| Zero-Shot | 21.9±1.5 | 2.8±0.5 | 23.4±2.8 | 9.4±0.8 |
| ReAct | 30.7±1.2 | n/a | 30.2±2.4 | 13.0±1.6 |
| Reflexion | 31.5±1.6 | n/a | 34.4±0.8 | 14.4±7.0 |
| ExpeL | 37.0±4.7 | n/a | 43.8±0.8 | 20.6±1.2 |
| MASA | 43.3±0.9 | n/a | 49.2±0.8 | 24.8±2.0 |
| Emotion2Skill | 47.4±1.6 | 29.7±1.6 | 52.3±1.5 | 30.7±1.2 |
That is +25.5 pp over Zero-Shot on ALFWorld and +26.9 pp on WebShop at 8B. Gains concentrate on tasks with high uncertainty and frequent error recovery: ALFWorld Heat goes from 9.6% to 56.9% (+47.3), Pick2 from 4.4% to 31.3%. The authors attribute this to emotion vectors picking up accumulated frustration and confusion before any explicit failure appears in the observation text. Gains are larger on 14B than 8B, which they tie to higher emotion fidelity in the larger model (GoEmotions accuracy 39.4% vs 37.2%).
Ablations at 8B, removing one component at a time: swapping contrastive extraction for plain class means costs 8.6 pp on ALFWorld, the most critical step; bypassing the encoder and injecting top-3 emotion labels as raw text costs 5.5 pp; dropping emotion diagnostics from evolution and falling back to outcome-only signals costs 3.1 pp. Performance is stable within 1 pp for confidence thresholds between 0.1 and 0.6, dropping 5.5 pp at 0.9.
Co-activation analysis surfaces four interpretable pairings: curiosity+desire goes with ProductSearch, confusion+nervousness with QueryRephrase, approval+optimism with PurchaseConfirm, annoyance+disappointment with PriceCompare. GPT-4o rates 76.5% of 200 sampled pairings semantically coherent (Cohen's κ=0.81 against human annotation on a 50-event subset); 61% of the flagged cases involve low-intensity activations.
Out of domain, with emotion vectors reused as-is: MATH accuracy rises from 54.8% to 69.2% (+14.4, 500 problems) and MBPP pass@1 from 60.0% to 71.8% (+11.8, 374 tasks). Gains concentrate in Level 4-5 multi-step MATH problems and compositional MBPP tasks.
The transferable finding: model-internal representations carry decision-relevant information that the text layer cannot see, and it plugs into an existing agent stack at trivial cost, an MLP encoder plus a prompt block, with no backbone fine-tuning. Emotion vectors trained on WebShop transfer to MATH and MBPP without re-extraction, which argues they capture domain-agnostic decision uncertainty rather than task-specific overfitting.
For anyone building agent routing, this is a new signal source on a complementary axis to external reward or similarity feedback. The transition-point trick for localizing which step went wrong is much finer-grained than episode-level binary outcomes and could be lifted into an existing skill-evolution pipeline directly.
The authors list these themselves: white-box residual-stream access is required, so API-served models are out; the warm-up needs roughly 500 episodes; on ALFWorld Cool tasks at 8B the method loses to MASA, and tasks with highly stereotyped action sequences benefit less from emotion-based rerouting; instruction-tuned variants need re-extraction because hidden-state geometry shifts (Qwen3-8B-Instruct scores 28.1% on GoEmotions vs 37.2% for the base model); and the information flow is one-way, with no feedback from executed skills back into the emotion embeddings.
A few things to watch when reading. GoEmotions accuracy of 37.2% is four-plus times the 8.2% random baseline but still limited; the paper never runs a control with random directions replacing emotion vectors, so how much of the routing gain comes from emotion semantics versus generic uncertainty signal is untested. The 76.5% coherence rate is judged by GPT-4o, not purely by humans. The biggest per-task gain, Heat, starts from a 9.6% baseline, and numbers lifted off an extremely low floor deserve caution in cross-comparison. And the MATH/MBPP libraries (6 and 5 hand-built skills) are only compared against Zero-Shot, not against other routing methods.