Reason in Style: Discovering and Controlling Style in Language Models
Ioana Marinescu, Eric Karl Oermann, Kyunghyun Cho
cs.CL, cs.LG
2026-10-01
An autoencoder finds 6 unsupervised styles in 111K multi-teacher math traces; importance-weighted style SFT on Qwen3 beats vanilla SFT Pass@k across six math benchmarks.
Language models learn what a solution says and how it is written in one next-token objective. Frequent traces then dominate a distilled student, and rarer solution paths get little weight. A mismatch between teacher writing and the student prior can weaken SFT or make it worse. In math, the same problem can be solved correctly in many surface forms, and style is not teacher identity. The goal is to recover that structure without labels and use it as a control.
Independent solutions to the same question share problem information; leftover variation in organization, notation, length, verification, and backtracking is treated as style. An autoencoder maps each trace to a content vector c and a style vector z (256-d each), projects the pair into 8 continuous prefix tokens, and reconstructs the text. InfoNCE pulls together the two c vectors of a question. Lorth penalizes c-z alignment on the same trace so z has to store complementary information. Lrec keeps the pair informative. VICReg-style terms stop z from collapsing. Ldiv stops the two z vectors of a question from merging, which would turn z into a second question embedding.
Gemini 3.1 Pro writes 300 problems in four prompted styles (1,200 traces), labels held out of training. k-means on z recovers the four styles at ARI 0.77 and NMI 0.78. A probe reads style from z at 96.7% and from c at 23.3%. Reconstruction alone leaks style into both heads.
On 111,834 verified traces from nine teachers in SCAS, a GMM with K=6 on z yields s1 (short textbook) through s6 (long think-block). s5 is 41.3% of traces. Normalized mutual information with teacher ID is 0.17; a question has 3.38 styles on average. An LLM asked to invent styles on a 180-trace subset mostly recovers model identity (ARI 0.70). The autoencoder styles sit at 0.06.
Traces are prefixed with [style i] for full-parameter SFT of Qwen3-0.6B, 1.7B, and 4B Base. Labels alone track Vanilla SFT. Importance weights then rebalance, per question, toward a uniform distribution over observed styles, so rare styles get more gradient. At test time the k samples for Pass@k are split evenly across prefixes.
Qwen3-0.6B at k=256:
| Method | MATH-500 | AMC12-2025 | OlympiadBench | AIME 2024 | AIME 2025 | HMMT Feb 2025 |
| Vanilla SFT | 74.9% | 46.6% | 44.2% | 9.7% | 9.5% | 2.1% |
| Style SFT (weighted) | 89.8% | 68.5% | 60.1% | 29.7% | 30.1% | 11.4% |
Vanilla SFT often matches or undercuts the base model. Empirical prefixes and random prefixes do not recover the gap. The lift needs the discovered styles plus reweighting.
Mean diagonal style fidelity is 30.8%, 30.8%, and 32.9% for the three sizes (chance 16.7%). On MATH-500, correctness given a prefix runs from 13.9% (s6) to 30.6% (s4) at 0.6B, 22.2% to 43.1% at 1.7B, and 25.8% to 52.4% at 4B. Vanilla traces that happen to land in s4 are correct only 12.8% (0.6B) and 28.2% (4B) of the time; requesting s4 yields 30.6% and 52.4%.
Per-question sensitivity Δx averages 0.104, 0.125, and 0.164. At 0.6B, 59% of questions move by more than 5 points; at 4B the share is 84%. Split-sample selection gets 33.1%, 45.6%, and 57.0% with a per-question style versus 30.6%, 43.1%, and 52.4% with the single globally best style.
A multi-teacher corpus already contains several useful ways to write a solution. Standard SFT lets frequency collapse that mix. Making styles explicit and rebalancing their loss keeps diversity that is already in the data. For math distillation aimed at Pass@k, this is a conditioning-and-weighting knob on the same traces. Changing the prefix changes the chance of solving a fixed problem, and different problems want different styles.
This is still an incremental result, tied to math with automatic grading.
The authors note that the clusters depend on the objective, the corpus, and cutting z into six Gaussians. Style may be continuous or hierarchical. The study stays in mathematics. Why style and correctness interact, and how to pick a style at test time, are left open.
Mixed Pass@k matches Vanilla on the budget k; 256 draws per style would cost 6x generation. Mean diagonal fidelity is about 31%. Realized style is scored by nearest centroid in a handcrafted feature space, not in the learned z. K=6 is chosen by validation Pass@k. There is no reweight-only ablation, so the gain cannot be split into balancing versus controllability. AIME and HMMT have 30 problems each, so Pass@256 point estimates are noisy.