Small Foundation Models of Human Cognition and Behaviour
Nick Oh, Fernand Gobet
COLM'26
cs.AI, cs.CY
2026-08-05
Fine-tuning 14 models from 135M to 14B on 10.7 million psychology trials (Psych-101), the authors find that scale barely matters in-distribution: with a higher adapter rank, 0.6B parameters match the 70B Centaur. Scale reasserts itself only on unseen experiments. Masking the content of stimuli and feedback destroys 75.7% of learned information and drops models below chance, showing they use what participants were shown rather than statistical shortcuts in the choice sequence.
In 2025, Centaur fine-tuned Llama-3.1-70B on Psych-101 (160 psychology experiments, 10.7 million trial-level choices) to build a cognitive proxy that predicts human behaviour, beating dedicated cognitive models on seen experiments. That opened a paradigm but left two questions: do you really need 70 billion parameters? And do these models learn task structure, or do they pick up statistical shortcuts in the choice sequence?
This COLM'26 paper answers both. It trains 14 base models from 135M to 14B across four families (Qwen3, Llama-3, SmolLM, OLMo), then decomposes the prompt into four information channels for structural ablation.
Training: rank-stabilized LoRA (rank swept 4 to 64) on Psych-101, a single A100, cross-entropy masked to human-choice tokens, with dataset size swept separately.
Two evaluation tracks: Psych-101 (in-distribution, predicting held-out participants) and Psych-201-RT (out-of-distribution, 18 entirely new experiments).
The diagnostic hinges on decomposing the prompt. Each prompt is split into four channels: task instruction I, experimental stimuli S, outcome feedback F, and choice history C. Four ablation conditions follow: original, instruction-ablated, content-masked (stimuli and feedback replaced with placeholders while keeping layout), and history-only. This separates "uses content" from "uses only the layout template". A trial-order permutation then tests whether the model respects the task's exchangeability structure.
In-distribution, scale barely matters. Fine-tuning buys about ten times more than scale: Qwen3-0.6B improves from 0.731 to 0.528 nats after fine-tuning (a 0.203 gain), while scaling from 0.6B to 14B gains only 0.019. Adapter rank substitutes for parameters: Qwentaur-0.6B at rank 64 (0.513) matches Qwentaur-14B at rank 4 (0.516) with 23 times fewer parameters. The size needed to match Centaur-70B falls with rank: 8B at rank 4, down to 0.6B from rank 32 on.
Out of distribution, scale reasserts itself:
| Setting | Band across eight matched models |
| Psych-101 (in-distribution) | 0.028 nats (0.509 to 0.537) |
| Psych-201-RT (out-of-distribution) | 0.244 nats (0.788 to 1.033) |
After fine-tuning, the in-distribution scale advantage nearly vanishes (0.014, 0.026) while the out-of-distribution advantage largely persists (0.104, 0.172). Making a model larger buys better transfer, not a better fit to Psych-101.
The ablation shows the models really use experimental content. Information retention falls from original (1.00) to instruction-ablated (0.84) to content-masked (minus 0.12) to history-only (minus 0.26), each step significant. Partitioned, stimuli and feedback content account for 75.7%, instruction for 12.5%, and template for 11.7%. Masking content pushes models below chance, so the "choice history alone suffices" shortcut hypothesis fails. The permutation test gives order-invariant predictions on tasks with independent trials and stays sensitive on adaptive tasks, as the design requires.
For cognitive and behavioural modellers, the practical upshot is that 70B is not required: 0.6B to 8B with enough adapter rank suffices within seen paradigms, freeing compute to widen paradigm coverage. The methodological contribution is larger. The authors recast these models as noise ceiling estimators, which give an empirical upper bound on how much human behaviour in a paradigm is predictable at all, and thereby mark where existing interpretable theories still fall short. They are not theories of cognition (unlike ACT-R or Soar, they posit no architecture or mechanism), but as measuring instruments they are useful.
The authors concede several: data subsets are drawn by participant session (measuring depth, more participants per experiment, not breadth, more distinct paradigms), so returns to breadth are unmeasured; participants are nested within experiments, blocking a cross-experiment person-by-task decomposition; all models are dense decoder-only transformers with LoRA, so full fine-tuning and mixture-of-experts or state-space architectures are untested. One more concern: every conclusion is anchored to a single data source (Psych-101) and a single "next choice" supervised objective, so the noise ceiling itself may partly be a ceiling of the method rather than of the data. The authors flag this but do not separate it.