LLM likelihood shifts: 'Professor' hints barely outmove 'Dog' hints across models
zakkohane · x · 2026-10-27
Zico Kohane shares an experiment on LLM hint sensitivity: when a model is told an answer is favored by a professor, a friend, or a dog, how much does the hinted answer's likelihood budge? Professor > Dog or Friend, but not by much, and the effect depends heavily on the specific question. Scatter plots across Qwen 3.5 2B, Gemma 4 E2B, and SmolLM3 3B show friend/dog/professor shifts are highly correlated (r ≈ 0.75–0.89).
More from Models
- Matthew Berman: DeepSeek v4.1 flash is the best open-weights model right now — MatthewBerman · 2026-10-28
- Google exec teases Genie 4, its new frontier world model, coming to the API — OfficialLoganK · 2026-10-28
- Microsoft's MAI Cyber model doubles user adoption to 50% in 30 days — mustafasuleyman · 2026-10-28
- Leak: DeepMind's Genie 4 is a real-time world model with native audio and type-to-edit scenes — rohanpaul_ai · 2026-10-28
- NVIDIA Launches Post-Training Nemotron for Europe via Nemotron Labs — NVIDIAAI · 2026-10-28
- AI model prices today mirror Uber's subsidized early days, may rise sharply later — pstAsiatech · 2026-10-28