Testing 1.2k concepts: some prompts remain inarticulable no matter the model
sanmikoyejo · x · 2026-10-31
- A thread from Alexander Spangh measures whether 1.2k concepts across 8 qualitative domains (coding, math, law, academia, journalism, fiction, humor, etc. — the CreativePreferences set) can be articulated via prompts between LLMs.
- Core finding: as models scale, most concepts CAN be articulated, but a small subset cannot — no better prompt, bigger model, or more examples can express prompts about certain practices, identities, and social groups.
Related event: Study Finds 65 of 1,270 Concepts Can Never Be Taught to LLMs via Language(2 posts)→
More from Research
- Lacuna's Standing On traces any repo's research roots — nanoGPT stands on 140 papers — hugo_larochelle · 2026-10-31
- Three Mathematicians Join Lex Fridman to Discuss AI Solving Navier-Stokes, Quasi-Riemann — khademinori · 2026-10-31
- ANSER-Bench: 2,235 analytical search tasks trip up SOTA Deep Research in ~50% of cases — CShorten30 · 2026-10-31
- Stanford: AI reads routine ECGs to flag sudden cardiac death risk in 86% with no known warning signs — oziadias · 2026-10-31
- Zip-NeRF Gets Orders of Magnitude Faster to Train, but Rendering Still Crawls at 3-4 FPS — jonstephens85 · 2026-10-31
- Researchers reconstruct Meta Muse's hidden charter from 884 chain-of-thought samples — sethlazar · 2026-10-31