WORTH Benchmark Uses Original Mystery Novels to Expose Models' Inconsistent Evidence Discovery
BreadStudio · hf · 2026-10-27
Bread Studio's WORTH Benchmark has models read unannotated original mystery novels written by professional authors, autonomously selecting key clue sentences, identifying suspects and explaining reasoning. Across 70 formal runs on two episodes, clue hits and suspect judgments proved desynchronized—in Episode 02, 31 of 37 scoreable runs simultaneously picked the same wrong set of participants.
More from Research
- Physicist on OpenAI's math result: it's mathematical physics, not theoretical physics — skdh · 2026-10-27
- Mechanistic Interpretability and Fast Memory Recall Are the Same Problem — mayfer · 2026-10-27
- Tsinghua team open-sources C2C: LLMs swap KV-Caches directly for 2.5x faster, more accurate collaboration — solyarisoftware · 2026-10-27
- Qwen model has a dedicated MoE expert for Chinese cloud concepts, sparking URL hallucinations — builderjaydub · 2026-10-27
- OpenAI preprint claims 3-SAT solvable in sub-exponential time, challenging ETH — cephaloform · 2026-10-27
- DRFLOW: EMNLP benchmark shows deep research agents struggle at personalized workflow prediction — mageed · 2026-10-27