PhilBench quizzes 97 LLMs with the 2020 PhilPapers Survey of 1,785 philosophers
pwlot · x · 2026-11-18
PhilBench turns the 2020 PhilPapers Survey — answered by 1,785 professional philosophers — into a benchmark administered to 97 LLMs, probing what philosophical positions models hold and whether they act on them. The author highlights several standout findings in a thread. An interesting attempt to convert philosophy's largest empirical survey into an LLM evaluation benchmark.
More from Models
- GPT-6.2 Sol tops browser-use leaderboard at 82%, 35% cheaper and 2x faster than predecessor — charliermarsh · 2026-11-18
- Decision models beat LLMs at structured output: Liquid AI d1 runs 2.7-3.6x faster than LFM3 — mervenoyann · 2026-11-18
- Why Anthropic matters: competition, pricing and open source — dh7net · 2026-11-18
- GitHub to Retire Six Copilot Models on Dec 14, 2026, Auto-Switching Enterprise Users — rcleffke · 2026-11-18
- Dev's Deep Dive: GPT-6.2 Astra Improved but Still 'Directionally Very Bad' at Debugging — kimmonismus · 2026-11-18
- OpenAI Charged for a ChatGPT Business Premium Seat for a Month That Can't Be Used — physmathastro · 2026-11-18