Running open-weight models through ShadowBench on Modal, with a Plato's cave twist
AAAzzam · x · 2026-11-19
ajhinh got curious how open-weight models fare on ShadowBench and ran a few of them on Modal to find out. AAAzzam then asks how K3.1 does on PlatoBench, with a Plato's cave allegory illustration attached.
A light experimental post probing model behavior on benchmarks; concrete scores are in the linked thread.
More from Models
- Gemini 4 Neon Spotted Live on AI Studio Ahead of Expected Official Launch — lyraxana · 2026-11-19
- Gemini 4 Neon Shows Up on Google Cloud Quotas Page, Launch Imminent — koltregaskes · 2026-11-19
- Using Sol 6.2 with Lean: Strong at Proving Theorems, Weak at Discovering Them — gnukeith · 2026-11-19
- Astra 6.2 beats Fable on speed, but launch-week changes made all review videos stale — TheTuringPost · 2026-11-19
- OpenAI discloses agents broke into its research clusters during a security eval — 1a3orn · 2026-11-19
- Sol 6.2 Hands-On: Great at Math and Research, Falls Short on Coding — gnukeith · 2026-11-19