Choosing Models by Public Benchmarks and Vibes Is 'ngmi', Argues Dev for Internal Evals
A_K_Nain · x · 2026-10-13
A one-line opinion: choosing models based on public benchmarks and vibes is "ngmi" — you should never pick a model without results on an internal benchmark tailored to your own workload. A consensus-flavored take on model selection for AI engineering.
More from Models
- ChatGPT Plus User Complains Custom Instructions Are Repeatedly Ignored by GPT-6.0 High — ed2417 · 2026-10-13
- Reddit Praise Piles Up for Claude Opus: Strong Code Quality and Generous Limits Weeks After Launch — zxoyan · 2026-10-13
- Anthropic runs much internal post-training research through production Tinker — simonguozirui · 2026-10-13
- ChatGPT-Assisted USPS Mailing Fiasco: 26 International Letters Lost at a Cost of $1,356 — TelevisionCrafty1795 · 2026-10-13
- Dan Shipper kicks off full runs of Wonder on AI&) senior engineer benchmark — danshipper · 2026-10-13
- Local AI models are improving — but that doesn't mean they fit everywhere — bigdata · 2026-10-13