GPT-4o and 4.1 die Jan 14: a team's task-by-task eval of OpenAI's cheaper Luna replacements
max_gladysh · reddit · 2026-10-21
OpenAI is shutting down GPT-4o, GPT-4o mini, GPT-4.1 and GPT-4.1 mini (plus fine-tunes) on Jan 14, recommending GPT-6 Luna. BotsCrew ran a few hundred requests through blind LLM-as-judge evals comparing GPT-5.6 Luna to GPT-5.2:
- Safe to swap: message classification (same quality, 88% cheaper), data extraction (85%), RAG templated content (fewer invented facts, 83%), short conversation summaries (better, 91%)
- Stay put: long multi-tool conversations (worse — it guesses where 5.2 says "can't verify") and guardrails (missed 1 in 10 harmful messages vs zero)
Gotchas: Luna only runs at default temperature; GPT-6 Luna does tool calling via Chat Completions only at reasoningeffort "none".
Migration advice: move one request type at a time with before/after evals, probe questions your data can't answer, count guardrail misses not accuracy, test long dialogues, and check real spend including cache/reasoning tokens. GPT-6 Luna is 95% cheaper than 5.2 but noticeably weaker than 5.6 Luna — use only where price dominates.
More from coding & agent
- Training an original-character LoRA for MiniMax H3: full workflow and pitfalls — DeliciousGorilla · 2026-10-21
- Dev builds open-source Chrome extension unifying Gemini and Antigravity on Windows — iamfivestars · 2026-10-21
- MCP server SDK CVE-2026-111770: shared McpServer leaks responses across clients — Future_AGI · 2026-10-21
- When a Customer's AI Assistant Calls Support, Who Checks What It's Authorized to Do? — IrfanZahoor_950 · 2026-10-21
- Two weeks in SF: fundraising goes binary, top devs now review agents not code — ai · 2026-10-21
- Hugging Face livestream tomorrow: scaling RL and evals to train frontier agents — ben_burtenshaw · 2026-10-21