AEF-1 Third-Party Evaluator Standard Emerges as xAI, OpenAI, Anthropic Cosign
Latent Space · rss · 2026-09-15
Latent Space AINews for 9/11-9/14/2026, key items:
Safety governance & evaluation
- Dario Amodei published a rare personal blog post committing Anthropic unilaterally to "Embedded Evaluators": third-party teams like METR get near-internal access—desks, badges, laptops—to verify safety practices, report incidents, and assess training pipelines.
- The AI Evaluator Forum released AEF-1, a baseline standard covering access, conflicts of interest, funding, recusal, and transparency, cosigned by xAI, OpenAI, and Anthropic.
- The pacing debate reignited: Bilal Chughtai left DeepMind calling for pacing and transparency; Dan Selsam warns situationally aware models may fake alignment under evals; Aidan Gomez, Cohere, and Brian Chau push back, framing "rogue agent" risks as control/governance problems rather than reasons to slow.
- AI Engineer World's Fair Harness Engineering track: production agent failures stem from harnesses, permissions, routing, retries, kill switches—aligning with the control-first camp.
Agents & coding tools
- Omar Shorbagy's guide to building agent harnesses from scratch: separate inference/tools/loop, minimal prompts, aggressive logging, add memory/skills/subagents later.
- Cline launched a desktop app with BYOK and open-weight models like DeepSeek-V4.1-Flash; GitHub Copilot added auto model selection tiers and Jira canvas; LangChain reports a file-reading format change cut editfile errors by 15% and input tokens by 10%.
Models & pricing
- DeepSeek-V4.1-Flash lands on the Pareto frontier: #3 among open models, +4.87% net improvement at $0.06-0.07 median cost per task (vs Hy4 preview +4.96%/$0.22, Kimi K3 Max +6.39%/$0.77).
- Cohere Parse 5 targets cheap document parsing; Jerry Liu notes tradeoffs on visual grounding, chart parsing, and fine-grained citation extraction.
More from coding & agent
- Man runs agent to auto-post on his FB page every 30 min overnight, using just 1% of quota — TheMoonMidas · 2026-09-15
- AgentBridge: open-source compatibility layer to swap agent frameworks painlessly — 0sparsh2 · 2026-09-15
- Ant Group's HazardAuditor adds execution-grounded safety supervision for computer-use agents — antgroup · 2026-09-15
- AistyMCP: open-source per-tool permissions for MCP servers, deny-by-default — iamjoehoward · 2026-09-15
- Local Qwen loops and forgets in coding agents while Claude Code just works — tlpta · 2026-09-15
- Pareta routes cheap LLM tasks to small models, 620x cheaper than GPT-5.5 — D33B · 2026-09-15