Multi-agent system runs full ML projects end to end: 50 tool calls, 27 minutes
kmeanskaran · x · 2026-11-09
A developer shares a multi-agent system that runs ML projects end to end: agents take a raw CSV, explore data, engineer features, build and compare models, and review each other via an Engineer + Skeptic loop, with human approval configurable per stage (config.yml autonomy.askhuman) and decisions logged to decisions.json. A separate analyst agent answers data questions with tables or charts.
Key details:
- Agents don't read data directly—they write and run code that reads it, then decide from the results
- The skeptic gives a verdict plus up to 3 fixes per stage; harness escalates to humans on warnings (fair-lending warnings always)
- Real run: 8 agent steps, 50 tool calls, 328k tokens, 54% cache hit, 27.7 minutes total
- Deployed via Terraform + GitHub Actions to Google Cloud Run using Workload Identity Federation (no stored keys, main branch only)
More from coding & agent
- Open-source Rewind for X extension logs every post you scroll past so refreshes stop losing good tweets — dotey · 2026-11-10
- "Just retry it" hides three LLM failure types — only one is fixed by retrying — ClickOk5811 · 2026-11-10
- Meta's LLM-generated failing tests catch 74 privacy SEVs, cutting incidents 20% — Meta_Engineers · 2026-11-10
- From Data Scientist to AI Engineer: The Evolving Skill Stack Explained — mdancho84 · 2026-11-10
- Hark Labs' model + agent + VM experience beats Codex, says early reviewer — Scobleizer · 2026-11-10
- Dev rebuilds Pokémon Red in 2.5D from his own ROM cartridge using an agentic workflow — NathanWilbanks_ · 2026-11-10