Arize Rebuilt Its Agent Three Times: One 8K-Char Memory File Beat a Knowledge Graph
AI Engineer · youtube · 2026-10-28
Arize AI PM Fuad Ali walks through how the company's in-product agent Alex went from a 2023 prototype with "50% hallucination" to a single agent with planning, memory and on-demand skills, built entirely with custom harness code rather than a framework. Key decisions:
- Plan ownership: move the plan into the harness instead of leaving it to the model.
- Turn termination: make "finish" an explicit gate defining what ends a turn.
- Big results as files: treat large tool outputs like files rather than stuffing them into context.
- Memory: one 8,000-character memory file per user beat vector stores and knowledge graphs.
- Consolidation: merge 10 specialist agents into one.
- Model choice: the newest model isn't always best.
- Testing: use production sessions as the test suite.
He closes on what still needs a human and which design choices survived every rebuild.
More from coding & agent
- Supercharging LLM classification with logprobs: soft classifiers you can threshold-tune — JnBrymn · 2026-10-28
- Codex daily build series day 23: skipping iCloud plugins for a reset — dotey · 2026-10-28
- Cloudflare Access adds fully customizable pages with HTML + Liquid templates — dinasaur_404 · 2026-10-28
- Opus retried every session 20.6 million times during a memory leak hunt — johnlindquist · 2026-10-28
- Open-Source Skill Teaches AI Agents to Design and Produce Videos — songguoxiansen · 2026-10-28
- Not Everything Needs an Agent: Fixed Steps Are Recipes, Save Agents for Judgment — adamse · 2026-10-28