Comet cut Opik dataset ingestion time from 39 minutes to 80 seconds by fixing 8 bottlenecks
dl_weekly · x · 2026-11-22
Comet engineer Thiago Hora details how the team made dataset and experiment ingestion in Opik (open-source LLM/agent tracing & eval platform) 30× faster, cutting a 30K-item upload from 39 minutes to 80 seconds.
- Context: every evaluation writes experiment items with traces and scores to ClickHouse, then reads them back for comparison
- Eight bottlenecks were fixed, including a needless per-dataset lock and O(n²) parameter binding
- Eval speed depends both on app output speed and on how fast the platform writes/reads analytics data — deep engineering work paid off massively
More from coding & agent
- Hugging Face's Lewis Tunstall Maps the Road from LLMs to Agent Swarms — _lewtun · 2026-11-22
- Grok Build CLI vs AFK Pilot vs T3 Code: feature comparison across three grok runners — PawelHuryn · 2026-11-22
- A local 8B model made a GGUF quant cheat sheet — all 16 numbers check out — Impressive-Kick293 · 2026-11-22
- GPT-6.2 is hard to steer via system prompt — OpenAI says use it directly in Codex — pvncher · 2026-11-22
- Ghost Mode: open-source Rust Code Mode cuts ~60% of agent tool-call token costs — legend-of-alan · 2026-11-22
- Hangar: open-source macOS/Linux/Windows VMs to sandbox AI coding agents — ozgursoy · 2026-11-22