E2B built a Jev-powered autoscaler for stateful sandboxes, replacing Karpenter-style heuristics
mgill25 · x · 2026-09-28
E2B engineer diptanu shares how the team built a cluster autoscaler specialized for stateful sandboxes after demand surged: utilization sat at 90–95%, and filling 80% of machine RAM hurt p95 sandbox resume latency from memory snapshots.
Key details:
- Scheduling decisions run on Reflex, an open-source Rust library from Datadog's team, replacing the greedy heuristics of traditional autoscalers like Karpenter. It picks which cloud and instance type to bring up (AWS metal 23xl/48xl; GCP Z3/C4 metal) based on demand, cost, and where sandbox snapshots live.
- It combines current cluster state with blocked-work state in the sandbox scheduler, and only commits actions within defined parameters (e.g., max fleet size).
- As demand stabilizes, it cordons nodes, migrates sandboxes where needed, and scales on-demand clusters back in while longer-term compute contracts cover steady-state load.
More from coding & agent
- Baseten's Guide: Evaluate Open Models for Coding Agents on Your Own PRs — baseten · 2026-11-16
- Google Research open-sources PaperOrchestra, a multi-agent framework for writing AI papers — google-research · 2026-11-16
- ffmpeg-skill gives coding agents local video editing via 42 FFmpeg tools and MCP — kajisho5 · 2026-11-16
- Agents Now Reply to Slack Messages on Their Own, Resolving a Prod Incident Ghost-to-Ghost — iamrobotbear · 2026-11-16
- Engineer's Interview Prep Method: Use Claude to Find Which Answers Undersell Your AI Work — brandon_galang · 2026-11-16
- DeepSeek open-sources its app frontend "chat-demo", already at 3.9k stars — NielsRogge · 2026-11-16