CI demand grew 9x in a week: Celesto ditched Celery for NATS, cut queue time 20s
aniketmaurya · x · 2026-10-28
Developer Aniket Maurya details how his CI platform Celesto handled a 9x surge in CI minutes in one week:
- Compute: users ran tests concurrently on tens of 8-16 vCPU machines, forcing a major expansion of the machine pool.
- Queueing: Celery couldn't scale for async job management, so the most affected tasks were switched to NATS (full migration this week), cutting queue time by 20 seconds under 100+ concurrent load.
- Security: hardening work ongoing as usage grows (details truncated).
An attached screenshot shows user feedback confirming the queue-time optimization.
Related event: AI Code Boom Spikes Celesto CI Usage 9x in a Week, Forcing Rapid Scaling(2 posts)→
More from Infra
- PC downturn frees wafers but not the AI bottlenecks: packaging and power cap accelerators — BenBajarin · 2026-10-28
- Wan2.2 TI2V-5B on a single RTX 4090: ~8.5 min per 5s clip, faces broke in 2 of 3 tests — NoEagle3604 · 2026-10-28
- Cloud Providers Can't Forget What Enterprises Actually Run On — DavidLinthicum · 2026-10-28
- HUMAIN details site and power groundwork behind its AI infrastructure buildout — HUMAIN · 2026-10-28
- llama.cpp PR fuses MoE matmuls and GLU, boosting Qwen3 CUDA throughput up to 17% — jacek2023 · 2026-10-28
- A 5-minute visual explainer of what happens when you call an LLM API — HowDevelop · 2026-10-28