RAG and RL tool calling are pushing CPUs back into ML training resource math
StasBekman · x · 2026-10-01
Stas Bekman highlights a shift in ML resource patterns: CPUs were once only for DataLoader work. RAG workloads (database queries) started adding CPU load around 2025, and by 2026 RL tool calling — compiling, running, and validating generated code — is pushing demand further, sometimes requiring dedicated CPU nodes so co-located cores don't stall GPUs. He updated the CPU chapter of his ml-engineering repo (19.1k stars) and speculates CPUs may evolve to be more GPU-like, offloading more work.
Related event: RAG and RL tool calls put CPUs back in ML training(3 posts)→
More from Infra
- BIS issues rule blocking remote access to US AI chips via third-country data centers — teortaxesTex · 2026-11-16
- Qdrant 1.20.1: 50% fewer cold-storage reads, ~2x faster quantization, agent-ready setup — qdrant_engine · 2026-11-16
- One 16GB GPU runs songs, voice, images, 720p video and local RAG — all offline and open source — llama-of-death · 2026-11-16
- Starlink Now Protects Greek Trains From Collisions With 5cm-Accurate Tracking of 134 Locomotives — XFreeze · 2026-11-16
- Google donates Agent Substrate, gVisor and OpenRL to CNCF; Agent Substrate 1.0 goes GA on GKE — dl_weekly · 2026-11-16
- Nvidia scandal isn't slowing down: Huang's near-$100B power web under strain — The Verge AI · 2026-11-16