AWS Walkthrough: Run:ai GPU Fractioning Lets Two Pods Share One A10G With Enforced Memory Limits
AWS ML Blog · rss · 2026-11-20
The AWS ML Blog published a hands-on walkthrough for sharing a single GPU across pods on Amazon EKS using NVIDIA Run:ai's GPU Fractioning (open-sourced Oct 2026).
- Context: Existing sharing options trade off differently — MIG hardware partitions, device-plugin time-slicing without memory isolation, MPS with config-dependent fault domains. Run:ai GPU Fractioning layers Kubernetes scheduling, enforced per-Pod GPU-memory contracts, and metrics on top of MPS.
- Validated result: Two pods shared one A10G (23 GiB); each allocated 6 GiB within an 8 GiB limit, while an over-limit allocation failed as expected.
- Fit: Idle-prone notebooks, short eval jobs, GPU compatibility tests, small inference services; works on GPUs without MIG. Note: memory limits don't reserve proportional compute, and density gains don't guarantee throughput or cost savings.
- Implementation: Because the Fractioning Operator v0.1.7 needs driver R615+, the tutorial builds a custom AL2023 AMI (driver 615.71.10) with Packer, validated on EKS 1.36 + KAI Scheduler 0.18.5, with step-by-step instructions and cleanup; sample code in the AWS Samples repo.
More from Infra
- Open-source Intern harness: Opus 5.5 hands off to Hy4 Flash, 92.4% at 1/3 the cost — m4rkGS · 2026-11-20
- Report: AI is vastly more compute-intensive than any prior workload, and needs no new user base — BenBajarin · 2026-11-20
- Analyst: Robot hardware demand will overwhelm supply by orders of magnitude for years — Rewkang · 2026-11-20
- AMD was crowned winner of Nvidia's China crisis — but it has a China problem of its own — CurieuxExplorer · 2026-11-20
- Matt Slotnick questions Hunterbrook's AMD story: 'seems really flimsy' — matt_slotnick · 2026-11-20
- Local legal PDF test: hy4-flash beats QFN on citation accuracy at 10 tps vs 15 — tsunami_forever · 2026-11-20