ForgeSystem: AI agents forge per-workload fused kernels, beating SGLang by 1.21x
aftahi_ai · x · 2026-11-20
A paper from ModelBest Inc. and Tsinghua proposes ForgeSystem, an architecture where AI agents produce demand-specific software artefacts instead of relying on general frameworks.
Core claim: most module boundaries in frameworks like PyTorch or Megatron exist for compatibility, not speed — a hidden tax every user pays. Hand-writing a fused stack per workload was never worth human effort, but agents flip that economics.
Key facts:
- Training: merges gradient clipping, Adam update, and precision conversion — three separate passes over parameters — into one fused kernel with a single memory round-trip
- Inference: organizes an entire decoding step (compute, data loading, on-chip storage) around one data path
- Results: 1.21x over SGLang on decoding; matches or beats expert libraries on 18 of 22 kernel tasks
The authors' open question: if every workload can get its own fused stack, what happens to the general frameworks we all build on?
More from Infra
- Gemini Ultra's $100 plan includes 1000 monthly Colab credits—about 55 H100 hours — iskander · 2026-11-22
- Hy team pushes back on ultra-sparse MoE: sparsity recovers efficiency, not quality — douglovefishing12 · 2026-11-22
- SpaceX lenders push back on $40B of AI chips launched into space — flngr · 2026-11-22
- Homie: an offline Mac notch voice assistant running Qwen 3.5 4B locally — KungFuSaifooo · 2026-11-22
- AMD and PyTorch ship a playbook to run GPT-OSS-20B inference locally with ROCm — PyTorch · 2026-11-22
- Microsoft's full Outlook outage PIR confirms AI tooling wasn't to blame — aronchick · 2026-11-22