MegaCapybara: RTX 5090-only inference engine hits 2000+ t/s, 2x faster than Ninfer
BringTea_666 · reddit · 2026-10-03
A new open engine purpose-built for RTX 5090 and Qwen3 27B claims 2x the decode speed of Ninfer: 500+ t/s single-stream, 2000+ t/s across 12 concurrent agents with 800k context. Highlights:
- Dynamic kernels per model/task/context-length, plus Dflash2 speculative decoding with DSpark confidence scheduling and draft trees.
- Benchmarked weights: quantized weights ship with metadata showing KL divergence and top-1% loss vs BF16 at every setting, comparable against unsloth quants in the launcher.
- Agent serving: scheduler prioritizes multi-job throughput; fan out 30+ agents.
- Cache management: prefill once per job; context-swapped tasks resume in 0.03s without re-prefill.
- Loop Guard: escalates repetition penalty on agent loops, then fires a stop signal for frontend recovery.
Ships with a GUI launcher, HF auto-downloader, and CLI/bat export. Weights are on Hugging Face; source code to be released later.
More from coding & agent
- FT: US tech outages blamed on rush to fix bugs found by AI — nordicinst · 2026-11-19
- Claude Code overtakes Cursor and Copilot in job-post mentions ahead of Anthropic IPO — antgoldbloom · 2026-11-19
- Dev submits PR adding periodic checkpoints to GKE sandboxes so agents don't lose progress — yoimnotkesku · 2026-11-19
- AWS adds private CA support to Bedrock AgentCore Identity for connecting agents to self-hosted IdPs — AWS ML Blog · 2026-11-19
- Orchestrator's Cloud Agents graduate from beta to general availability — julianweisser · 2026-11-19
- Zero users, 33 pages shipped: dev crowdsources audit for AI-agent trust schema — TheHamzaShahbaz · 2026-11-19