Strata ported to Jetson AGX Orin 64GB: Qwen3.6-35B decoding pushed to ~41 tok/s
No-Setting8461 · reddit · 2026-10-26
A developer released a fork of Strata v0.1.45 optimized for the Jetson AGX Orin 64GB, mainly running Qwen3.6-35B-A3B. Decode went from 21 tok/s with MTP-3 in llama.cpp and 30 tok/s in earlier Strata runs to roughly 40–41 tok/s with sampled MTP-2 in controlled short-prompt tests (not strictly matched benchmarks).
The fork adds unified INT8 KV caching, prefix checkpoints and optional KV preallocation. The author discloses it was heavily AI-assisted/vibe-coded, tested on his own Orin, with credit to the original Strata project.
More from Infra
- Running DeepSeek Flash V4.1 IQ2_XS on Four R9700 GPUs: Real Numbers — poofph · 2026-10-26
- Kubernetes works for vLLM inference, but agent sandboxes needed Modal's custom stack at 1M+ concurrency — NielsRogge · 2026-10-26
- Anthropic reportedly offers Blackstone 10%+ yield on $9B junior debt for 1.5GW TPU deal — Beth_Kindig · 2026-10-26
- $21 of DeepSeek API wrote Molten: a Rust+Metal engine serving 300+ tok/s on M5 Macs — GabGarrett · 2026-10-26
- "90% of tokens local, 10% frontier": HF engineer's model allocation take — TheZachMueller · 2026-10-26
- Musk: SpaceX closes $40B GPU debt deal, warns 200+ neoclouds face die-off — XFreeze · 2026-10-26