Custom llama.cpp fork pushes 27B models to 90+ TPS on an RTX 3090
Brief-Tap-6616 · reddit · 2026-09-15
A Reddit user released llamAmpere, a llama.cpp fork optimized for NVIDIA's Ampere architecture (RTX 30-series), with gains also carrying over to Blackwell and Lovelace cards.
- Recommended config hits 90+ TPS up to 100K tokens (at temp 1), with context support up to 240K
- Ships with a Qwen3.8-27B IQ4XS GGUF quant: roughly 4KM quality but considerably faster
- Author claims 80% speedup over near competitors at 200K context, making local 27B inference faster than API speeds on a single 3090
More from Infra
- To dethrone Postgres, new databases must prioritize ecosystem integrations from day one — mgill25 · 2026-11-02
- ESP32-powered 4-inch touchscreen builds a TUI dashboard for monitoring a DGX inference cluster — CATLLM · 2026-11-02
- OpenRouter crosses 10 quadrillion annualized token run rate, doubling every 10.5 weeks — deedydas · 2026-11-02
- Palmer Luckey asks: can you use datacenter RAM for things prohibited by the operator's terms of service? — avaitopiper · 2026-11-02
- Disable MTP for 20-30% faster local LLM inference on limited RAM — InternalMode8159 · 2026-11-02
- Kioxia up 216% in a year but down 68% from its June 2026 peak — zephyr_z9 · 2026-11-02