llama.cpp merges FP8 KV-cache support in 710-line PR
Kahvana · reddit · 2026-11-11
ggml-org/llama.cpp merged PR #28392 "Add FP8 KV-cache support" by ORippler: 24 commits, 16 files, roughly +710/-25 lines, after 13 reviews. FP8 KV-cache cuts memory use for long-context inference, a practical win for local and on-device deployments.
More from Infra
- Gamers built the GPU market, now AI demand prices them out — Reddit venting — ark_forever · 2026-11-11
- CyberGate: vLLM plugin steers LLM refusals per-domain without touching weights — DebtFluffy9219 · 2026-11-11
- Huawei's TSMC 'White Glove' Chip Orders Resurface in Debate over Megaspeed's $4.8B Nvidia Buys — 0xkarasy · 2026-11-11
- SoftBank posts first net loss in 2 years after $3.7B data center writedown — ns123abc · 2026-11-11
- Data Center Backlash Spreads: Community Outrage Kills Upstate NY Project on Abandoned Steel Plant — sebkrier · 2026-11-11
- AMD's AI World Tour Japan: global monthly token consumption up 158x in two years — DavidBennett__ · 2026-11-11