Benchmarking OSCAR2 KV Cache in llama.cpp
giveen · reddit · 2026-07-19
## OSCAR2 KV Cache Benchmark in llama.cpp The author tested the **oscar2** KV cache solution in llama.cpp on an RTX 5090, comparing it against `f16/f16` and `q2_0/q2_0`. For Qwen3.6-27B, the generation speed of `oscar2/oscar2` is approximately **42 t/s**, which is about **35%** faster than `q2_0` while closely matching the performance level of `f16`. They also tested performance across different context lengths: from **4K to 1M**, the generation speed of `oscar2` remains mostly flat without obvious degradation. In contrast, `f16` quickly hits OOM at longer contexts, whereas `oscar2` continues to run. The overall conclusion is that this cache solution perfectly balances speed and VRAM usage in long-context scenarios.
More from Infra
- Larry Fink says China is ahead in the AI energy race, citing 100 GW nuclear buildout — rohanpaul_ai · 2026-07-21
- Local AI may pay back in 6–7 years and cut long-term costs by 30–40% — DavidLinthicum · 2026-07-21
- TSMC reportedly plans up to 10% chipmaking price hikes in 2027 — kimmonismus · 2026-07-21
- More open models and llama.cpp updates are coming, says Merve Noyan — mervenoyann · 2026-07-21
- Why adding a second LLM provider breaks more than the API surface — Ok_Extension6373 · 2026-07-21
- UK AI datacentres face backlash over heat, noise and land use — nordicinst · 2026-07-21