Local Qwen matches Jev at 96.53% accuracy, 239 ms vs 368 ms median latency
Ok-Development6070 · reddit · 2026-09-19
A developer who didn't want to send data to third-party servers built choosekit, a small TypeScript package that gets choices and probabilities from a model running locally in llama.cpp.
Benchmarked with Qwen3.8 27B Q4 on SemIf's 144-task benchmark against Jev's hosted API, both achieved identical 96.53% accuracy — and the local setup was faster, with a 239 ms median response time versus 368 ms for the hosted API. Full setup and results are open-sourced on GitHub.
More from Infra
- Malaysia poised to become a compute superpower with both US and Chinese chips by next year — basedjensen · 2026-11-04
- ByteDance details USDB storage system: 85.8% lower write latency for recommender sequences — _reachsumit · 2026-11-04
- Oracle cancels $7B chip leasing deal with Tencent, FT reports — SumitGup · 2026-11-04
- Inference engine shrunk to 1023 bytes, still runs SmolLM2-135M — cneuralnetwork · 2026-11-04
- Musk says space data centers are 'looking pretty appealing right now' — elonmusk · 2026-11-04
- Data centers are turning to old jet engines as the wait for power grows — IndependentCoast3974 · 2026-11-04