askmdb: fine-tuned Qwen models turn plain English into MongoDB queries, 0.8B hits 83.6% locally
AlternativeEnough352 · reddit · 2026-11-11
A developer released askmdb, a single-binary local CLI that turns natural language into MongoDB aggregate pipelines using fine-tuned Qwen models (0.8B–9B, Apache-2.0 GGUF). No Python, no server, no API keys; a deterministic C++ verifier whitelists every pipeline step, so a hidden $out inside $facet is always blocked.
Benchmarks (500 EN/ES questions, correctness = pipeline returns the same result as reference):
- askmdb-9b: 91.6%, askmdb-4b: 89.6%, askmdb-2b: 86.8%, askmdb-0.8b (CPU): 83.6% at 1.18s median latency
- Stock Qwen3.5-9B: 47.8%; Gemma 4 E4B: 45.4%
Key lesson: v1 plateaued at 68.6%. Grouping failures by sub-cause revealed zero $convert examples in training data (the generator quietly avoided mixed-type fields). Adding 5,000 targeted examples lifted it from 343 → 418 correct. Total cost: $14 of rented RTX 5090 (12 GPU-hours). Code MIT; author is asking for tips on sub-1B $lookup joins.
More from coding & agent
- Open-source Agent Standup taps Claude's new Whiteboards and Sheets for agent standups and timesheets — stickygoose · 2026-11-11
- Nod details its agent evals: 98% task completion across 678 runs, zero of 216 memory injections succeed — rohanpaul_ai · 2026-11-11
- 9-year Rust veteran argues Zig beats Rust in the AI coding era — nicolascraske · 2026-11-11
- Cheap builder + frontier reviewer loop matches flagship on 66 tasks at ~1/16 the cost ($0.05 vs $0.72) — KrystalUnity · 2026-11-11
- As Hand-Coding Fades, the Edge Shifts to Research Insights LLMs Can't Supply — janusch_patas · 2026-11-11
- An LLM Hallucination Delivered 72 Eggs; Lesson: Always Review the Plan First — JFPuget · 2026-11-11