Local ripple-65B scores 38/49/27 on agentic coding: errors encountered, not draft quality, decided the grade

Ok-Brush8377 · reddit · 2026-11-07

The author ran ripple-65b-preview (q4km, llama.cpp) fully locally on a 64GB MacBook Pro through atomic agent: build a 5-part Python app graded by 50 hidden tests, 3 runs scored 38, 49 and 27 (control: qwen 3.8 27b got 27/29/13).
Both models share the same habit: read files, then write the whole app (363–442 lines) in one 5-minute step at 12.6 tok/s. The difference: ripple always executes the code afterward (27b ran it 0–2 times), but stops at the first command that works and calls it done.
The 49/50 run got unlucky early — a semicolon CSV crash, a fix that broke indentation, failed edits — forcing 24 short edit/read/run steps, which incidentally fixed most of what the hidden tests check. The lesson: the score tracked how many errors the model ran into, not first-draft quality.
atomic agent's loop detector caught it re-reading the same lines (visible in logs). Author discloses they work on atomic agent and notes it's a preview model with n=3.
Ends with an open question: any prompt that makes local models test edge cases before saying done instead of stopping at the first green run?

More from coding & agent