Dev open-sources local prompt optimizer on DSPy+Ollama, boosting llama3.2 3B from 39% to 89% with GEPA
MortisAndTen · reddit · 2026-09-09
Tired of cloud-only prompt 'optimizers', a developer built PromptCraft (MIT, FastAPI + DSPy + Next.js) running fully on Ollama: it returns rewrites with scores and diffs, and with 10-20 examples scores every candidate on held-out samples (k-fold or fixed split), picking few-shot examples via nomic-embed-text for coverage. On a ticket-priority task with llama3.2 3B on M4 Pro: meta-prompt rewrite alone scored worse (39% vs 44%), GEPA at default 80/20 split did nothing, but at 50/50 it hit 50→67, 28→44 and 39→89 — with the author honestly noting the optimistic bias. The winner prompt emerged from metric feedback that the correct label was 'buried in 38 words', ending with 'answer with a single word'. Includes lineage views, thumbs-down-as-constraint, and local dataset generation with dedup.
More from coding & agent
- Eraser Open-Sources AI-Native, Coordinate-Aware Diagram Format — yawnxyz · 2026-09-09
- Code formatting was solved in the 80s: Ada's DIANA IR stored no text sources at all — maxleiter · 2026-09-09
- QA Engineer Uses Codex to Learn DevOps, Then Wonders What's Left to Learn — Recent-Tangerine2745 · 2026-09-09
- WebMCP Gains Steam: MCP Tools Embedded in Web Pages for Browser Agents, No Server Needed — every · 2026-09-09
- Reducto doubles Extract speed, hits 99% citation coverage, flat 2¢/page pricing — ycombinator · 2026-09-09
- Vercel Switches to Flat-Rate CDN Pricing to End Surprise Bandwidth Bills — cramforce · 2026-09-09