OntoPrune cuts 83% of context tokens, speeding local CPU TTFT by 6.7x
vigmarcarlo · reddit · 2026-10-05
OntoPrune: context pruning that makes small coding models viable on pure CPU
Targeting the two pain points of running small local coding models (Qwen 2.5 Coder 1.5B/3B, Gemma 2B, DeepSeek Coder) — slow prompt evaluation and noise-induced hallucinations — the author open-sourced OntoPrune (MIT, fully offline Python middleware):
- Parses source code into an in-memory knowledge graph via an internal ontology;
- Extracts the exact 1-hop closure of the function being edited via SPARQL (only the classes/functions/interfaces it actually touches);
- Renders the pruned graph back into clean, typed Python stubs;
- Verifies generated code against the contract AST to catch API hallucinations.
Benchmarks (12-core CPU + Ollama, qwen2.5-coder:3b)
- Input tokens: 2,390 → 406 (-83%)
- TTFT: 22.4s → 3.3s (6.7x faster)
- Total generation: 59.9s → 16.5s (-72.5%)
- Hallucinations: full-file context invented 1 non-existent method call; OntoPrune had 0
- OntoPrune's own overhead: just 9.9ms (AST parse + RDF build + SPARQL query)
- On cloud Gemini Flash: 2,815 → 393 tokens, 86% cost reduction
Engineering notes
- Zero RDF exposure: you and the LLM only see plain Python signatures and stubs;
- Ships an MCP server (ontoprune-mcp) for Cursor, Claude Desktop and other agents;
- Multi-module resolution follows project imports without choking on circular deps;
- Contract verification deterministically flags hallucinated APIs in CI/CD.
Install with pip install ontoprune; supports CLI piping straight into Ollama and ships a reproducible benchmark script.
More from coding & agent
- Otis: an open-source AI agent that auto-sets up llama.cpp and unifies local and hosted inference — Objective-Pair8231 · 2026-10-05
- Alibaba's FlyAI Skill Lets Claude Code Search Real-Time Flights and Hotels — tom_doerr · 2026-10-05
- Tutorial: Building a Floating City in Blender Using Claude Opus 5.5 and Blender MCP — sidahuj · 2026-10-05
- Voxel-style videos straight from Opus 5.5: custom skill output called commercially usable — AlchainHust · 2026-10-05
- After testing 200+ vibe coding tools, here are the 40 design resources worth keeping — damienghader · 2026-10-05
- Tenuo open-sources task-scoped authorization for MCP: least-privilege tools with delegation that can only shrink — daniel_tenuo · 2026-10-05