Open-Source JevGraph Cuts KG Extraction Cost 3x vs DeepSeek With 2.9x Speedup
richie9830 · reddit · 2026-09-21
The author open-sourced JevGraph, a pipeline that turns documents into evidence-backed knowledge graphs with an LLM, and benchmarked it against GPT-5.6 Luna and DeepSeek V4.1 Flash.
Setup: 160 frozen positive cases from FewRel 1.0 trainwiki (16 relations × 10 cases, seed 17), with identical cases, choice order, and scoring across all three systems.
Results:
- Latency (p95): Jev 431ms — 2.9x faster than DeepSeek (1242ms) and 3.9x faster than Luna (1699ms)
- Cost per 100 planned decisions: Jev 0.410¢, Luna 1.225¢, DeepSeek 1.462¢
- Accuracy (uncovered cases counted wrong): Jev 87.5%, Luna 89.375%, DeepSeek 93.125%
The takeaway isn't "Jev wins everything": it's substantially faster and cheaper on this task, with a real accuracy tradeoff. The upstream pipeline builds on LlamaIndex's DocJev work.
More from Models
- Why yes/no answers are fast for LLMs: output tokens dominate latency — tinyfool · 2026-09-21
- ChatGPT reportedly removes free-tier chat limits, offering unlimited text chats — nikola_mr64990 · 2026-09-21
- Users say top-tier Astra is too costly, hope GPT-6 fixes token economics — CtrlAltDwayne · 2026-09-21
- TypeSafe's JEV model fully open with $5 free credit, powers 1-second 3D scene generation — tinyfool · 2026-09-21
- Prof. Tom Yeh Posts Kimi 3 Seminar Recording with Nathan Lambert on KDA Architecture — ProfTomYeh · 2026-09-21
- rxwei on Jev: Co-Designing SDKs and Models Unlocks a New Paradigm — rxwei · 2026-09-21