Bridging LLM Decision Readouts into DuckDB: Zero-Token Probabilistic Classification via LuaJIT UDFs
Shoddy_Telephone9702 · reddit · 2026-09-22
The author shares an approach to wrap a typed decision readout head (jevask) into DuckDB via LuaJIT UDFs: read top-k probability distributions at the single decision token with zero extra generation, mathematically guarantee schema-safe outputs, and project probabilities/confidence directly into SQL columns for filtering and gating. Key lessons: keep logic server-side (transport-only design), and return explicit 'error:' strings since thrown exceptions get swallowed into NULL in duckdb-luajit. Code available on GitHub.
More from Infra
- Whittle distills on HF: 27B-A3B MoE quant claimed to run on 8GB VRAM laptops — depressedclassical · 2026-09-22
- Qdrant benchmark: post-upload latency spikes are optimizers, tuned configs yield up to 100x faster search — qdrant_engine · 2026-09-22
- Inference-free SPLADE: retrieval at BM25-like query cost without per-query inference — qdrant_engine · 2026-09-22
- What Cloudflare can't do: D1 caps at 10GB, is single-threaded, and no real Postgres — Paimaamu · 2026-09-22
- Rackspace joins NVIDIA Cloud Partner Program with Blackwell pods for regulated enterprises — DavidLinthicum · 2026-09-22
- Reverse-engineered Splash format ports Qwen3.8-27B to Mac, 88 tok/s at 64k on M5 Max — SeveralViolins · 2026-09-22