RTX 5090 writes nightly stock briefs with a numbers gate so the LLM can't invent figures
JakeChj · reddit · 2026-09-03
A developer shares a fully local pipeline on an RTX 5090: market data lands in SQLite, deterministic code computes every table and figure, and a local vLLM (Qwen 27B AWQ) writes only a two-paragraph summary. A guard rejects any summary whose numeric tokens aren't present in the input JSON (one retry, then publish without it), so invented numbers never ship. Key gotcha: with Korean+JSON input, thinking mode silently consumed the entire maxtokens budget — chattemplatekwargs: {enablethinking: false} fixed it where reasoningeffort couldn't. Live output and gate design shared.
More from Infra
- FastVideo-FastH3 appears on MLX listing ahead of actual model release — Structure-These · 2026-09-03
- South Korean exports jump 69% YoY in August on AI hardware demand — VraserX · 2026-09-03
- CXMT reaches 10% global DRAM market share in Q2, Counterpoint Research says — zephyr_z9 · 2026-09-03
- Open-Source RL Framework Miles Debuts for Enterprise LLM and VLM Post-Training — AravSrinivas · 2026-09-03
- You Don't Run a Model, You Run Kernels: Why Inference Performance Hides in Fused Kernels — EAccelerate_42 · 2026-09-03
- KAIST's Declarative Attention lets LLMs skip most KV cache reads — kaist-ai · 2026-09-03