Input 4-5x reduction achieved with sentence/keyword trie on chat
No_Sky9786 · reddit · 2026-08-17
Shared a sentence and keyword-based trie structure that reduces chat input scale by 4-5x. Struggling with automatic budget selection; at 25%, accuracy matches benchmarks and performs well on actual chat input, though it often retrieves too much. Looking for a better retrieval algorithm than CELF.
More from Infra
- Qwen3.8-27B-Ridge-3.7bpw released, shrinking model size to 11.7GB — udmrzn · 2026-08-17
- Limiting GPU max frequency cuts power from 47W to 23W on DGX Spark cluster with minimal decode impact — MaziyarPanahi · 2026-08-17
- Comfy LTX/H3 VRAM Spike Causes Freezes — Dapper_Astronaut_603 · 2026-08-17
- Piper Sandler maps the 8-layer AI infrastructure stack and key stocks — brucemacv · 2026-08-17
- Deconstructing the financing structures behind massive AI compute deals — demian_ai · 2026-08-17
- Bitcoin's 500x Supercompute Edge Powers Bittensor's Decentralized Inference — markjeffrey · 2026-08-17