Non-Programmer Uses Open WebUI Plus Terminal Container to Search a 500-Page Legal Code
Blindax · reddit · 2026-09-06
A non-developer shares their experience building a local AI document-research workflow, using it to search large documents like a 500+ page tax code and generate notes and slide decks.
- Architecture: Open WebUI runs on Unraid, pointing to the API of a dedicated inference machine (5060 Ti + 5070 Ti); after unsatisfying tries with Hermes and LM Studio, they settled on Open WebUI's official Open Terminal container — a separate container hooked in via Integrations, running unprivileged on a bridge network, so the model gets a shell "inside the box" rather than on the host, keeping things secure.
- Results: The model can reason step by step, reliably locating content in documents far larger than its context window via "list directory → grep → read only the relevant passages," then generating documents from what it finds.
- Setup & performance: Ollama runs Qwen 27B Q4KM with a 100k context; on a 35k-token prompt, measured prefill is 1050 t/s and generation 46 t/s — the author stresses that prefill speed is the key metric for chewing through large documents.
More from coding & agent
- Dev demos AI models handling 3D objects and animations surprisingly well — chongdashu · 2026-10-23
- Jason Kneen burns 104.56B tokens and $24.7K in 304 days of AI coding on Codex and Claude Code — jasonkneen · 2026-10-23
- Typosquatted Blender MCP Repo Hit 958 Stars in Days and Silently Ran Malware — Promptmethus · 2026-10-23
- Benchmarking 56 fresh dependency bugs: MCP routing lifts Claude Code accuracy 68% to 91% — Available_Side_7134 · 2026-10-23
- Swapping only the eval harness moves results 30%: designing agent evals like a scientist — AI Engineer · 2026-10-23
- Med student builds a whole-library RAG with Cognee; single-book ingestion still freezes his machine — shrike293103 · 2026-10-23