Non-Programmer Uses Open WebUI Plus Terminal Container to Search a 500-Page Legal Code
Blindax · reddit · 2026-09-06
A non-developer shares their experience building a local AI document-research workflow, using it to search large documents like a 500+ page tax code and generate notes and slide decks.
- Architecture: Open WebUI runs on Unraid, pointing to the API of a dedicated inference machine (5060 Ti + 5070 Ti); after unsatisfying tries with Hermes and LM Studio, they settled on Open WebUI's official Open Terminal container — a separate container hooked in via Integrations, running unprivileged on a bridge network, so the model gets a shell "inside the box" rather than on the host, keeping things secure.
- Results: The model can reason step by step, reliably locating content in documents far larger than its context window via "list directory → grep → read only the relevant passages," then generating documents from what it finds.
- Setup & performance: Ollama runs Qwen 27B Q4KM with a 100k context; on a 35k-token prompt, measured prefill is 1050 t/s and generation 46 t/s — the author stresses that prefill speed is the key metric for chewing through large documents.
More from coding & agent
- Lightpanda: Open-Source Headless Browser for AI Agents, 11x Faster Than Chrome — Shruti_0810 · 2026-09-06
- A complete AGENT.md: one dev's tuned global ruleset for coding agents — alexcovo_eth · 2026-09-06
- Dev proposes open-source self-healing agent loop: trajectory eval turns failures into skills — EnchantedHawk · 2026-09-06
- Polygres turns your Postgres into a hybrid search context layer for AI agents — Scobleizer · 2026-09-06
- If you can't automate your dev workflow with AI, that's a bad sign — paul_cal · 2026-09-06
- Running Qwen 3.8 27B Locally to Drive Blender via MCP and Build a 3D Llama — jacek2023 · 2026-09-06