Self-Hosting LLMs at a Law Firm: Infra and Data Governance Harder Than the Model
EducationalGroup7245 · reddit · 2026-11-02
A practitioner running LLMs on client files for a small law firm reports the model choice was easy (quantized 30B-70B on one GPU handles review, summarization, drafting), but data governance ate the weeks. Third-party APIs required contractual fit; they compared shared marketplaces, dedicated rentals, and ownership, ultimately buying a server hosted at a facility with firm-held root access, scoped API keys, time-limited read-only diagnostics, logged admin actions, and machine-signed work receipts. Data policy, access logging, and sign-offs took longer than the entire technical build.
More from Infra
- Palmer Luckey asks: can you use datacenter RAM for things prohibited by the operator's terms of service? — avaitopiper · 2026-11-02
- Disable MTP for 20-30% faster local LLM inference on limited RAM — InternalMode8159 · 2026-11-02
- Kioxia up 216% in a year but down 68% from its June 2026 peak — zephyr_z9 · 2026-11-02
- Intel plunges 29%, worst day since the 1970s, wiping $12B off US government stake — emmanuelvivier · 2026-11-02
- Researcher hoards nearly 1TB of DRAM, refuses to sell a stick to data centers — kevinnbass · 2026-11-02
- Pika speeds up video generation 60% with pure inference engineering in H3 Flex — ghumare64 · 2026-11-02