Debating inference-time abliteration vs. downloading 100GB+ uncensored model dumps
pmttyji · reddit · 2026-11-15
A r/LocalLLaMA thread discusses the storage pain of locally hosted abliterated/uncensored models: mid-size models (27-35B) are fine as pre-abliterated GGUF downloads, but large models cost 100+GB each per variant.
- The poster hoards multiple mid-size abliterated models and asks whether big models should use pre-made Abliterated GGUFs or AbliteratedOutTheBox instead.
- AbliteratedOutTheBox applies abliteration at inference time, avoiding a separate download for every model and saving disk space.
- Links to earlier threads on inference-time abliteration and the Strata release are included, plus a request for similar tools.
More from Infra
- Google TPUv7 beats Nvidia B200/GB300 on inference cost in SemiAnalysis benchmarks — SumitGup · 2026-11-15
- Court Refuses to Dismiss DRAM Supply Conspiracy Suit Against Samsung, SK Hynix, Micron — SumitGup · 2026-11-15
- 2009 video shows Jensen Huang sizing CUDA at billions when NVDA traded at $0.2 — GregoryDiamos · 2026-11-15
- Ex-NVIDIA engineer recalls 2012: Jensen Huang backed the deep learning cluster behind Deep Speech — GregoryDiamos · 2026-11-15
- EU's Open Model EUROPA Eyes ~1T Params, Per-GPU Throughput Up 14x — mawcopolow · 2026-11-15
- Buy vs. Rent vs. API for Inference: Utilization Decides Your $/GPU-hour — Panther1537 · 2026-11-15