Skip the GB-heavy abliterated models: a tiny rank-1 LoRA uncensors at runtime

returnity · reddit · 2026-11-16

A Reddit PSA: stop downloading hundreds of GB of abliterated models—most abliteration techniques are rank-1 ablations whose refusal projection fits in a tiny LoRA you can apply at runtime to a quant you already have.

Method and data:

Usage (llama.cpp):

bash

llama-server -m Qwen3.8-Flash-Next-UD-Q5KXL...gguf \

--lora HUIHUI-V2-...-LoRA-F32.gguf --lora-init-without-apply -fa on --jinja

Adding a lora field per-request enables on-demand switching: serve the unmodified model, and only enable the LoRA on refusal and resend. The author suggests a pi extension to automate this (at the cost of a full re-prefill). The technique also works in Strata and ds4/dwarfstar.

Original post →

More from Infra

Infra channel →