Beyond LORAs: soft prompts and GPU-maxing agent swarms proposed as inference cost paradigms

JoshPurtell · x · 2026-11-18

JoshPurtell outlines three paths for LLM inference cost: doing LORA-based inference properly (increasingly unlikely), exploring paradigms like soft prompts / KV (a relevant repo seems undecided but shows promising early results), or using agent swarms to max-load single-tenant GPUs. A reply notes soft-prompts haven't been heard about since 2022 but loves the direction — with autoresearch agents able to push arbitrary setups to SOTA (resin, deepseek-ultrafast), there's no excuse not to have zero-penalty adapted inference regarding batching/interactivity.

Related event: Developers Explore New Paths Beyond LORA for Cheaper LLM Inference(2 posts)→

Original post →

More from Infra

Infra channel →