Start custom LLMs at 30B: runs on a single GPU and works with LoRA
ziv_ravid · x · 2026-11-20
Ziv Ravid's thread (posts 11-12): across all these cases the idea is the same — train smaller, more efficient models on your own data. For an LLM he'd start at 30B: it still runs on a single GPU and you can run LoRA on it. Small models specializing in agentic tasks are also worth handing limited task sets to.
More from coding & agent
- NormGym benchmark shows agents learn from own norm violations but not from others — xuanalogue · 2026-11-20
- Every adopted 'compound engineering' company-wide so its codebase gets easier to maintain — every · 2026-11-20
- Not every agent task needs a general LLM — specialized small models by task type — ziv_ravid · 2026-11-20
- Before fine-tuning, distill traces into skills in-context to test if training is worth it — ziv_ravid · 2026-11-20
- Fine-tuning is mostly iteration: cleaning data beats tuning hyperparameters — ziv_ravid · 2026-11-20
- Classification problems don't need a general LLM — use classifiers plus text encoders — ziv_ravid · 2026-11-20