Classification problems don't need a general LLM — use classifiers plus text encoders
ziv_ravid · x · 2026-11-20
Ziv Ravid's thread (posts 9-10): after the initial steps it's an iteration loop (train → inspect eval failures → fix data → repeat), and cleaning data usually beats hyperparameter tuning. Model choice varies by problem: for classification, use a classification model or combine one with a text encoder rather than defaulting to a general LLM.
More from coding & agent
- NormGym benchmark shows agents learn from own norm violations but not from others — xuanalogue · 2026-11-20
- Every adopted 'compound engineering' company-wide so its codebase gets easier to maintain — every · 2026-11-20
- Start custom LLMs at 30B: runs on a single GPU and works with LoRA — ziv_ravid · 2026-11-20
- Not every agent task needs a general LLM — specialized small models by task type — ziv_ravid · 2026-11-20
- Before fine-tuning, distill traces into skills in-context to test if training is worth it — ziv_ravid · 2026-11-20
- Fine-tuning is mostly iteration: cleaning data beats tuning hyperparameters — ziv_ravid · 2026-11-20