kalomaze's LoRA fine-tuning tests show overfitting begins around epoch 3
Researcher kalomaze shared empirical LoRA fine-tuning results on an 8B model, showing mild overfitting typically begins around the third epoch. Across roughly 90M tokens of training data, multi-epoch training with WSD decay performed best.
2026-10-31 ~ 2026-10-31 · 3 related posts
- kalomaze shares LoRA finetuning loss curves: 3 epochs plus WSD decay beats 1-epoch regime — kalomaze · 2026-10-31
- kalomaze's LoRA finetuning curves show overfitting typically starts at 3 epochs — kalomaze · 2026-10-31
- 8B finetune experiment: light overfitting starts around 3 epochs per phase — kalomaze · 2026-10-31