8B finetune experiment: light overfitting starts around 3 epochs per phase
kalomaze · x · 2026-10-31
kalomaze shares empirical curves from finetuning an 8B model: running 3 epochs for both training phases shows light overfitting typically starting at around 3 passes in his experience — the green eval/loss curve bottoms out near step 15.5k and climbs back up, while pre-overfitting runs keep descending below 0.80.
He also notes the eval set is deliberately mixed-quality for diversity/coverage. A practical epoch-count reference for anyone finetuning models.
Related event: kalomaze's LoRA fine-tuning tests show overfitting begins around epoch 3(3 posts)→
More from Research
- NVIDIA Nemotron 3.5 scales LLM self-verification, +9-10 points on agentic benchmarks — sarahcat21 · 2026-10-31
- Unofficial OpenTargets MCP Server Brings Gene-Drug-Disease Data to LLMs — modelcontextprotocol · 2026-10-31
- Researchers find 5 personality structures in LLM weight space, Big Five-like — luke_drago_ · 2026-10-31
- NVS benchmarks are misaligned: proper pose alignment puts 3D Gaussians back at SOTA — kwangmoo_yi · 2026-10-31
- arXiv moderator: submission surge causes months-long holds, new moderators wanted — tdietterich · 2026-10-31
- arXiv policy checks reject class projects, proposals, surveys and position papers — tdietterich · 2026-10-31