Experiment: continued pretraining vs RAG on Qwen 3.5 4B for accuracy and performance
funJS · reddit · 2026-09-12
The author ran a hands-on comparison on Qwen 3.5 4B: continued pretraining (CPT) to internalize domain knowledge versus RAG on the base model, measuring accuracy and performance to quantify internalizing knowledge vs retrieving it on-the-fly. Full findings are in the linked write-up.
More from Research
- Continued pretraining vs RAG: an accuracy and performance comparison on Qwen 3.5 4B — funJS · 2026-09-12
- Researcher speculates spatial reasoning leap comes from Blender training data — yoavartzi · 2026-09-12
- tszzl pushes back on Fermi paper: 50-OOM lognormal abiogenesis prior under-justified — tszzl · 2026-09-12
- Continued pretraining vs RAG on Qwen 3.5 4B: a hands-on accuracy and performance comparison — funJS · 2026-09-12
- OpenAI launches GPT-Rosalind for biological reasoning in API and Codex — OpenAIDevs · 2026-09-12
- MIT quantum lab uses GPT-5.6 Sol and Codex to automate chip measurements — OpenAI · 2026-09-12