Quasar Shows Mid-Training Benchmark Improvements
const_reborn · x · 2026-07-14
Quasar shared updates on its current training run: training is only about 10% complete, but several benchmarks already show significant improvement.
Key Changes
- MMLU: 68.40% → 69.01%
- MMLU-Pro: 33.20% → 33.72%
- GPQA: 25.60% → 26.01%
- ARC Challenge: 63.00% → 64.10%
- ARC Easy: 80.10% → 83.96%
- HellaSwag: 74.00% → 74.79%
- MATH-500: 71.40% → 72.26%
A few metrics saw a slight drop, such as PIQA and OpenBookQA. The team notes that training has only been running for two weeks with limited compute power; results are expected to improve further as more compute is added.
More from Models
- NVIDIA says Nemotron 3 Ultra scored 30/42 on the 2026 IMO problems — NVIDIAAI · 2026-07-22
- OpenAI is reportedly briefing U.S. lawmakers on its next model family — kimmonismus · 2026-07-22
- Muse Spark 1.1 lands at 1495 on Text Arena with standout agentic-coding price performance — ycombinator · 2026-07-22
- Advanced AI Models Are Becoming Impossible to Plug and Play — emollick · 2026-07-22
- Google Gemini's AI Problem: No Leading Model for Core Workloads — bindureddy · 2026-07-22
- Model Offers 1M Token Context Window at Just $0.33/1M Tokens — MickeySteamboat · 2026-07-22