Stanford's WHALE paper alternates tuning LLM weights and harness, raising reward-hacking fears
teortaxesTex · x · 2026-09-13
A new Stanford paper tackles a common blind spot in LLM evaluation and tuning: practitioners either burn compute fine-tuning weights with a frozen harness, or hand-craft agent scaffolds around a black-box model — and the two bottleneck each other. Better weights can't fix retrieval logic that misses documents, and clever control flow is useless if the base model isn't sharp enough.
The authors propose WHALE, which alternates between the two: tuning weights against the current harness via rejection sampling, then iterating the harness against the updated weights, so model and scaffold co-evolve.
Commentator teortaxesTex flags a worrying direction for anyone concerned about reward hacking: you may end up with both a slop model and a slop harness optimized to look good together.
More from Research
- Proxy Policy Steering adapts frozen VLA models to new tasks at inference time — weichiuma · 2026-09-14
- New research: standard SGD matches AdamW for LLM RL training, with far less memory overhead — zhaoran_wang · 2026-09-14
- RSI work separates practical harness self-improvement from unproven intelligence explosion — arthurcolle · 2026-09-14
- Amazon Proposes Query-Aware Index Pruning to Optimize Retrieval Under Budget Constraints — _reachsumit · 2026-09-14
- New Paper Finds Retrieval Signals Give No Reliable Routing Gain in Adaptive Multimodal RAG — _reachsumit · 2026-09-14
- Google: Graph RAG Cuts API Hallucination Rate from 56.4% to 16.2% in Java-to-Python Migration — _reachsumit · 2026-09-14