Study Warns: Perplexity Metrics May Mask Pruning Damage
cwolferesearch · x · 2026-07-06
Commenting on a pruning work, researcher cwolferesearch pointed out that using perplexity for evaluation can sometimes "hide" the damage inflicted on the model by pruning, making it crucial to carefully select evaluation metrics. They emphasized that "cherry-picking evaluations" is a universal lesson applicable across all research fields.
More from Research
- Cognition's SWE-2 uses a KKT duality argument in RL to shift the effort Pareto curve — YouJiacheng · 2026-09-11
- VidMap uses RoMa coarse matching on all frames, fine-scale only for keyframes — ducha_aiki · 2026-09-11
- Bug Hunt Bench author: leaderboard noise is about 2-3 points — PawelHuryn · 2026-09-11
- PNAS paper shows a tiny billiard-ball system is a universal computer — undecidability lives in two dimensions — eigensteve · 2026-09-11
- New paper: Absolute pose estimation from affine cues and gravity direction — ducha_aiki · 2026-09-11
- LoMa Paper Ships REALLY HardPairs Dataset, Accepted at ECCV 2026 — ducha_aiki · 2026-09-11