Borrowing Capture-Recapture from Ecology to Bound Rubric Performance
sanmikoyejo · x · 2026-10-29
This thread post details VAT's robustness design: how do you know rubrics are maxed out, and whether dense-model scores are real signal or spurious features? VAT discovers metrics, writes prompts for them, detects spurious variables, and estimates asymptotic bounds for each quantity. The key trick borrows population capture-recapture from ecology — given how many metrics have been discovered, estimate how many more remain and what they're worth.
More from Research
- TTME beats test-time training on math discovery with 84% fewer FLOPs — rohanpaul_ai · 2026-10-29
- Pushing one Qwen3-14B neuron turns the model into an 'Earth-Protecting Hero' — Sauers_ · 2026-10-29
- Hugging Face curates must-read blog collection, LLM training playbooks lead — ariG23498 · 2026-10-29
- Can homomorphic encryption make private LLM personalization practical? — burkov · 2026-10-29
- UIST 2026 Best Paper Honorable Mention: Attribution Gradients Unfolds AI Citations Step by Step — lucy3_li · 2026-10-29
- In-context learning lifts LLM music valence prediction from R² 0.11 to 0.43 — affige_yang · 2026-10-29