Vals-Smith turns your codebase into a custom benchmark to measure which model best resolves your coding tasks
arrakis_ai · x · 2026-07-23
ValsAI introduces Vals-Smith, which converts your merged pull requests into real coding tasks and measures the percentage a model can actually resolve. Public benchmarks tell you which model is strongest overall, not which is best on your code. Vals-Smith tells you which model to trust with your code as new models ship weekly.
More from coding & agent
- Stanford's PaperFormalizer turns research papers into Lean-verified versions with author in the loop — StanfordUniversity · 2026-10-27
- Hugging Face TRL makes score centering + masked importance sampling default in async GRPO — _lewtun · 2026-10-27
- AutoResearch agent's weird training curve gets dug into instead of discarded — ZimingLiu11 · 2026-10-27
- Firecrawl offers 2.5x credits to a16z Speedrun founders, unveils $75M Series B — devdigest · 2026-10-27
- Chums: open-source Apple Watch app puts Claude Code and Codex on your wrist — brandon_galang · 2026-10-27
- Open-source Clairic Studio turns pasted text into typed data and working code in 5 languages — dfinke · 2026-10-27