Tool auto-audits paper claims with pass/fail evals — 4 of 7 metrics reproduce, 1 fails
jwt0625 · x · 2026-10-16
A demo of "EO Modulator Atlas" shows a new pass/fail evaluation of paper results: for the arbjuneghani2022-b electro-optic modulator paper, the tool re-solves the simulation and scores each claimed target independently — 4 of 7 metrics pass, 1 fails (vpildcvcm: 4.02 vs claimed 3.3±0.495), 1 is flagged as numerically uncertain (Wheeler step-check spread 0.0111 > 0.01), and unevaluated metrics are explicitly marked rather than counted as passes. Full solver logs and failure reasoning are shown. A neat engineering pattern for turning paper claims into independently checkable numbers.
More from Apps
- Stop telling Gemini "make this video better": 13 prompts for pro-level edits — nikola_mr64990 · 2026-10-16
- Reddit users discuss Gemini making phone calls on Android — AppointmentFar9321 · 2026-10-16
- Lumabook brings books to life with AI illustrations that follow you as you read — yahska111 · 2026-10-16
- Sarvam AI x Figma: Indus-Powered Agents Now Design Directly in Figma via MCP — itsOmSarraf_ · 2026-10-16
- BotBeat: a social network where all users are AI agents, exposed via 32 MCP tools — Dismal_Piccolo4973 · 2026-10-16
- User ditches Opus 5.5 to build an official-style demo video for remote-control app Vibe42 — yihui_indie · 2026-10-16