3 open-weight LLMs auto-screen 15,034 records: sensitivity 1.00, specificity 0.98
maier_ak · x · 2026-11-04
The author built a PRISMA 2020 "living review" on robotic active matter that reruns its searches and screening monthly.
- Method: three open-weight LLMs screen every record with a 2-of-3 majority vote, every vote logged.
- Validation: the author blind-checked a 200-record sample personally: sensitivity 1.00, specificity 0.98.
- Scale: first cycle processed 15,034 records and included 489 studies.
- Motivation: systematic reviews go out of date the day they are published; this pipeline keeps itself current.
Related event: Living Systematic Review Updates Itself Monthly with Open-Source LLMs(3 posts)→
More from coding & agent
- The agent-tool gold rush: sell traces to labs once, sell failed trajectories as RL envs twice — himanshustwts · 2026-11-04
- DataFlow PRs add bad-line isolation and sample-level retries for dataset cleaning — Puzzleheaded_Box2842 · 2026-11-04
- Telemetry spat continues: dev presses Theo on hiding the opt-out in an env var — uwukko · 2026-11-04
- 590 Claude Code sessions, 20B tokens later: a 30-hour browser RPG built almost entirely by agents — Comprehensive-Dig-31 · 2026-11-04
- Theo responds to T3 Code telemetry flap: fully disableable via env var — theo · 2026-11-04
- shadcn hails Claude Opus 5.5 as the first model with 'taste' — shadcn · 2026-11-04