LG-Bench, a graph-structured life-science LLM benchmark with 10k+ questions, accepted to NeurIPS 2026
geoffwolfe · x · 2026-10-28
LG-Bench, a life-science evaluation benchmark from Shanghai AI Laboratory and collaborators, is accepted to NeurIPS 2026; paper and code are public.
- Motivation: flat question lists ignore the topology of interconnected scientific knowledge, and a single accuracy score hides where a model is shaky
- Scale & construction: 10,000+ high-quality multiple-choice questions across medicine, biology, and chemistry; a weighted evaluation graph is built via bidirectional matching and semantic similarity, with questions as nodes and edge weights as semantic relations
- Two new metrics: Global Coherence Score (GCS) measures answer consistency within semantically related neighborhoods; Knowledge Balance Score (KBS) analyzes error distribution across the graph to reveal conceptual blind spots
- Impact: shifts evaluation from flat accuracy to structure-aware diagnosis of LLM knowledge organization
More from Research
- PDRVLA adds physical dynamics tokens to VLA models, topping LIBERO with 99.3% — Hanqing Wang · 2026-10-28
- HKUST paper: policy secretly encodes harness preferences; return = harness ceiling minus mismatch — HKUSTGZ · 2026-10-28
- Alibaba DAMO releases RynnWorld-Latent: cross-embodiment world model pretrained on 553.5M frames — Alibaba-DAMO-Academy · 2026-10-28
- New Paper Shows Gemini-Found Softmax Bug Fix Still Fails Property Tests — brandondamos · 2026-10-28
- A 9-Paper Reading Path to Understand How Modern LLMs Are Pretrained, Finetuned and Aligned — cneuralnetwork · 2026-10-28
- re:dvx control vector library launches, author claims SOTA innovations — voooooogel · 2026-10-28