SBERT2S1 turns biomedical retrieval encoders into calibrated one-pass typed decision models
Pritam Deka · hf · 2026-10-05
New work asks whether biomedical sentence encoders trained for retrieval make good starting points for typed decision models that answer schema-constrained questions in one forward pass. Releases include SBERT2S1 (bi-encoder, cross-head C and prior-fused residual PFR heads), the BIODECIDE benchmark, and MEDLINE-S1 with 243k training decisions. Key findings: retrieval training helps PFR in 10 of 15 comparisons but hurts the C head; C beats PFR under every objective; the open RLCD recipe trails cross-entropy by 2.5–3.0 points due to reward normalization inflating the noisy score-function term 3.6–15×, largely recoverable with an unbiased leave-one-out estimator; after temperature scaling, no objective is clearly better calibrated than cross-entropy. Code, labels and a model are released.
More from Research
- Meta's NAVA-WAM pretrains robot action policies directly from action-free videos — meta · 2026-10-05
- gamfit: open-source Rust engine fits GAMs from a formula with REML-chosen smoothing — Sauers_ · 2026-10-05
- Fine-tuned Llama 3.1 8B for medical decisions hits 84% per-field accuracy, only 30-34% perfect rows — Forsaken_Cut8542 · 2026-10-05
- SJTU's LIFT adds force sensing to VLAs with zero force-labeled pretraining data — jiqizhixin · 2026-10-05
- LOOM stabilizes looped MoEs at 9-12 loops, beating standard MoE at iso-FLOP — SonglinYang4 · 2026-10-05
- Paper accepted at ACML 2026, but author can't afford to present it — Jealous_Key_4030 · 2026-10-05