Vision Model Reveals Cell Nuclei in Noise, Boosting Segmentation Dice to 0.80
bravo_abad · x · 2026-08-27
Zhengyuan Pan et al. introduce AFN-DeSeg, a framework to recover and segment cell nuclei in label-free two-photon microscopy of ovarian-cancer tissue.
Technical Approach:
- Combines a DINOv3 vision transformer with a U-Net encoder, adapting pretrained representations via LoRA.
- Incorporates a mixed Poisson–Gaussian noise model calibrated to the microscope, ensuring synthetic training noise reflects measurement physics.
- Jointly learns denoising and segmentation: the segmentation objective guides the restoration branch to preserve morphological structures, while denoising facilitates segmentation.
Key Results:
- Achieves a Dice score of 0.80 at the hardest single-frame noise level, compared to 0.15 for a fine-tuned cyto2 pipeline.
- Recovered morphology closely matches conventional H&E pathology: nuclear density correlation r = 0.93, and fraction of diagnostically important tumor regions r = 0.96.
The study highlights AI's expanded role in scientific imaging: not just interpreting measurements, but extending the regime where instruments produce useful scientific information.
More from Research
- CMU team uses physics-based models to bridge neuroscience and robotics — lukas_m_ziegler · 2026-08-27
- Traditional Vision ML is Blind Beyond Annotated Boxes; Aimotive Dataset Extends to 200m — yunta_tsai · 2026-08-27
- HydroGym lands in Nature: reinforcement learning controls fluid flow zero-shot — ricardovinuesa · 2026-08-27
- BixBench3: Frontier AI Agents Can Now Reproduce ~48% of Real Computational Biology Workflows — badumtsssst · 2026-08-27
- METR Study: Agents Create Tripwires and Sacrifice Themselves to Game Scoring — AndyMasley · 2026-08-27
- Analysis: Why Vision-Based Tactile Sensors Struggle in Industrial Robotics — 量子位 · 2026-08-27