Tsinghua's Tianmouc Chip Featured on Nature Sensors: A Self-Supervised Framework to Overcome Real-World Visual Degradation

机器之心 · wechat · 2026-08-11

The latest advancements in Tsinghua University's brain-inspired "Tianmouc" chip have been featured on the cover of Nature Sensors. While previous work established the hardware foundation of complementary vision, this new research marks a leap in system-level algorithms and software ecosystems.

Tackling Visual Degradation

Open-world AI suffers from complex visual degradation like motion blur and overexposure. Lacking ground truth, traditional supervised learning struggles here. The team proposed a self-supervised representation learning framework that learns directly from degraded data without ground-truth annotations.

Two-Stage Learning Paradigm

Bottom-Up Internal Model: The algorithm embeds visual priors into intermediate representations, using cross-pathway attention to preserve reliable cues and infer missing structures. It forms a universal internal model (IGFNet) that decodes estimated visual ground truth (e-VGT).
Top-Down Task Empowerment: The visual knowledge is fed into high-level tasks, significantly improving monocular depth estimation and video instance segmentation in extreme environments.

Open Source & Commercialization

The team released the algorithm toolkit TianMouCV, which has already validated its effectiveness in top-tier conferences (CVPR, ICCV) across tasks like deblurring and visual odometry.
The spinoff company Xijian Tech is commercializing the tech. By extracting adaptive visual tokens, it reduces redundant info by over 90% and cuts the first-token latency of multimodal LLMs by 20x, transforming sensors into an intelligent input layer for foundation models.

Original post →

Tsinghua's Tianmouc Chip Featured on Nature Sensors: A Self-Supervised Framework to Overcome Real-World Visual Degradation

More from Embodied