Agentic SfM: Training MLLMs with RL Toolchains to Boost 3D Reconstruction
gabriberton · x · 2026-07-04
Researchers propose the concept of "Agentic SfM": training Multimodal Large Language Models (MLLMs) via reinforcement learning to use tools like image cropping, image matching, and COLMAP to solve 3D reconstruction challenges in complex scenes. Current image matching models perform poorly on low-overlap image pairs, whereas MLLMs (like QwenVL) can effectively distinguish between scenes with similar appearances (doppelgangers). Because the task is verifiable and supports curriculum learning sorted by difficulty, training costs are relatively low, making the overall approach highly feasible.
More from coding & agent
- GPT-6 Astra beats Factorio with enemies in 44 in-game hours at ~$4,500 API cost — liminal_bardo · 2026-09-11
- 105 hidden bugs, 2 repos: DeepSeek V4.1 Flash fixes 24 at $1.80 vs Opus 5's 27 at $51.33 — ChartsJournalX · 2026-09-11
- Investment Analyst Asks How to Build a Claude-Based Diligence Agent Stack — Careless_Tie2286 · 2026-09-11
- Treating agents like 50 First Dates: a 3-layer context system so every conversation doesn't start from zero — evielync · 2026-09-11
- Running the Firefox MCP on Android via Termux, ngrok, and mcp-proxy — Nervous-Strain7544 · 2026-09-11
- SmolVM open-sources persistent computer infrastructure for agents that outlive chat sessions — aniketmaurya · 2026-09-11