TensorAgent Adds Local Multimodal Indexing for Documents, Images, Audio and Video
fuzhongkai · reddit · 2026-11-01
The author of TensorSharp describes TensorAgent's local knowledge base, which pulls documents, CSVs, images, audio and video into a searchable collection while keeping links back to source files.
How indexing works
- Documents and CSVs contribute extracted text and structured content.
- Images contribute descriptions and visible text via a compatible vision model.
- Audio contributes transcriptions via a compatible audio model.
- Video contributes information from sampled frames; the soundtrack is not indexed.
Retrieval defaults to BM25 with optional embeddings; the author is explicit that this is not all modalities embedded into one shared vector space. Coverage is explicit: media extraction can be partial, indexing a file does not mean every detail was understood, and collections must be enabled per conversation.
From index to task
A walkthrough uses a fictional café campaign where a PDF says 15% off and a poster says 20%. The workflow retrieves the conflicting evidence, opens the full sales CSV for calculation, and produces a source-linked action plan. Search locates evidence; access to the original file lets a calculation use the complete dataset.
The 83-second video uses the real UI, synthetic data and scripted responses — not a live model run or benchmark — and the calculations were independently checked in Python.
More from coding & agent
- Patrick Collison's 'liquid software' experiment: freeform user feedback auto-implemented by agents, 150 improvements in days — zeeg · 2026-11-01
- pi-rs: Rust port of pi coding agent swaps in rquickjs for up to 10x faster performance — jasonkneen · 2026-11-01
- Claude Managed Agents docs: how to wire Skills, tools and MCP servers to agents — cfregly · 2026-11-01
- Anthropic's Managed Agents Can't Import Full Plugin Bundles Yet, Engineer Confirms — cfregly · 2026-11-01
- Dev: Lean is incredibly useful for keeping LLMs from being silly in math — willcb · 2026-11-01
- Subvocal team details how they built the 'future of human-computer interaction' — Scobleizer · 2026-11-01