OmniSearch puts text, images, audio, and video into one semantic search space
victorialslocum · x · 2026-07-21
Why unified multimodal search matters
OmniSearch demonstrates a shared semantic space for text, images, audio, and video, so a text query can retrieve audio or video and an image can find related content across modalities.
Key idea
Instead of converting everything into text first, the system embeds raw media directly:
- Podcasts can be searched without transcripts.
- PDF pages can be indexed as images to preserve layout and diagrams.
- Video moments can be found even when narration is vague.
- Retrieved media can be passed directly into a multimodal generation model.
Trade-offs
The post also notes the practical costs: audio and video need chunking, large vectors get expensive fast, and text-only corpora are still cheaper to handle with text embeddings.
Demo and notebooks are linked in the original post.
More from Multimodal
- HeyGen adds a media-sourcing skill for coding agents with 75k images and 10k tracks — HeyGen · 2026-07-22
- Reddit user seeks ComfyUI NSFW text-to-image and image-to-video workflows under 20 GB VRAM — hobbyist2020 · 2026-07-22
- Krea 2 users recommend a two-pass Clownshark sampler setup for sharper image details — listopalafoto · 2026-07-22
- Gemini Omni Flash turns a boat cabin into a cave in Flow by Google — chrisfirst · 2026-07-22
- A simple workflow to turn a photo into an image prompt using Gemini, Grok, or GPT Image — harshitagu72595 · 2026-07-22
- A Reddit user proposes a consistency LoRA to keep anime and game scenes visually stable — ThirdWorldBoy21 · 2026-07-22