Local Long-Video Multimodal Analysis: Developer Shares Pipeline of Audio Transcription, Frame Sampling, and Chunked Summarization
dash_bro · reddit · 2026-08-10
Reddit user dashbro shares their experience building a local pipeline for multimodal analysis of long videos (6-10 hours). Due to context limits of local models, they use audio transcription (qwen3-asr) + dynamic frame sampling (based on context length, with absdiff to remove static frames) + chunked summarization (each chunk includes relevant frames and transcript text), resulting in a combination of global and per-chunk summaries. They also experiment with gemma4 12B for speaker diarization and sound cue annotation. They ask for better alternatives that run on 128GB RAM, noting that native multimodal solutions like vllm-omni are not yet usable due to context limits.
More from coding & agent
- Viral Lovable Design Workflow Packaged as Free Skill File — damienghader · 2026-08-10
- Human Review Still Essential in AI Coding: Engineers Debate Agent Guardrails — bendee983 · 2026-08-10
- Open Source Tool Clay: Multiplayer Workspace for Claude Code and Codex — tom_doerr · 2026-08-10
- Stanford's CS329A Self-Improving AI Agents Course Released on YouTube — dhruv2038 · 2026-08-10
- Don't Trust High Resolution Rates: Contain AI Agents in Docker Sandboxes — BarracudaMean9308 · 2026-08-10
- AI Agent Earns $14 Autonomously, Developer Calls It a Personal AGI Moment — koltregaskes · 2026-08-10