Jeffrey Emanuel ships agent skill turning audio into spectrogram karaoke videos
doodlestein · x · 2026-10-17
Jeffrey Emanuel released a new AI agent skill, spectrogram-videos-with-karaoke-style-subtitles, that converts any audio/video with speech into an MP4 featuring a scrolling spectrogram and word-by-word karaoke captions in each speaker's color.
- Uses FrankenWhisper for ASR and speaker ID; pitch (35%+ difference) decides who said each line
- Frames drawn with numpy + Pillow piped raw to ffmpeg, so no drawtext/subtitles filter needed
- One transcription powers phrase search over long recordings — it found all 4 occurrences of a phrase in a 30-minute file, with --supercut splicing and --batch clipping
- revoice re-speaks recordings with one TTS voice per speaker (FrankenTTS or macOS say) and transcribes clips back for QA; --bleep bleeps names with automatic checks
- Also supports translated second lines, pitch curves (70–1000 Hz, 89.7% frame agreement with Praat on 24 voices), layout presets, and writes SRT/WebVTT/ASS/LRC/Markdown plus a --plan QA listing
More from coding & agent
- fx v0.0.14 ships durable agents: same API survives crashes on laptop, cloud via Vercel — cramforce · 2026-10-17
- OpenCall playbook: agent swarm does on-call triage with human approval and auto fix PRs — please-dont-deploy · 2026-10-17
- Anthropic launches Claude Haiku 5.5: 75% cheaper, 39.2% on Terminal-Bench vs GPT-6 Luna's 16.4% — dl_weekly · 2026-10-17
- Spotify MCP Server lets AI assistants like Cursor and Claude control playback — modelcontextprotocol · 2026-10-17
- pqs-mcp-server scores any prompt across 8 dimensions before the model runs — modelcontextprotocol · 2026-10-17
- AI Industry Worker Says Nobody He Knows Uses xAI's Grok Build — SimonBalmain · 2026-10-17