Whisper base plus an 11M-param distilled BERT caption videos entirely in the browser

Pantheon3D · reddit · 2026-11-17

A Blender short-form tutorial creator built a captioning tool that runs entirely in-browser, no cloud uploads: transcription uses transformers.js whisper-basetimestamped (WebGPU fp32 on real GPUs, WASM with q8 decoder elsewhere — software WebGPU adapters turned out slower than WASM). The caption tagger (word emphasis, line breaks, punctuation/casing) evolved from hand-written rules to an 11M-param BERT distilled from a 2B model (8 labels/token, 12MB, q8 WASM). Training data came from LFM3-Omni-2B via llama.cpp at 190 scripts/min on a 4070 Ti Super with 128 parallel slots; the model only replaced rules after beating them on held-out data. Cloud fallback exists for noisy audio; tagger is English-only. Author is hunting for anyone running whisper small/turbo at usable browser speed.

Original post →

More from Apps

Apps channel →