SynTalk: open-source gesture-to-speech with dual camera and flex-glove pipelines
garvgoel2005 · reddit · 2026-11-07
A developer details SynTalk, an accessibility project turning hand gestures into text and speech using a webcam (MediaPipe, 21 landmarks), a DIY flex-sensor glove (4 readings), or both.
Key technical decisions:
- Camera samples are 63-dim wrist-relative vectors; glove samples are 4 normalized sensor values — mathematically different, so training data is separated at the source level: one classifier per source, with document IDs embedding the source so a camera "water" gesture can't overwrite the glove's;
- An in-browser KNN classifier votes across the 5 nearest examples by cosine similarity and returns an explicit "I don't know" below threshold;
- Rule-based glove fallback: frames must persist across samples with a cooldown to prevent repeat output;
- React client for interaction, a thin Node server reading serial glove data, Firebase for history, and Hindsight as a cross-session memory layer.
The through-line: making "context stays attached to its source" a property of the data, not a comment.
More from coding & agent
- Scyvera hits 3,264 PyPI downloads, scores your AI agent's blast radius 0-100 — Trout_dev · 2026-11-07
- OpenRouter unveils Auto Router & Auto Code: Opus 5.5-level coding at 74% less cost — AccBalanced · 2026-11-07
- When AI customer service judges your appeal with its own fraud flag, who's accountable? — Creamy-And-Crowded · 2026-11-07
- Teardown: a Claude Code skill that turns any machine into a 3D simulator, 2.4k stars in a week — socialwithaayan · 2026-11-07
- Meta acquires AI coding tool Conductor; cloud data deletion starts Nov 20 — Div_pradeep · 2026-11-07
- Open-source Valet runs your homelab with local LLMs across 185+ services — egfwivbwwruigwrhfbsr · 2026-11-07