Open-Source Project Combines Hand Tracking with Gemini for AI Video Effects
mhdfaran · x · 2026-08-12
Developer sophiamyang open-sourced finger-frame-effect-ai on GitHub, a project enabling hand gesture control over AI video effects.
- Core Concept: Users frame a shot with their fingers; the system tracks the area between the hands and fills it with an AI-generated video. The AI window moves dynamically with the hand gestures.
- Tech Stack: The project integrates MediaPipe for hand tracking with Gemini's video generation capabilities (specifically a video restyling model).
- Generation Modes: It supports restyling the entire video into various styles like 3D animated movies, anime, claymation, and watercolor. The whole clip, including motion and blinks, is regenerated and composited.
Related event: Open Source Project Enables Gesture-Controlled AI Video Effects(2 posts)→
More from coding & agent
- Grok 4.6 available in Cursor and API with 2x usage for the first week — Daniel_Farinax · 2026-08-12
- Ref Raises $4M to Fix 'Velocity Sickness' in AI Coding Agents — round · 2026-08-12
- Practical Tip: Run LLM Agent Experiments Free via Kaggle CLI with T4 GPUs — mariofilhoml · 2026-08-12
- Dev Changes 'Proceed' Button to Encouraging Text to Please AI Agents — doodlestein · 2026-08-12
- DeepLearning.AI and JetBrains Launch Free Short Course on AI Coding Workflows — DeepLearningAI · 2026-08-12
- mini-swe-agent is the Best Harness for NVIDIA Nemotron on Terminalbench — OfirPress · 2026-08-12