Open-source C-based LLM engine runs Gemma voice conversations on Jetson Orin, faster than llama.cpp
cortexist · reddit · 2026-09-08
A developer demoed a fully open-source two-device voice conversation system: Gemma 12B on an RTX PRO 4500 Blackwell and Gemma E2B on a Jetson Orin NX 16GB (similar performance expected on Orin Nano Super 8GB), both using a reSpeaker Flex 4-mic array and 3W speaker.
Inference is handled by Cortexist Little Gemma, a small LLM engine written in C for CUDA devices — claimed to be faster than llama.cpp on Jetson Orin with no degradation on long voice prompts. The pipeline supports lip sync, expressions, and gestures. Everything is open source.
More from Embodied
- DexPort: a handheld interface with a real robot hand for dexterous-hand data collection — pabbeel · 2026-10-27
- ROS names Alejandro Hernández Cordero new project lead, eyes 2027 release — ___Mufasaa · 2026-10-27
- ROS Meetup coming to Google's NYC office Oct 29 with speakers from Tangent, Cohesive Robotics, Intrinsic — kscottz · 2026-10-27
- Tesla officially expands robotaxi service area across much of Houston, Texas — Polymarket · 2026-10-27
- Walden Robotics teaches robots to call for help, blending remote assist with autonomy — adnothing · 2026-10-27
- Folding-screen hinges: dozens of ultra-precise parts, one grain of sand can kill the panel — BenBajarin · 2026-10-27