EmbeddingGemma 2 runs real-time embeddings on a MacBook CPU alone: 20ms text, 100ms images
generativist · x · 2026-10-28
Developer mmastrac demoed a real-time /v1/systemone embeddings endpoint built on Google's EmbeddingGemma 2: 20ms on text and 100ms on images, trainable in real time, able to distill an upstream teacher model, and multimodal by default. The kicker, as the author and retweeter both stress: it runs entirely on a MacBook's CPU — no GPU, no Metal, no Neural Engine. With the right harness, a model this size performs remarkably well.
More from Infra
- Hyperbolic raises $30M Series A1 to make fragmented GPU compute easier to use and trade — qinzytech · 2026-10-28
- Reflection AI runs 1.3B sandboxes in largest RL run, peaking at 883K vCPUs — zacharynado · 2026-10-28
- GPU capacity bust more likely than a bank run, argues AI compute cycle thread — bookwormengr · 2026-10-28
- Analyst: AI constraint map now tracks 50 components as everything hits bottlenecks — BenBajarin · 2026-10-28
- AXA Uses NVIDIA Earth-2 to Simulate 1,000+ Cyclone Seasons for Storm Risk Modeling — NVIDIAAI · 2026-10-28
- 2026 wafer fab equipment forecast raised to $153B as the cycle lengthens — BenBajarin · 2026-10-28