EmbeddingGemma2 on CPU matches OpenAI's embedding-3-large in 96-config retrieval benchmark
tomaarsen · x · 2026-10-14
Developer dorkitude benchmarked Google's new EmbeddingGemma2, which runs on a CPU, against leading hosted models: it's comparable to OpenAI's embedding-3-large and Qwen 8B. The open-source study (Go/Cobra + turbopuffer) completed 96/96 configurations across six models with 76,704 primary query evaluations — and was itself one-shotted to an agent from a phone. Replying, tomaarsen adds a key caveat: never use closed-source embeddings for search, since deprecation forces a full reindex of all docs.
More from coding & agent
- After 3 years, HTMS ships first client project compiling HTML into JavaScript — CtrlAltDwayne · 2026-10-14
- Codex tells its human to back off while it takes over a Premiere export — joonasvirtanen · 2026-10-14
- Dan Vega's dev2next talk: a 68-slide roadmap from Claude Code beginner to power user — therealdanvega · 2026-10-14
- Dev builds /sitrep skill to summarize multi-agent project state in one command — kevinkern · 2026-10-14
- Beancat launches an AI agent for media production with transparent per-run dollar pricing — Scobleizer · 2026-10-14
- Two devs burn $2,025 in API costs building a game with Claude, screenshots show progress — zeeg · 2026-10-14