Never run EmbeddingGemma 2 in FP16: use BF16 or FP32 to avoid NaNs
tomaarsen · x · 2026-10-07
tomaarsen warns that EmbeddingGemma 2 must run in BF16 or FP32, not FP16: the model's activation range exceeds FP16's dynamic range, and the model card warns of NaNs or silently degraded embeddings. BF16 is recommended where natively supported; FP32 elsewhere, including most CPUs. Sentence Transformers users: pip install -U sentence-transformers transformers.
More from Models
- llama.cpp ships Day-0 support for Google's EmbeddingGemma 2 — ggerganov · 2026-10-07
- Trying to Plug Open-Source Mistral Into an Agentic Coder Just Doesn't Work, Says Berman — MatthewBerman · 2026-10-07
- antirez: DeepSeek v4.1 outscores Mistral Large 4 on DeepSWE 1.1 and other benchmarks — antirez · 2026-10-07
- StartLux claims its 27B model beats Jev AI on 31 of 38 benchmarks, self-reported results — Dr_Singularity · 2026-10-07
- 24 models tested on 669 clinical decisions: Jev stays #1 as two free models close in — MaziyarPanahi · 2026-10-07
- EmbeddingGemma 2 dim tradeoff: 128d shrinks vectors 6x but MMEB drops to 45.65 — tomaarsen · 2026-10-07