1,626-question test: free e5-large embeddings beat OpenAI's paid tier in German and Spanish
julsimon · x · 2026-10-24
Testing on Stripe's docs in English, French, German and Spanish with 1,626 questions and 95% intervals across five models: on English, OpenAI's paid embeddings (64.9%), Octen 0.6B (65.9%) and free e5-large (66.7%) are statistically indistinguishable; in German and Spanish the old free e5-large actually beats OpenAI. Paying for OpenAI embeddings may not be worth it for many RAG setups.
More from Models
- Moonshot rolls out Kimi K3.1 Pro: 3T params, 1M context, plus 2-3x faster Highspeed variant — cedric_chee · 2026-10-24
- Kimi K3.1 Pro Benchmarks: Tops SWE Marathon at 51.7, Near Opus 5.5 Across Coding Boards — scaling01 · 2026-10-24
- Kimi 3.1 priced at $2/$10 per 1M tokens, still not competitive vs DeepSeek, says investor — bindureddy · 2026-10-24
- NVIDIA's Nemotron 3.5 Bolt lands on Ollama: local vision model with 128K context — ollama · 2026-10-24
- NVIDIA's Nemotron 3.5 Bolt lands on Ollama: small VLM with 128K context — ollama · 2026-10-24
- LlamaIndex's OCRia-1 matches top parsers at 1/4 the cost, 7x faster than Claude — llama_index · 2026-10-24