F2LLM + Zerank 2 Top Local RAG Pipeline in 15-Language Benchmark
seamonn · reddit · 2026-08-09
The author extensively tested various local Embedding and Reranker combinations to build a translation memory server supporting 15 languages.
- Winning Combo: F2LLM V2 (4B/8B) paired with Zerank 2 (4B) achieved the highest MRR (0.92) and Recall@20 (99.2%), beating commercial APIs like Voyage and common local pairs like BGE M3.
- Model Insights: F2LLM V2 is hailed as the current SOTA fully open-source embedding model (including data and code). Zerank 2 is a top-tier reranker recently fully open-sourced after Notion acquired Zeroentropy.
- Caveats: Despite dominating leaderboards, Qwen 3 embedding/reranker models performed mediocrely in this real-world cross-lingual task. The 8B F2LLM offers only marginal improvements over 4B, making the latency tradeoff unjustified.
More from Models
- willcb: Opus 5.5 doesn't smell bigger, it smells like they fixed the data — willcb · 2026-10-16
- Unverified claim: Apple and Meta's 816B-parameter Muse Spark Open runs on two Mac Studios — TansuYegen · 2026-10-16
- Gemini 4 model card: model eliminated months of trial-and-error on bioweapons research — Puzzleheaded-King584 · 2026-10-16
- Every AI picks 7: Anthropic engineer's rule for escaping model taste convergence — rubenhassid · 2026-10-16
- Brain teaser: strip all attention, MLP and embedding matrices from a Llama 3 — how many trainable params remain? — Sauers_ · 2026-10-16
- Lean v4.35.0 ships verified kernels — one written and proven consistent by Claude — geoffreyirving · 2026-10-16