Community-built open-weight LLM leaderboard with human expert evals calls for contributors

EmilPi · reddit · 2026-10-10

A Redditor launched beta.locallm.top, a community leaderboard for open-weight models with a distinctive methodology:

• Human evaluation only: domain experts (mostly the author's friends) score answers; coding is evaluated solely by the author; deliberately no LLM-as-judge;
• Multidimensional: includes coding and agentic benchmarks, filterable by domain/language/intent; users who submitted private queries see different results;
• Local hardware: everything runs on one 2xRTX 3090 + 128GB RAM PC, open to anyone creating queries and rating answers; agentic runs are slow to appear;
• In progress: expanding agentic coding, one-shot coding, agentic retrieval and agentic story-writing benchmarks; recently added Qwen3.8-27B (Uncensored), with GLM-5.3-Flash-UD-IQ3XSS and Mellum2.1-12B-A2.5B queued;
• Call for help: recruiting compute, evaluators and developers.

Caveats admitted: imbalanced domain coverage, incomplete classification, rough UI, focus on small models and low quants.

Original post →

More from Models

• ChatGPT web can actually listen to audio, but the same model can't via API or Codex — Possible_Statement84 · 2026-10-23
• MiniMax M3.1 Flash free for 3 days on 0G, with 1M context and 512K output — MiniMax_AI · 2026-10-23
• TLA+ expert corrects Grok: TLC only exhausts finite state spaces, not all behaviors — tianyin_xu · 2026-10-23
• OpenAI blocks manual reset with 13% usage left: 'no reset needed right now' — AccomplishedSugar490 · 2026-10-23
• Meta's 816B Muse Spark Open scores 39, losing to 180B Qwen3.8-Flash-Next at 40 — BTA_Labs · 2026-10-23
• Gemini refuses to generate image in Docs, blames document size limit — viksit · 2026-10-23

Models channel →