Ling 3.0 Tiny vs Gemma 26B-A4B: 3x Smaller VRAM, But Accuracy Halved

autonoma_2042 · reddit · 2026-09-21

In an audiobook pipeline (attributing dialogue speakers for per-character TTS), a developer ran an A/B test of Ling 3.0 Tiny (7.9B total / 1.3B active, 4.8GB at Q4, fits fully in 8GB VRAM) against the incumbent Gemma 4 26B-A4B (QAT Q4, 14GB, experts in system RAM):

• Hardware: NVIDIA T1000 8GB (no tensor cores) + Ryzen 5 7600 / 93GB DDR5
• Ling notes: hybrid KDA/MLA attention, 128 experts with 8+1 active; tiny KV cache (54MiB at 8K), 4.9GB resident; llama.cpp master supports bailingmoe3 natively
• Throughput: 245 tok/s prefill, 54 tok/s decode on a 32-quotation request
• But Ling writes 50–70% more reasoning text, overflowing 8K context more often, and the checkpoint ships no NEXTN weights, so no speculative decoding

Results: on 485 hand-corrected lines, Gemma scored 94.6% overall (worst chapter 90.6%) vs Ling's 56.9% (worst 43.8%), with the gap widening on longer chapters; Gemma was also faster wall-clock.

Takeaway: Ling 3.0 Tiny's memory and throughput advantages don't translate to quality on strict JSON extraction — the 26B MoE wins decisively.

Original post →

More from Models

• LLMs still struggle at niche-domain labeling: a cluster-then-LLM pipeline with brittle second-pass labels — FanaHOVA · 2026-09-21
• A "decide" Primitive: Browser On-Device AI as a Native JS Control Flow — cocktailpeanut · 2026-09-21
• Rumor: GPT-6 Sol Lands Tuesday, Internal 'Bel' Deemed AGI; Opus 5.5 May Drop Monday — imjustnewatai · 2026-09-21
• Jev underperforms: researchers find better alternatives to GLiNER2-class extractors — airesearch12 · 2026-09-21
• Nym rebuilds its agent around Jev, a fast classifier model, for speed and cost gains — moyix · 2026-09-21
• StarCraft Returns as a General AI Benchmark — Elo Scores as Legible Intelligence Indicators — teortaxesTex · 2026-09-21

Models channel →