1.7B model with Width-7 convolutions beat a 12B Qwen on long-horizon reasoning after one SFT run
Thrumpwart · reddit · 2026-10-31
A Reddit experimenter doing RYS layer duplication and geometry tuning on small models found that Liquid AI's LFM3 1.7B — expanded to 3B — beat their best Qwen 3.5 9B (12B after layer duplication, SFT + 8 RL runs + SLERP merges) after a single SFT run of 107 minutes (2,600 steps, 100k samples), versus 14–18 hours per Qwen SFT. The reason: LFM3's Width-7 Convolutions/SWA/GQA architecture, with 20 stacked Width-7 Gated ShortConv layers, aligns with long-horizon state-tracking reasoning, while Qwen's Gated Deltanet is suboptimal. Ironically Liquid AI explicitly advises against using LFM3 for complex multi-step reasoning, and two LLMs said it wouldn't work. Lesson: architecture-task fit can matter more than parameter count.
More from Models
- SenseNova-U1.5-8B hits 125 tok/s on M5 Ultra in llama.cpp and spots a $1 receipt error — Binary_orchid · 2026-10-31
- Pundit: Anthropic killing 5-hour limit plus Fable launch makes it unstoppable — iruletheworldmo · 2026-10-31
- ChatGPT Daybreak Blue sessions may train OpenAI models even with training opt-out off — littmath · 2026-10-31
- Claude refuses to draw its own logo over copyright concerns — 141_1337 · 2026-10-31
- AI cracks a 27-year-old pillar of computer science, OpenAI and Anthropic claim — Fcking_Chuck · 2026-10-31
- Hassabis teases unreleased Gemini 4 at No.1 on Artificial Analysis Index — _YonYonson_ · 2026-10-31