Mostik's hidden-state "bridge" lets a 4B Qwen match a 753B GLM on ARC-AGI3 at 1/20 cost
量子位 · wechat · 2026-09-07
Mostik, a 4-month-old team of 15 (12 PhDs, Fields medalist Stanislav Smirnov as chief scientist), trains a small "bridge" that passes GLM-5.2's (753B) hidden states directly to a phone-runnable Qwen-3.5 (4B), with no text in between and both models frozen. Insight: models emit 17 bits per token while discarding 2MB of internal state — exactly where the real computation lives (cf. Anthropic's J-space).
Results:
- The 4B model closes 50% of the gap to the 753B model, +25% accuracy; 2x on hard subsets
- The big model only does cheap prefill, skipping expensive decoding — 1/20 inference cost
- Combo nears top scores on ARC-AGI3, beating all text-relay baselines
Next: distillation via teacher hidden states, specialization without losing generality, and using the bridge as a safety observability surface. Full details withheld until the competition ends.
Related event: Startup Mostik Tops ARC-AGI 3 with Latent-Space Model Bridge(3 posts)→
More from Models
- Diffusion LLM bias rankings shift with sampling configs, persist after debiasing — Bollegala · 2026-10-23
- Running ~100 DeepSeek v4.1 flash agents in parallel for under $20/month — zack_overflow · 2026-10-23
- Researcher laments open-source audio models are surprisingly subpar, even representation ones — gowthami_s · 2026-10-23
- User claims they generated video with "Claude Opus 5.5", unannounced model name surfaces — jwt0625 · 2026-10-23
- Security researchers push back as OpenAI restricts Astra access; BlackHat 2027 mocked as 'cyber model' bazaar — cyb3rops · 2026-10-23
- Mistral Large 4 Leaked on Hugging Face: 1T Params, 49B Active, Open Weights Oct 31 — mervenoyann · 2026-10-23