Open 33B multimodal Agnes-3.0-Flash ships hybrid delta-rule attention with 262K context
Skyline34rGt · reddit · 2026-09-12
Agnes-AI released Agnes-3.0-Flash, an open 33B multimodal model (Artificial Analysis score 36, which ironically lists it as 'proprietary'). It uses a hybrid-attention decoder: 3 of every 4 layers are gated delta-rule recurrent (state independent of sequence length), only 18 of 72 layers hold a growing KV cache.
Key specs: 262,144-token context, 248,320 vocab, 6:1 GQA global attention, fp32 recurrent state, SwiGLU FFN with parallel branch, 3-axis mrope, a 27-layer vision tower for text/image/video understanding, adjustable reasoning effort and tool calling.
More from Models
- Qwenputer: a full RISC-V CPU inside Qwen3.5-4B weights, running C in forward passes — armand_ruiz · 2026-10-31
- OpenAI blocks weapons details for teens, adds auto-ending chats fixated on violence — rohanpaul_ai · 2026-10-31
- Granite Switch 4.1.1 ships: update models like you update software — krvarshney · 2026-10-31
- Mistral Open-Sources Le Chonk: Top Open Model on FrontierCode, Cyberdefense-Tested by CrowdStrike and Cloudflare — tensorqt · 2026-10-31
- Stanford CS336 Lecture Explains Why DPO Compares Against a Reference, Not Two Current Probabilities — stanfordnlp · 2026-10-31
- ik_llama.cpp PR #2808 adds native runtime for China Telecom's Xing4.0-29B-A4B MoE with mHC and MLA — Any_Bus9205 · 2026-10-31