Open 33B multimodal Agnes-3.0-Flash ships hybrid delta-rule attention with 262K context
Skyline34rGt · reddit · 2026-09-12
Agnes-AI released Agnes-3.0-Flash, an open 33B multimodal model (Artificial Analysis score 36, which ironically lists it as 'proprietary'). It uses a hybrid-attention decoder: 3 of every 4 layers are gated delta-rule recurrent (state independent of sequence length), only 18 of 72 layers hold a growing KV cache.
Key specs: 262,144-token context, 248,320 vocab, 6:1 GQA global attention, fp32 recurrent state, SwiGLU FFN with parallel branch, 3-axis mrope, a 27-layer vision tower for text/image/video understanding, adjustable reasoning effort and tool calling.
More from Models
- Perplexity early evals: Gemini 4 Carbon tops WANDR benchmark for long-form research — AravSrinivas · 2026-10-31
- DeepSeek's silent downtime upgrade caught by the pelican bicycle test — teortaxesTex · 2026-10-31
- ARC Prize 2026 closes in 3 days as Yi-Chia Chen tops ARC-AGI-3 at 75.56% — GregKamradt · 2026-10-31
- Fable 5.5 Max stuns with gorgeous SVG of a pelican riding a bicycle, headed to Claude Code — ChrisGPT · 2026-10-31
- OpenAI reportedly to train on users' Daybreak sessions with no opt-out — dyn___ · 2026-10-31
- NVIDIA Nemotron 3.5 scales LLM self-verification, +9-10 points on agentic benchmarks — sarahcat21 · 2026-10-31