Trillion-Parameter Models Abandon Transformers as Architectures Diverge
imjustnewatai · x · 2026-08-26
Recent breakthroughs highlight a shift away from traditional Transformer architectures:
- Trillion-Parameter Neural Operator Model: Completely removes the Transformer architecture. It uses neural operators to learn continuous physical fields across space and time, capable of ingesting over 5 trillion input elements and generating an entire 4D trajectory in one shot.
- Pathway's Dragon Hatchling: A tiny 1.5M parameter model using recurrent latent reasoning and Hebbian memory. It updates internal state from demonstrations and scored 29.5% pass@2 on public ARC-AGI-1 at a cost of $0.00070 per task.
- Core Automation: Led by the teams behind o1/o3 and Gemini pretraining, this initiative aims to meta-learn long-horizon learning directly into the architecture, allowing models to keep adapting after deployment.
Related event: Ex-NVIDIA Scientists' Startup Launches Neural Operator Physical AI Model(13 posts)→
More from Models
- Reasonable post-trains Nemotron 3.5 on 3,000 Verus proofs, lifting pass@1 ~30 points to 73.3% — fhuszar · 2026-10-16
- NVIDIA's Nemotron 3.5 Super VL hits CoreWeave: 12B-active multimodal MoE at $0.25/M input tokens — wandb · 2026-10-16
- Opus 5 might be a very deep model or use looping, commentator speculates — scaling01 · 2026-10-16
- GPT-6.1 Sol audit finds 6% of arXiv papers had false main theorems, down to 1.5% recently — nitarshan · 2026-10-16
- Nvidia-backed post-training pushes Nemotron 3.5 VL past frontier on dermatology diagnosis — LoganGrasby · 2026-10-16
- Gemini 4 Argon scores 97.5% on ARC-AGI-3, second lab above 95% — patience_cave · 2026-10-16