kalomaze: BLT baselines outdated, hnet depth mismatched at same FLOPS
kalomaze · x · 2026-10-29
kalomaze critiques hybrid architecture baselines in an ongoing debate:
- BLT shouldn't be in the baseline class by October 2026 — it's been 2 years, and the "separately trained bolted-on network with ugly cross attention" heuristic has lost its charm
- The hnet baseline effectively has 8+ more layers of serial mamba blocks (true shapes unclear/unreported), so the reallocation substantially changes effective depth — depth isn't matched at the same FLOPS
- They also don't report wall-clock time
Core point: current architecture comparison studies have methodological gaps in depth matching and latency reporting.
More from Research
- Sitting between two data vendors: one sells the benchmark, one sells the hill-climbing data — xeophon · 2026-10-29
- Anthropic study: experts follow their AI agent's lean on 86% of judgment calls — rseroter · 2026-10-29
- New paper challenges JEPA feature-selection theory: unpredictable features can be kept — andreea7b · 2026-10-29
- SREGym to debut at Bench Summit, targeting AI SRE evaluation frontier — tianyin_xu · 2026-10-29
- AA-Robotics benchmark: GPT-6 Astra tops robot-arm control, still ~100x slower than humans — ArtificialAnlys · 2026-10-29
- Delta-Matching closes the FP8 attention gap: all 7 core matmuls in FP8, loss matches BF16 — songhan_mit · 2026-10-29