kalomaze calls out hnet baseline: effective depth unmatched, no wall clock reported
kalomaze · x · 2026-10-29
kalomaze raises technical objections to an architecture comparison using an hnet baseline: the baseline appears to have effectively 8+ more layers worth of serial mamba blocks (true shapes unclear/unreported), so the parameter reallocation substantially changes effective depth — meaning depth is not matched at equal FLOPS.
He adds that a fixed-size unit of computation is preferable to one that changes across training, a known weakness of hnet that makes fair comparison at equivalent bpb + FLOPS difficult, and criticizes the missing wall-clock numbers.
More from Research
- Google DeepMind engineer shows five CUDA optimizations that speed up 3D Gaussian Splatting by 10.5x — janusch_patas · 2026-10-29
- AI Audit Finds 2,035 Errors in 138 Papers by Economists Saez and Zucman — sebkrier · 2026-10-29
- Granary-v2 open speech model expands to 48 languages with open data and pipeline — rdesh26 · 2026-10-29
- Tinker launches Dimuon, a Muon-based optimizer that beats AdamW on LoRA fine-tuning — xennygrimmato_ · 2026-10-29
- Six CUDA kernel optimizations squeeze another 1.78× speedup out of 3DGS training — gkopanas · 2026-10-29
- Cross-lingual LLM effects paper wins Outstanding Paper award at EMNLP 2026 — anas_ant · 2026-10-29