H-Net v2: Byte-Level Modeling Gets 3-5x Efficiency Gains, Matches 11T-Token Model with 10x Less Data
SonglinYang4 · x · 2026-10-29
Albert Gu (Mamba co-author) argues intelligence should come from simple, general building blocks that learn abstractions directly from raw data — the bitter lesson. Byte-level language modeling is a microcosm of this ideal: if LLMs can't learn end-to-end from bytes, how will we handle the world's less-understood data?
Key results from the new H-Net (hierarchical networks) version:
- Up to 3-5x efficiency gains in controlled scaling law experiments
- At larger scale, matches an open-source model trained on 11T tokens with 10x less data and compute
Gu frames this as a real step toward AI that can learn from all data.
Related event: Albert Gu argues byte-level hierarchical networks are the next scaling step(2 posts)→
More from Research
- Any3R: one feedforward model handles 13 camera types for 3D reconstruction, beating OmniVGGT and PanoVGGT — zhenjun_zhao · 2026-10-29
- GeoGSplat: geometry priors top DTU and NRGBD with just 65K Gaussians — zhenjun_zhao · 2026-10-29
- SG²SLAM: geometry-informed 3DGS SLAM beats baselines on KITTI and Waymo — zhenjun_zhao · 2026-10-29
- Hexels: hexagonal primitives beat 2DGS by 4.5-7.4 dB at lower VRAM — zhenjun_zhao · 2026-10-29
- StereoSplat: metric-scale novel view synthesis from stereo, AbsRel 0.03 — zhenjun_zhao · 2026-10-29
- Researchers build a Claude artifact to interactively map genomic motifs to transcription factors — jmschreiber91 · 2026-10-29