Cartesia's H-Net shows in-context tokenization: models can learn to chunk data without a tokenizer
TheTuringPost · x · 2026-11-06
The Turing Post covers Cartesia's new H-Net demonstrating "in-context tokenization": models can learn to read without a tokenizer, as context changes how they group information into chunks.
- Given a made-up word, H-Net first splits it into pieces, then groups it more coherently as exposure grows; the same happens for recurring patterns and names in code
- A separate H-Net trained on bacterial DNA picked up on a human gene's structure — and it works for DNA prediction
- Compared to SmolLM3, H-Net needed much less training data for similar results
The bigger question: can systems learn to organize each kind of data on their own? A companion video explains how H-Net works.
More from Research
- Apple's HALO: Fully Autonomous Perceptive Humanoid Motion From Simulation — CyberRobooo · 2026-11-06
- ICLR reviews starting to leak, authors still waiting on scores — Invariant_n_Cauchy · 2026-11-06
- Researcher defends AI-based ranking of 1660 STOC submissions after backlash — vitalyFM · 2026-11-06
- Stanford launches Collaborative Intelligence project on human-agent teamwork for future of work — Diyi_Yang · 2026-11-06
- Chai Discovery launches, open-sources Chai-1, lands deals with seven big pharma firms — chaitjo · 2026-11-06
- Johns Hopkins study compares four failure data sources for robot reward models — mangahomanga · 2026-11-06