Learned n-grams compress indexing to group tokens like chars into words
antirez · x · 2026-08-30
N-grams operate at a higher level similar to grouping characters into words (e.g., 'apple'). Since bi- and trigrams are too numerous, a compressing hashing is used for indexing.
More from Research
- A missing parameter in scaling laws: H, the human effort behind learning signals — lateinteraction · 2026-10-16
- One ancient fold, many jobs: how the immunoglobulin domain evolved into antibodies, MHC and titin — arjunrajlab · 2026-10-16
- Researcher Argues Conferences Should Desk-Reject Far More Papers as Flooding Strains Reviews — boknilev · 2026-10-16
- Universal forgery attack breaks Chinese PQC signature Origami using only the public key — matthew_d_green · 2026-10-16
- How do you attribute agent gains: model training or infrastructure? — jrxna · 2026-10-16
- First-ever paper on multi-agent swarm safety lands NeurIPS spotlight — LauraRuis · 2026-10-16