Packing four loci per uint8: a memory trick that slashes DNA sequence encoding by 4x
jmschreiber91 · x · 2026-10-20
The author shares a low-memory encoding trick for DNA sequence analysis (e.g. predict, deepliftshap):
- Step 1: store loci instead of one-hot; read them straight from disk into a packed sequence to save memory and allocation time, decoding each batch back to standard one-hot
- Step 2 (a technique the author says is rarely seen): a uint8 storing values 0-3 wastes 6 bits — pack multiple positions into the same uint8, representing four adjacent positions per uint8 for another 4x memory saving
More from Research
- Gojo project generates 108 ornament designs from 36 skeletons with hierarchical pattern-aware coloring — debreuil · 2026-10-20
- Paper: 'Before They Can Solve' predicts a base model's post-training potential early — hyunw_kim · 2026-10-20
- Base-model probes predict cross-benchmark coding-agent potential at ρ=0.976 — hyunw_kim · 2026-10-20
- Three probes track post-training SWE-bench rankings, BPB hits ρ=0.964 — hyunw_kim · 2026-10-20
- NVIDIA asks: must you post-train every base model to find the best coding agent? — hyunw_kim · 2026-10-20
- SaTML got 450 submissions, 2x last year — chairs early-rejected ~40% of them — m2saxon · 2026-10-20