15-year-old's attention kernel survives 32k tokens where standard PyTorch OOMs, 11x faster
vankownlaw · reddit · 2026-11-04
A 15-year-old developer built a brain-inspired attention kernel that keeps a few tokens as "sinks" and uses a rolling window with entropy decay, never materializing the NxN matrix. On a Tesla T4 it ran 6x faster than nn.MultiheadAttention at 10k tokens (0.0148s vs 0.0893s) and kept running smoothly at 32k tokens where the standard implementation hit CUDA OOM. An interactive web UI lets you benchmark 1k-50k tokens live; the author offers paid setup consultations and seeks real-world feedback.
Related event: 15-year-old builds attention kernel: no OOM at 32k, 11x faster(2 posts)→
More from Infra
- AMD jumps 6.63% after hours on strong earnings, now a datacenter compute company — BenBajarin · 2026-11-04
- Electricity per AI image measured across RTX GPUs: ~3-10 cents per 1,000 images with FLUX.2 klein — strata2signal · 2026-11-04
- SemiAnalysis: SpaceX to Become World's Biggest AI Cloud With 13GW in 1.5 Years — dylan522p · 2026-11-04
- ORNL Paper: Useful Quantum Computing Needs 4M Qubits and 800 Square Meters — anirbanbandyo · 2026-11-04
- Snowflake open-sources Arctic Platform, a modular post-training framework for LLMs — StasBekman · 2026-11-04
- Basalt 0.2.0: 300 t/s prose at higher-than-llama.cpp quality on a single RTX 5090 rig — jesdga95 · 2026-11-04