Deepity: hand-tuned C++/CUDA predictive coding library beats jpc and ngc-learn on MNIST
Important-Home4431 · reddit · 2026-10-27
- The author released Deepity 0.1.0a3, a from-scratch predictive coding (PC) engine: a C++20 core with hand-written AVX2/AVX-512 SIMD kernels, a thin Python layer via nanobind, installable with pip install pydeepity.
- Algorithm: each layer in a PCN holds a local belief compared against top-down predictions; training alternates 'settling' (layers locally reduce prediction error) with weight updates using only local error — no backward graph, no weight transposes.
- MNIST benchmark (784-512-512-10 sigmoid, T=20, batch 250, 100 epochs, AdamW): Deepity 98.21% in 487.5s; jpc (JAX) 97.55% in 531.4s; ngc-learn 98% in 1707.7s. A PyTorch backprop reference gets 97.51% in 627.8s (one pass vs 20 settling passes, not a fair comparison). Kernel benchmark sustains 690 GFLOPS/s; end-to-end 675 GFLOPS/s.
- Variants: plain classic-settling, Direct Kolen-Pollack feedback (fewer settling steps, some accuracy cost), and a 'kitchen sink' network with µPC scaling, residual connections, incremental PC, error-based PC, momentum settling — all optional toggles, off by default.
- Status: CPU path is mature and benchmarked; CUDA backend (cuBLAS + CUTLASS + CUDA graph capture) runs but large-scale training is still being tuned; convolution is im2col+cuBLAS, not cuDNN. MIT-licensed; seeking C++/CUDA performance and DevOps contributors.
More from Research
- UK AISI: under $80 of fine-tuning fully restores filtered bio capabilities in open-weight models — connoraxiotes · 2026-10-27
- RFD4-Proteina runs on an RTX 3060: protein design from $0.001 per design — AllThingsApx · 2026-10-27
- University of Freiburg's Generative Intelligence Lab recruiting funded PhDs via ELLIS, apply by Oct 31 — yuyinzhou_cs · 2026-10-27
- Nature Physics: reprogrammable refractive index enables arbitrary multimode wave control for ML — MarioKrenn6240 · 2026-10-27
- Wakaru open-sources ontologize: labeling datasets from model activations — amplifiedamp · 2026-10-27
- EMNLP Oral paper traces multilingual representations in LLMs with cross-layer transcoders — ZhijingJin · 2026-10-27