HARNESS: pure-Rust LLM inference engine claims 70B models on 8GB VRAM
9gxa05s8fa8sh · reddit · 2026-10-20
A new GitHub project, HARNESS, is a pure-Rust LLM inference engine that doubles as a bio-inspired spiking neural network (SNN) runtime. Headline claims include running 70B models on 8GB VRAM, LIF spiking attention, a hippocampal dual-memory architecture, and its own PagedAttention v3.
Spotted on Reddit, the project has no independent verification yet; treat the aggressive memory-compression claims with caution and test it yourself.
More from Infra
- turbopuffer cuts disk read I/O by up to 80% with smarter vector rescoring — DanielLockyer · 2026-10-20
- vLLM Speculators v0.9.0: 2x faster speculative decoding training, 50% less storage — vllm_project · 2026-10-20
- Datalab ships Indra OCR 1.1: 1M-param model does 44 pages/s on CPU, near DeepSeek OCR 2 accuracy — VikParuchuri · 2026-10-20
- Local LLMs on a 96GB M3 Ultra: Qwen3.8-27B Q8_0 Wins for Coding, With Full Config Shared — FanDiscombobulated38 · 2026-10-20
- Unsloth ships DeepSeek V4.1-Flash GGUFs: 1-bit cuts 508GB to 250GB keeping 88.2% accuracy — danielhanchen · 2026-10-20
- Hugging Face's Local AI 202 shows massive speed-ups with speculative decoding — huggingface · 2026-10-20