Could RAM and SSD become the bottleneck as next-gen n-gram models offload to storage?
accelerate_to_asi · reddit · 2026-09-24
A Reddit poster asks whether offloading knowledge to SSD/RAM in next-gen n-gram models, freeing VRAM, simply shifts the bottleneck to memory and storage — estimating even small 27B-class models could need hundreds of GB RAM or terabytes of storage, noting Qwen3.8 Flash Next's 2-bit quantization is still larger than Qwen3.8 27B at 4-bit. The post invites discussion of compute-storage trade-offs for local deployment.
More from Infra
- UK Sovereign AI invests in Fractile to build next-gen frontier AI chips — charlieharris01 · 2026-11-11
- What happened when I moved vector retrieval into Snowflake SQL instead of orchestrating RAG outside — dima806_dima · 2026-11-11
- AI chip startup Fractile raises $720M Series C for DRAM-on-processor design — Tom_Westgarth15 · 2026-11-11
- Mobile AI Factory: agentic pipeline builds on-device SLMs that beat 5-shot frontier models — kalyan_kpl · 2026-11-11
- Vivo's Starlink direct-to-cell service hits 10M users in two weeks across Brazil — XFreeze · 2026-11-11
- Nan Ya Plastics to spend NT$11.53B on fifth glass fabric plant on AI server demand — zephyr_z9 · 2026-11-11