Project Maya v1.1.73 brings Inkling 2 Mini to 8GB GPUs, 144 tok/s on a 4090
teortaxesTex · x · 2026-11-22
Project Maya released v1.1.73, greatly expanding hardware support for Inkling 2 Mini:
- Runs on NVIDIA cards from 8GB: hot experts stay in VRAM, the rest in RAM, with GPU/CPU split measured once per machine (13GB RAM needed with an 8GB card); outputs are identical wherever an expert runs
- Older GPUs supported: V100 and RTX 20-series get dedicated fast kernels; on Linux, AMD Radeon 24GB cards and Strix / Gorgon Halo are supported (tested on Radeon 8065S)
- Auto drafting (--drafting auto) proposes tokens via the official drafter plus conversation context, with answers unchanged and faster mode auto-selected
- Fixes for --vram-reserve-mib being ignored and Hy4-Flash on dual-GPU PCs
User teortaxesTex reports an easy 144 tok/s on a 4090, calling it the first good model for a single-GPU box — though he still wants a true DeepSeek-Nano with an honest 1M context, noting the current 32K context already needs 1.2GB of cache.
More from Infra
- Fugaku supercomputer breach: 430 SSH keys stolen, 233 had no passphrase — kingslyj · 2026-11-22
- OMEN: free open-source tool tests 714 GPU setups to pick the best card for local AI models — Srijith_Legend · 2026-11-22
- Cerebras launches disaggregated inference with AMD, delivering 5x more tokens per megawatt — AccBalanced · 2026-11-22
- GTA VI is a megakernel: devs marvel at its GPU-driven software rendering pipeline — charles_irl · 2026-11-22
- Nvidia's circular GPU financing questioned: funded purchases, guaranteed debt, insurer backstop — MatthewChang · 2026-11-22
- Ex-Google TPU engineer: cutting frontier research will ultimately sacrifice the hardware business — jasondeanlee · 2026-11-22