Inkling-276B Multimodal Model Local Test: 3-bit Quantization Hits 37 t/s Decode
MaziyarPanahi · x · 2026-08-02
Developer Maziyar Panahi shared local benchmark results for the multimodal model Inkling-Small (276B A12B architecture) running on Metal hardware.
Using a 3-bit quantization, the model occupies 91.19 GiB of memory. It achieved a prefill speed (pp512) of roughly 510 tokens/s and a decode speed (tg64) of 37 tokens/s. In a practical test, the model successfully processed simultaneous image and audio inputs, accurately analyzing a 2-minute medical audio to deduce that a patient's heart failure worsened due to stopping their diuretic.
More from Models
- Mistral touts STEM benchmark win, but leaderboard shows it ranking sixth overall — MistralAI · 2026-10-16
- Local tool calling compared: how Gemma 4, Llama 3 and Mistral differ — leslysandra · 2026-10-16
- $2 Gemini Pro accounts flood the gray market, spawning waves of AI spam comments — huangyun_122 · 2026-10-16
- Haider: Opus 5.5 vs Astra on SWE tasks — Astra has a higher floor, reasoning in a tier of its own — haider1 · 2026-10-16
- Artificial Analysis launches Decision Index for the emerging decision model category — ArtificialAnlys · 2026-10-16
- Liquid AI's d1-ultra tops Artificial Analysis Decision Index with 85% non-hallucination rate — ArtificialAnlys · 2026-10-16