Inkling 2 Small's width-4 sconv forces a tiny rolling 3-token conv cache per layer at inference
andrew_n_carr · x · 2026-11-13
andrewncarr points out an inference detail: the width-4 sconv in Inkling 2 Small means inference engines have to maintain a tiny rolling 3-token convolution cache per layer alongside the usual KV cache.
A useful implementation reminder that the architecture deviates from vanilla transformers and inference stacks need dedicated caching support for its convolutional component.
More from Infra
- DeepSeek V4.1-Flash seeds resist 4.3B-seed search; Huawei chip shift rumored — TheNormanMu · 2026-11-13
- OpenCartridge's trust bill: no auth, public bucket, Docker socket exposed — maier_ak · 2026-11-13
- camenduru open-sources OpenCartridge, packaging AI models as one-click 'cartridges' — maier_ak · 2026-11-13
- Game assets on your own GPU: what OpenCartridge does and what it asks you to trust — maier_ak · 2026-11-13
- Core Memory podcast: Sceye's $100M high-altitude airship wins SoftBank fleet order for Japan broadband — kyliebytes · 2026-11-13
- Torus explains closed-loop liquid cooling: no on-site water hookup needed — ccerrato147 · 2026-11-13