1-Bit Quantized Version of Inkling Released
danielhanchen · x · 2026-07-16
The author announced a collaboration with Thinking Machines to create a Dynamic 1-bit GGUF quantized version of Inkling, highlighting the following:
- A 86% reduction in size, shrinking from 1.9TB down to 270GB.
- Retains 74.2% of the top-1% accuracy.
- Added vision and audio support.
- Provides GGUF files and related guides, emphasizing that it can run on hardware with around 280GB of resources.
More from Infra
- Encrypted GPT-2 speedrun challenges FHE inference: best is 3.031 s/token, goal sub-1s — tensorqt · 2026-10-27
- Open-source Latch compiles PyTorch into training megakernels, 2.75x faster on H100 — sguada · 2026-10-27
- Stanford launches GPU-centric supercomputer Marlowe for data science and AI research — StanfordHAI · 2026-10-27
- NVIDIA Claims HGX Rubin NVL8 Delivers 10x More Tokens per Megawatt Than B200 — thursdai_pod · 2026-10-27
- Together lets multiple LoRA post-training experiments share the same GPU capacity — togethercompute · 2026-10-27
- Commoditization dilemma: either human labor or GPUs become the commodity — _AustinCalvert_ · 2026-10-27