Don't Run This Model in FP16: Activations Exceed Its Dynamic Range
tomaarsen · x · 2026-10-07
The author warns against running the model in FP16 and recommends BF16 or FP32 instead. The model's activation range exceeds FP16's dynamic range, and the model card warns of NaNs or silently degraded embeddings. Use BF16 where natively supported, and FP32 elsewhere, including most CPUs.
The quoted post adds multimodal input defaults: video is sampled at 1 frame per second, audio should be mono at 16 kHz, and the vision budget is configurable from 70 to 1,120 soft tokens per image/frame — more tokens trade latency and context capacity for finer visual detail.
More from coding & agent
- Meta's chief AI officer: an agent swarm can outperform a team of 100 engineers — garrytan · 2026-10-07
- Dev gives his non-coder wife remote access to local Codex: 'just tell it what you want' — Yamapama · 2026-10-07
- Dev uses Claude to adapt Sudoku app to new iPhone Duo's dual screens in one conversation — SeanOliver · 2026-10-07
- Vignette: an open-source screenshot annotation tool built for two-way agent collaboration — narphorium · 2026-10-07
- Hookdeck Open-Sources a Bridge That Converts Webhooks into MCP Events for Agents — phobos7 · 2026-10-07
- autosana claims first testing platform to support iPhone Duo in open and closed states — anthara_ai · 2026-10-07