Memory-constrained scaling debate: inactive params are dead weight, dense may win intelligence per GB
samsja19 · x · 2026-10-21
Tim Darcet argues that under a total memory constraint, unactivated (sparse) parameters are literally dead weight — implying dense architectures may deliver the best intelligence per GB. samsja19 follows up asking whether looping parameter reuse could push this further, noting local GPUs have flops to spare.
More from Research
- New leaderboard ranks 57 decision models: d1-ultra tops browser tasks at 74% for $0.026 per 1K decisions — AIMultiple · 2026-10-21
- Ampersand's Persimmon user model cuts models' evaluation awareness by 40 points — alexisjross · 2026-10-21
- Snorkel AI audit: 55% of TerminalBench 4.0 fixes had reward hacking issues themselves — ravithejads · 2026-10-21
- EAGLE-4 hits 3.59x decoding speedup on Qwen3.8-27B, beating DFlash 2 and DSpark — TKGaming_11 · 2026-10-21
- Basalt monocular-inertial VIO ported to WASM: SLAM now runs live in mobile browsers — rsasaki0109 · 2026-10-21
- TRACK paper: sim racing telemetry framework finds only speed and consistency transfer across cars — JeremyCMorgan · 2026-10-21