New metric suggests dense models benefit significantly at bs=1
teortaxesTex · x · 2026-08-15
Discussion on a new evaluation metric focusing on active sampling at a block size of k (e.g., 9). The argument suggests this is mostly relevant for local models and significantly benefits dense models when batch size (bs) is 1.
More from Infra
- vLLM Introduces Adaptive Verification for Speculative Decoding with DSpark — vllm_project · 2026-08-15
- Open source closes the gap with closed labs: Quality gap now just months — togethercompute · 2026-08-15
- How Nuclear Will Unlock Energy Abundance with Valar Atomics Founder — No Priors · 2026-08-15
- Nvidia reportedly cuts OpenAI data center guarantee from $250B to below $120B — rohanpaul_ai · 2026-08-15
- DeepSeek V4 on Cloudflare Workers AI: First 1M Token Context Models — ritakozlov · 2026-08-15
- Regulators speed connections for AI facilities, require infrastructure payment — pstAsiatech · 2026-08-15