ICML Paper: LLMs Store 3.6 Bits of Memory Per Parameter

An ICML paper shared by Nvidia estimates that GPT-style LLMs have a memory capacity of about 3.6 bits per parameter. The research distinguishes between 'unintentional memorization' and 'generalization' to accurately measure model capacity.

2026-07-07 ~ 2026-07-07 · 2 related posts

• ICML论文：LLM每参数约3.6比特记忆容量 — NVIDIAAI · 2026-07-07
• Paper: How Much Do LLMs Remember? ~3.6 Bits Per Param — pmttyji · 2026-07-07