Character.AI's old memory-efficient architecture post was 'meaningful intel' back in the Llama 3 era
voooooogel · x · 2026-10-14
- voooooogel notes the irony that Character.AI, the "waifu ERP company" founded by Shazeer, once published blog posts that were genuinely valuable architecture intel when the OSS frontier was Llama 3 (or DeepSeek v2).
- The attached post describes their memory-efficient design: Multi-Query Attention (8x smaller KV cache vs GQA), hybrid attention horizons (sliding-window local attention on most layers, only 1 in 6 layers global), and cross-layer KV sharing (another 2-3x cut), shrinking KV cache by 20x+ without quality regression.
- Quoted commentary criticizes Character.AI for getting addicted to free-user analytics and dithering on adult-content moderation, missing the boat repeatedly.
More from Fun
- Formalized proof completed; author suggests using AI to explore proof strategies — tak3sh8 · 2026-10-14
- 'Oh no!': AI video of a man chased by a giant monster goes viral — phusa29 · 2026-10-14
- Blogger predicts an AI shock event within 16 weeks, paired with OpenAI-themed AI images — flowersslop · 2026-10-14
- Hard-to-read ML papers are often just badly written, researchers argue — gabriberton · 2026-10-14
- Y Combinator gets its own Jeopardy category — ycombinator · 2026-10-14
- Bots in freebots.lol have already taken 5,000+ photos of their own 3D city — Daniel_Farinax · 2026-10-14