Why 1536 dimensions? Reddit debates the odd default in embedding models
Gay-Guy-With-GF · reddit · 2026-10-03
A Reddit technical question: why do embedding models like OpenAI's default to 1536 dimensions rather than 1024 or 2048? The poster understands hardware-friendly multiples (64/128/256/512) but wonders whether 1536 = 3×512 is an empirical sweet spot between representation quality and memory/compute, or driven by architectural/hardware convenience. The thread invites answers from people who have actually trained embedding models, making it a useful primer on the engineering tradeoffs behind embedding dimension choices.
More from Research
- arXiv trends suggest AI uplift hits quantum physics within a year, all physics by late 2029 — cephaloform · 2026-10-03
- Pretraining data mixing is 'vibes-based' while RL is brutally mathematical, says willcb — willcb · 2026-10-03
- Berkeley's Humanoid Intelligence Center wins both tracks at IROS 2026 RoCo Challenge — berkeley_ai · 2026-10-03
- OpenStamp embeds watermarks into open-source LLM weights so users can't strip them — danish037 · 2026-10-03
- CruxBench lands at NeurIPS: frontier LLMs barely beat random at asking the right questions — mengyer · 2026-10-03
- Lampinen calls out neurosymbolic camp for walking back internal-symbolism claims — AndrewLampinen · 2026-10-03