Kafka in production: partition skew and rebalancing pauses bite hard
goyalshaliniuk · x · 2026-09-27
A ops-focused explainer on Kafka at scale: beyond development, production growth brings bigger partitions, consumer scaling, and more connectors. Common pitfalls include uneven partition distribution overloading brokers, slow consumers cascading into failures, and rebalances that halt consumption — all directly impacting reliability.
More from Infra
- World's fastest panel QR factorization on B200: how a GPU MODE contestant cracked chained dependencies — A_K_Nain · 2026-09-27
- Pro-data center rally clashes with protesters as scholar defends AI infrastructure — neil_chilson · 2026-09-27
- Dual RTX 5090 + 4070 TiS runs 27B FP8 with 258K context at 81 tok/s — Fz1zz · 2026-09-27
- Hardest part of building US data centers? Finding electricians, plumbers and welders — DominiqueCAPaul · 2026-09-27
- Guardian: Datacenter backlash shows democracy blindsiding Silicon Valley's AI power plays — nordicinst · 2026-09-27
- Dev caches hide 264 CVE-laden packages and 71.7 GiB no one audits, warns Cache Commander dev — julsimon · 2026-09-27