Polars 2.0 ships with breaking SQL changes, spill-to-disk and speedups
KhuyenTran16 · x · 2026-10-28
pola-rs released Polars 2.0.0, with Khuyen Tran highlighting its new spill-to-disk feature: memory-intensive operations (sorting, window functions, complex expressions) automatically move to disk at 80% RAM usage — though joins and group-by don't support spilling yet.
Key changes:
- Breaking: SQL window functions now run over aggregated rows with QUALIFY evaluated before projection; SQL numeric literals are typed as Decimal with truncated %/DIV; Parquet ENUM reads as pl.String; cut/qcut deprecated.
- Performance: fixed premature phase ending from join sampling, batched remote IPC scans, deferred Parquet prefetch to decode, scalar window evaluation in the streaming node, filter pushdown to scanlance, out-of-order IPC decoding, and more.
For data engineering, automatic disk spilling means large sorts no longer crash on limited-RAM machines.
Related event: Polars 2.0 Released with Disk Spilling and Performance Upgrades(2 posts)→
More from coding & agent
- Parent and first-grader build an addictive math game with a one-shot Claude prompt — neuroecology · 2026-10-29
- Man automates Hinge with Claude Code and a spare Android, gets 20-25 matches a day — pritisinghhhh · 2026-10-29
- 3 guardrails before letting AI coding agents merge to production — Arindam_1729 · 2026-10-29
- EverOS Cloud hands-on: persistent memory lets Claude recall shop policies across new chats — Aiden_Tech_Ai · 2026-10-29
- Ziplime: new Python library for AI-driven algorithmic trading and backtesting — mdancho84 · 2026-10-29
- Vibe coding RPG: he needed a resume, so he built a resume editor first — generativist · 2026-10-29