Falcata: CUDA-native GBDT rebuilds training loop, 14x faster than LightGBM

srchvrs · x · 2026-08-18

Leaf-wise growth in LightGBM-style trees is a strictly sequential loop (one histogram, one split, repeated thousands of times per tree). On GPU this means a kernel launch plus host-device sync per step, leaving the card mostly idle.
@kronerfelix and the author forked LightGBM into Falcata: whole levels grow in a single batched launch while provably producing the exact same tree a leaf-at-a-time learner would build — leaf-wise trees, level-wise launch counts.
On the same single GPU with matched or better held-out quality: 2.4x faster than XGBoost, 4.7x faster than CatBoost, 14x faster than LightGBM.
Flagship workload: Numerai (6.8M rows × 3,555 features).

More from Infra