Ablation table for Game24-from-Countdown shows hierarchical RL hitting 69.7 vs flat baseline 52.4

vxnuaj · x · 2026-10-27

The author shares an ablation table (Table 7) for Game24-from-Countdown, quipping that running it feels "like a video game".

Key points from the table:

Base test accuracy ≈ 64; the published flat recipe scores 52.4 (-11.7) while hierarchical H-ELF reaches 69.7 (+4.8)
GRR (600 iterations) lifts flat to 68.5 (+4.6)
An under-trained critic costs 5.7 points on average; masking the critic's view of coarse channels barely matters (+1.8)
Routing actor gradients through token channels preserves a +3.4 gain; coarse-only channels drop to 63.2
Random class encodings cost 2.1 points on average, indicating the code semantics matter; further controls (CFRL-recipe retraining, k-means class mapping, 3× backbone) are queued

More from Fun