RL and Long-Horizon Planning Strategies for Stochastic Single-Player Merge Puzzles

CaiwenGong · reddit · 2026-08-11

A developer is designing an AI for a stochastic single-player merge puzzle (similar to 2048) and is seeking algorithmic insights for planning and reinforcement learning.

Game Mechanics: The board features 6 stacks. Players move contiguous runs of equal tiles to trigger merges. A key mechanic is that random tile drops occur every 4 actions, with a 1-action preview of the upcoming random tiles.
Objectives: The goal is twofold: maximize single-game score and maximize total throughput within a 30-minute window (1,800 actions), framing it as a continuing average-reward problem rather than a standard episodic one.
Current Architecture: The model uses a column-permutation equivariant Policy/Value network with 394 input features. It includes shared column encoders and multiple value heads predicting future scores, distance to the next merge, and short-term death risk.
Challenges: The developer highlights the distinction between cold-start costs and mature-board efficiency, seeking methods to optimally allocate a limited planning budget under these constraints.

Original post →

RL and Long-Horizon Planning Strategies for Stochastic Single-Player Merge Puzzles

More from Research