DeepMind's DeepNash Agent Masters Complex Stratego Board Game
DeepMind researchers introduce DeepNash, an autonomous AI agent that learns to play the complex board game Stratego at an expert human level. The system uses a novel model-free reinforcement learning approach to navigate the game's massive decision tree and hidden information.
DeepMind researchers introduce DeepNash, an autonomous agent that learns to play the classic board game Stratego at a human expert level from scratch. Stratego presents a massive challenge for artificial intelligence because it combines imperfect information with an enormous game tree featuring over 10 sextillion possible starting deployments. Previous AI research struggles with these complex components, making this breakthrough a significant milestone for the field.
The DeepNash system relies on Regularized Nash Dynamics (R-NaD), which is a model-free reinforcement learning technique that guides the agent toward an ε-Nash equilibrium. This equilibrium ensures that the AI performs reliably even when facing a worst-case opponent. The architecture processes the game state through a deep neural network equipped with four distinct heads that calculate both the value function and the probability distributions for gameplay decisions.
The training process for DeepNash consists of three main phases: fundamental R-NaD training, fine-tuning of the learned policy, and test-time post-processing. The R-NaD algorithm itself operates through reward transformation, dynamics estimation, and policy updates, utilizing regularization to guarantee stable convergence. By adapting advanced estimators to the two-player imperfect information setting, the system effectively masters a game that has long stumped AI developers.