Research Repository

Rolling horizon methods for games with continuous states and actions

Samothrakis, S and Roberts, SA and Perez, D and Lucas, SM (2014) Rolling horizon methods for games with continuous states and actions. In: UNSPECIFIED, ? - ?.

[img]
Preview
Text
RollingHorizon-CIG-2014.pdf

Download (527kB) | Preview

Abstract

© 2014 IEEE. It is often the case that games have continuous dynamics and allow for continuous actions, possibly with with some added noise. For larger games with complicated dynamics, having agents learn offline behaviours in such a setting is a daunting task. On the other hand, provided a generative model is available, one might try to spread the cost of search/learning in a rolling horizon fashion (e.g. as in Monte Carlo Tree Search). In this paper we compare T-HOLOP (Truncated Hierarchical Open Loop Planning), an open loop planning algorithm at least partially inspired by MCTS, with a version of evolutionary planning that uses CMA-ES (which we call EVO-P) in two planning benchmark problems (Inverted Pendulum and the Double Integrator) and in Lunar Lander, a classic arcade game. We show that EVO-P outperforms T-HOLOP in the classic benchmarks, while T-HOLOP is unable to find a solution using the same heuristics. We conclude that off-the-shelf evolutionary algorithms can be used successfully in a rolling horizon setting, and that a different type of heuristics might be needed under different optimisation algorithms.

Item Type: Conference or Workshop Item (Paper)
Additional Information: Published proceedings: IEEE Conference on Computatonal Intelligence and Games, CIG
Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Divisions: Faculty of Science and Health > Computer Science and Electronic Engineering, School of
Depositing User: Jim Jamieson
Date Deposited: 04 Dec 2014 13:37
Last Modified: 30 Jan 2019 16:18
URI: http://repository.essex.ac.uk/id/eprint/11982

Actions (login required)

View Item View Item