Orateur
Description
We introduce the application of Monte-Carlo Tree Search (MCTS) for a well-defined large scale Markov decision process. As an example we investigate, the ship refuelling problem subject to subject stochastic fuel prices. As a benchmark model, we compare this MCTS-based implementation to pure stochastic programming approaches with scenario tree generation. The objective of the new approach is to demonstrate the use of MCTS to produce approximate solutions to large scale MDP's when they become intractable via standard mathematical programming or dynamic programming. Another objective is to show that our MCTS based solver can achieve near-optimal performance with respect to a stochastic programming solution, and a much lower computational cost. Moreover, a hybrid approach exists involving applying exact methods, such as stochastic programming, to solve the a value function of a sub-problem. This value estimate can be used in tandem with MCTS, to reduce the variance of Monte Carlo sampling, and/or speed up convergence to the optimal solution. The practical application of this technology would be to allow maritime planners to more effectively optimize for fuel prices in the face of increasing model complexity, and also potentially in model-free settings.
