Deep Reinforcement Learning for Revenue Management under Uncertainty in Master Stowage Planning on Container Vessels

Advanced planning policies obtained by machine learning have shown promising
results in solving well-known combinatorial optimization problems in transportation and logistics. However, a significant challenge arises when dealing with complex action spaces in realistic planning, where it is less straightforward for machine learning models to generate feasible actions. A relevant and complex example is master stowage planning on container vessels, which plays a crucial role in global trade and the green transition. This planning problem aims to maximize cargo revenue and minimize operational costs while addressing strict constraints and demand uncertainty. To tackle this challenge, our paper introduces a deep reinforcement learning framework with a general feasibility layer to solve a novel Markov decision process of master stowage planning under demand uncertainty. The experimental evaluation shows that our architecture efficiently finds feasible solutions for a multistage stochastic optimization problem, which is intractable using traditional benchmark methods from combinatorial optimization. Our approach demonstrates the potential of advanced planning policies to tackle complex, real-world problems, with implications for global trade and sustainability.