Operationalising Difficulty in Puzzle Games

The main line of investigation of this industrial PhD has been determining and modelling difficulty in games, specifically in mobile puzzle games. Difficulty plays a crucial role in player engagement in such games, so gaining a deeper understanding and being able to predict the perceived difficulty on a player level are important goals not only for researchers but for the industry at large.
The initial work focused on creating a playtesting agent that would be able to automatically play through new content in a commercial puzzle game. For this purpose, we developed a reinforcement learning setup that could operate within a number of technical constraints, such as no possibility of using player play traces or tree-search. While the agent did not reach human-level performance on the full-scale problem, a key finding was that the top
~10% performances of the agent on a level were strongly correlated with player data. Additionally, we proposed ways to train the playtesting agent in a quick and robust way.
With the agent not being enough by itself to predict the difficulty of new levels, we started to address the question of how to link agent behaviour to player behaviour. The first line
of research was more focused on answering the question of what difficulty actually is in puzzle games and what it entails to predict difficulty for any content a player has not yet encountered. There were two main findings from this work: first, we proposed a parametric distribution for modelling the number of actions players spend for completing a level, paving the way for dynamic difficulty adjustment, and second, how individual difficulty predictions for the players are possible using factorization machines by capturing player skill and intrinsic level difficulty with latent factors.
In the last line of research, the objective was to tie it all together – how can agent behaviour data be used together with personalised predictions for estimating the difficulty of not just old content but also new, novel content? The results showed that agent data does indeed have high predictive power on new content and can improve personalised predictions. While the factor-ization machine approach is useful for personalised predictions on old content, for predictions on new content, non-personalised predictions using a standard artificial neural network worked better. There is therefore not one approach that works the best in all use-cases, but in each of the use-cases, the accuracy of the methods is high enough for being used in a commercial context.