Challenges in Conducting Theory-Informed Empirical Player Experience Research

Empirical player experience (PX) research has seen increased calls for more and better theoryuse. Theory drives research programs, allows to synthesize research and can inform design.
These calls, however, rarely discuss the actual practice of doing theory-informed research. How
do we work effectively with theory in PX research? What challenges do we face; and, should
we not overcome them, what does it mean for empirical PX research to fail? In this thesis, I
approach these questions by presenting five research projects and reflecting on their respective
research processes, challenges, and failures.
Publication I is a systematic literature review of the reporting practices of Null Hypothesis
Significance Testing at CHI PLAY. A method commonly used to test theories and widely used in
empirical PX research. Based on the findings of Publication I, and a discussion of confirmatory,
theory-testing methods, I highlight the role of exploratory methods in PX research.
Publication II is an exploratory mixed-methods study on art experiences in videogames. The study utilizes empirical aesthetics to understand what players consider art experiences. It highlights the importance of epistemic emotions — i.e., Interest and Insight — as well as a range of
short and long term impacts the experiences had on players. As this paper employs a theory
devised to explain and predict the art experience of people, I reflect on the process of bringing
the theory over to player experience research by discussing how we used it in this paper and
how it guided the study design and codebook-based analysis.
Publication III studies empowering and disempowering player experiences. While empowerment is a common, desirable goal for HCI and HCI games research, it has a multitude of
conceptualizations and can overlap with other commonly used PX constructs such as emotional
play experiences and basic need satisfaction. I reflect on the overlap of different PX constructs
with empowerment and disempowerment and how theory guided the choice of behavioral measures and thematic analysis.
The last two projects I discuss have not been published. In this thesis I reflect on the process
of each project, the challenges they encountered, and what to learn from them. The first project
iii
sought to experimentally evoke emotional challenge and compare if Self-Determination Theory
or an art appraisal theory better predict the enjoyment and appreciation. After conducting the
study, results revealed mostly non-significant results, with further Bayesian analyses indicating
that the desired experimental manipulation was unreliable, throwing into question what can be
learned from the results. The second project is an ongoing exploratory experiment, investigating
the player experience of rewards through the lens of Cognitive Evaluation Theory. A pre-study
raised concerns about measuring the functional significance and reliability of the study manipulation, leading to a reassessment of the feasibility of the study design. I will reflect on how these
two studies were designed highlighting difficulties in translating theories into both testable hypotheses and experimental manipulations. I further discuss issues when measuring concepts
and when attempting to decide between the study being wrong, or the theory.
This thesis contributes to PX research in the following ways. First, this thesis highlights tangible opportunities and approaches to better theory-informed PX research. Second, it describes
concrete barriers and pitfalls to creating informative theory-informed PX research. Third, I give
a personal account on failure in my research practice, something that otherwise is scarcely talked
about, in the hope of sparking more reflection and discussion about HCI and PX research practice