Assessing Believability

Julian Togelius, Georgios N. Yannakakis, Sergey Karakovskiy, Noor Shaker

Research output: Conference Article in Proceeding or Book/Report chapterBook chapterResearchpeer-review


We discuss what it means for a non-player character (NPC) to be believable or human-like, and how we can accurately assess believability. We argue that participatory observation, where the human assessing believability takes part in the game, is prone to distortion effects. For many games, a fairer (or at least complementary) assessment might be made by an external observer that does not participate in the game, through comparing and ranking the performance of human and non-human agents playing a game. This assessment philosophy was embodied in the Turing Test Track of the recent Mario AI Championship, where non-expert bystanders evaluated the human-likeness of several agents and humans playing a version of Super Mario Bros. We analyze the results of this competition. Finally, we discuss the possibilities for forming models of believability and of maximizing believability through adjusting game content rather than NPC control logic.
Original languageEnglish
Title of host publicationBelievable Bots
Number of pages17
PublisherSpringer Publishing Company
Publication date2012
ISBN (Print)978-3-642-32322-5
Publication statusPublished - 2012


  • Believability Assessment
  • Non-Player Characters
  • Participatory Observation
  • External Observation
  • Turing Test Track


Dive into the research topics of 'Assessing Believability'. Together they form a unique fingerprint.

Cite this