IEEE Transactions on Games

Special Issue on Human-Centred AI in Game Evaluation

Most games are consciously designed with a specific experience or vision in mind. Games are commonly designed for entertainment and competition purposes, but self-expression, social critique, targeted learning, knowledge discovery as well as physical and mental health are also valid design objectives. Determining whether an objective is fulfilled is often quite difficult due to the complexity of modern games and the variability of human responses. For this reason, games are commonly playtested before being published. However, playtests are expensive and time-consuming and not all aspects of the game can be evaluated to the full extent before it is published.

There is thus a need for more concentrated and systematic work on evaluating/characterising games, its artefacts as well as player experience. Researchers have proposed approaches intended to assist game designers using methods from the field of artificial and computational intelligence (AI and CI, respectively). Still, to our knowledge, there is a surprising lack of generality and validation regarding these methods, even in scientific publications on game design. No central repository for methods currently exists. In this special issue, we want to focus on human-centered AI approaches aimining for a more holistic and systematic approach to game evaluation. We thus seek submissions on related topics for this special issue.

The following is a non-comprehensive list of suggested topics:

  • Uses of AI agents to evaluate game content
  • Measures for game evaluation
  • Game evaluation and play-testing for AR/VR
  • Relationship between AI agents and player experience
  • Automatic analysis of play-traces
  • Mixed-Initiative gameplay evaluation
  • Player modelling for game evaluation
  • Automatic evaluation for new game genres
  • Validation of automatic evaluation methods using human data
  • Generality of automatic evaluation methods
  • Differences between different evaluation methods (tested with AI or humans, qualitative vs. quantitative, objective vs subjective measures, etc.)
  • Evaluation measures and their relationship to business and research goals
  • Playtesting standards in industry
  • Correlation between objective and subjective measures
  • Ethics, privacy and legal aspects of using player data
  • Evaluation of generated content

We invite the submission of high quality papers on the topics above in the following formats:

  • Full papers
  • Short papers
  • Letters

Authors should follow normal IEEE Transactions on Games guidelines for their submissions, but clearly identify their papers for this special issue during the submission process. Extended versions of previously published conference or workshop papers are welcome, provided that the journal paper is a significant extension, and is accompanied by a cover letter explaining the additional contribution. See (https://www.transactions.games/submit/submission-guidelines) for author information guidelines and page length limits.

Important Dates

  • Paper submission November 1st, 2023
  • First decisions January 29th, 2024
  • Early access SI publication (online) March 2024
  • Publication in print End 2024

Guest Editors

  • Alena Denisova (University of York, UK)
  • Diego Pérez-Liébana (Queen Mary University of London, UK)
  • Vanessa Volz (modl.ai, DK)
  • Julian Frommel (Utrecht University, NL)
  • Sahar Asadi (King, SE)

Contents