Video Games and Computer Vision research have long held a symbiotic relationship. On the one hand, virtual worlds in games are often used for collecting training data or as testbeds for computer vision models since they provide a greater deal of flexibility, control and scalability in the data collection process compared to the real world. On the other hand, computer vision advancements have enabled us to push the frontiers of what is possible within these artificial game worlds and have transformed the processes with which these worlds are created. However, significant research questions still remain unaddressed both in the field (Computer Vision) and the domain (Games), which include technical and engineering challenges.

This special issue invites research papers aiming to bridge the existing gaps between computer vision research and games engineering, with the motive of bringing together the games research community and the computer vision community that have largely operated independently until now. We are inviting papers for two main tracks. The first track focuses on introducing novel techniques within computer vision research that can advance the field of digital games. The second track, instead, focuses on leveraging game technologies to advance state-of-the-art techniques in computer vision. The list of topics below is not inclusive of all research directions that will be represented.

1) Computer Vision for Games

• CV for game-playing, game testing and player modelling.
• Data-driven CV to improve game graphics, animations, level-design, etc. as well as procedural content generation.
• HCI through visual interfaces (gestures, posture, gaze, etc.).
• Extended reality games.
• Synthetic data and media generation based on users' emotions, behaviour, etc.
• Improving real-time applicability of vision models integrated within games and game engines.

2) Games for Computer Vision

• Game worlds that aid data augmentation techniques.
• Rich game-based labelled datasets for tasks such as object detection, segmentation, or depth and flow estimation.
• Ethics of game-based data collection and inference.
• Forward modelling in and for games.
• Generalisation and robustness in vision models leveraging a plethora of existing commercial games.
• Unsupervised pre-training of image/video representations and world transition models from gameplay data.

We invite the submission of high quality papers on the topics above in the full paper format. Authors should follow normal IEEE Transactions on Games guidelines for their submissions, but clearly identify their papers for this special issue during the submission process. Extended versions of previously published conference or workshop papers are welcome, provided that the journal paper is a significant extension, and is accompanied by a cover letter explaining the additional contribution. You may visit the submission guidelines for author information guidelines and page length limits.

Important Dates:

• Paper submission: January 31, 2024
• First decisions: May 31st, 2024
• Early access SI publication (online): August 2024
• Publication in print: End 2024

Guest Editors:

• Chintan Trivedi (University of Malta)
• Matthew Guzdial (University of Alberta)
• Konstantinos Makantasis (University of Malta)
• Julian Togelius (New York University)
• Nicu Sebe (University of Trento)