What papers/frameworks/techniques are considered ‘standard’ templates for measuring and analysing player engagement with digital games/serious games? Engagement being in the broadest sense to include immersion, flow and so on. Thank you.
I know of a (excelent) comparison between several techniques for comparing game evaluation, including biometic ones by Marcello Andres Gómez Maureira,
Maureira, M. A. G., Janssen, D. P., Gualeni, S., Westerlaken, M., & Calvi, L. (2013). Comparing Game User Research Methodologies for the Improvement of Level Design in a 2-D Platformer. In Advances in Computer Entertainment (pp. 77-92). Springer International Publishing.
He will probably refer to related work which might help you.
Yes, formative evaluation of games using interviews/questionnaires, metrics and biometrics, is at a very interesting stage. I intend to test frameworks for interviews/questionnaires (I have the GEQ in mind - that Marcello used, he had some reservations) and biometrics against a serious game prototype. The game is intended to support the rehabilitation of secure forensic mental health service users, and was developed with the service users input. Unfortunately I am unable to test using the service users, but I want to try and reach some understanding of the evaluation techniques. Validation of the effectiveness of a game with clinical aspirations is obviously very important, and hopefully such games will be given a little more time for careful analysis than those Marcello is dealing with in the commercial world.
A general overview over questionnaires has been given by Nordin et al. 2014 (http://www-users.cs.york.ac.uk/~pcairns/papers/Nordin_YDS2014.pdf). However, as rarely is the case, it's not exhaustive. I would also consider the MEC - Spatial Experience Questionnaire by Vorderer et al. 2014 (http://academic.csuohio.edu/kneuendorf/frames/MECFull.pdf).
I would on the other hand disagree that you can have a generic questionnaire or measurement method at all, since all of the aforementioned questionnaires and even biometrics make more or less restrictive assumptions about the games they test.
Hi Katharina, I totally (think) I agree about 'generic' measures - I guess the more mainstream the product and the audience the better fit generic questions become. Maybe in the future we may better understand biometric measures, especially in supporting users comments, but I agree asking the right questions and then interpreting the answers 'correctly' is an skill/art and very dependent on the product and the audience . We developed the game using a user-centred approach.
Jeanne Brockmeyer and colleagues published an influential article in JESP in 2009, in which they describe the development of the Game Engagement Questionnaire: https://www.researchgate.net/publication/223009530_The_development_of_the_Game_Engagement_Questionnaire_A_measure_of_engagement_in_video_game-playing
Boyle et al. did a systematic review of engagement measures: https://www.researchgate.net/publication/230641900_Engagement_in_digital_entertainment_games_A_systematic_review
And if you really want to dig deep into the concept of video game engagement, Andrew Przybylski and his colleagues wrote a highly influential paper: https://www.researchgate.net/publication/232497058_A_motivational_model_of_video_game_engagement
Hope you find this helpful.
Cheers,
André
Article The development of the Game Engagement Questionnaire: A meas...
Article Engagement in digital entertainment games: A systematic review
Article A Motivational Model of Video Game Engagement
I think Paul Cairns at York Uni does stuff on this kind of thing. His main interest is immersion and, depending on your definitions of immersion and engagement, some of his stuff may prove useful to you.
Thank you Dana. & Guillaume, thank you too - I definitely think games are going to become evermore reactive to real-time measurements taken from players.
For my research I have decided to use the Game Experience Questionnaire, I am keen to hear from anyone with experience of using it. Many thanks to everyone for your input to date.