I could answer u in a 100 ways. Sry. I am an open minded inter- and transdisciplinary thinker. Sometimes a bit crazy... So: That´s why i am asking you: "Which performance term do u wanna use?"

Just a few more sentences pls about: I just call it "Strategic steps 2 embed thoughts"

First: Call the paradigma, your view bout science and your view to handle the term and yourself by "doing science" (i.e. using the performance term mabye based on a action theory in an anthropological layout using a qualitative metric)

Second: Call the structural properties and "key terms - concepts" (constitutive properties) that contribute to the sciencific community, which shall be related 2 answer your question, the question of other researchers and the non-scientific community (Willimczik could help).. Sry i don´t like research without sense and / or use .

In other words "open the field of research maybe by asking yourself bout the interest of research". Try to triangulate your personal interests about the performance term and your usage of the term and let us know your thoughts and interests."

Or in a more simple way: Just loose a few more words about

2. Research field, Domain, Discipline, Field of Intervention, where to use the implications caused using the term in a systematic manner

3. and your Theoretical Apporach

4. The metrics will follow itself... :)

Sincerely Steve

Short Way:

Steve Funke - TU Chemnitz Germany

Sports- and (Media-)Education scientist, Ergonomics and Human Factors Research, Media Psychology, also Business Studies, Philosoph with Interests in SG Design, Workflow and I am a Player of Bad "Shooter Games" that are made by the US Army (Military SG Games hehe )

Themes:

Learning Performance, Movement / Action / Motor Performance, coordinative performance, "Performance as a term to desrcibe the effectiveness of agents and processes to provoke states" , Instructional Design Performance

Motor Tests, Learning Tests, Usability Tests

d(/^_^)b

One question: Do u use the term SG 4 a computer based digital game or in a more general matter without any technology. Maybe SG as utilized Game, where Learning without recognizing the normativ layer becomes indoctrination or Playing a game becomes work? Games as simulated realities and Playing as Mimesis?

Did u already scrutinise the idea of men that stays in background of using the performance term? Did u ever thought about Cognitive/ Mind Performance? Body Performance? Aesthetic Performance? Performance of Education based on SG - SG Design Approaches.... Producing SG´s and teaching how to think about SG and conduct reseach is, in a strategic way (Seen by the Education policy) it might be cheaper to invest in SG, then i traditionell education settings...

i.e. easier then to make a film, write a book or instruct kids in front of the classroom? Didactic invests and educational efforts.. "The Suitability of the Du Pont System to evaluate educational performance / performance of education management processes?" . "Instructions as Investigations into the human capital ressources / Enhancing abilities 2 work, 2 live, 2 play, 2 learn, 2 computate, 2 communicate etc..."

And which Performance do u want to Improve by utilizing games?

Improving Performance of Thinking, Moving, Acting (i.e. Playing a Role in a theater)?

Second question: What kind of usage do u prefer? Handling performance only as a process or only as a result?

Third question: If u were a teacher in school.. Would u grade or evaluate the performance?

@Christian S Loh

I want to interpret your phrase "serious games improve performance" correctly. Suppose, you are a school student and you want to use a Serious Game for one month to improve your intellectual abilities, e.g. mental arithmetic, attention concentration, decision-making, and so on. Suppose, after one month you improved your mental arithmetic and other's abilities, therefore you could receive the best evaluations on mathematics. Now I want to ask you: Can I use the improved mental arithmetic, attention concentration, decision-making, ... as the metrics "to measure performance improvement occurred" ? If your answer is YES, I want to use opportunity to offer to your consideration my "Bin Pacling Online Tool" (as the Serious Games) on the link http://www.fedulov.ge . Currently, this site is hosted by AccuWebHosting.com (USA, Denver). Please share your opinion.

Thank you in advance

Gennady

Thank you for your response!

I understand Steve's method which is to define what performance is (cognitive,etc) and then measure what you think should happen. This is the preferred academic research / dissertation method.

Michael's response is in a similar vein, asking the pretest - intervention -posttest, so if there is a change in the posttest, then it must be cause by the intervention. While this approach work for most education, it may and may not work for game-based learning. (Too many things going on in game, so one can never be sure IF the change in posttest is caused by the 'instruction/learning' or the game, or something else (outside of game). [This is the consensus from the AECT 2012 discussion panel on "Assessment for Game-based Learning".]

Actually I am thinking very broadly here. Let me use entertainment game as an example, if I said:

1) high score (Tetris),

2) best completion time (Halo flag capture),

3) secrets discovered,

4) challenges completed (Tomb Raider),

5) number of side quests competed (Skyrim),

I am guessing any gamer would understand straight away that these are the performance metrics used to measure their performance in those games. So there are a 'loose set' of metrics available to judge performance in entertainment games. High score for example, is applicable in many games. Not just Tetris.

So what about serious games? Without going into the details of any serious games? Is it some kind of generic performance metric one can use to 'measure' the performance of serious game, something that any players could easily understand? Is it 'best time' (how fast play-learners solve the problem), number of mission accomplished? Percentage of procedures followed? Number of checkpoints visited? The route traveled? Decision making procedure (steps taken)? How about comparing the performance of players against some kind of pre-established baseline?

[Some of the metrics mentioned above are actually drawn from military serious games.] I guess I am gearing this towards workplace training rather than classroom learning...

Or perhaps, you think such generic metrics are not possible/practical because serious games is different from game for entertainment... [Granted, these metrics may have to be tailored to match each game, but can we discuss broad/generic metrics?]

> Such alternative explanations can be narrowed down by using a control group.

My suggestion would be for researchers who are comparing one group of student with one Intervention against another group of students doing a second intervention to FIRST become familiarize with R.E. Clark's critiques on Media Comparison Studies (MCS).

The MCS literature/debate has been around for more than 30 years and is very well known to instructional deisgners, though not so among other educators...

As you suggested, control group is the answer... but this is true only in clinical studies and scientific research because 'control' is almost impossible in social sciences (like educational research). Since human are 'social,' many people learn from outside sources (apart from the intervention environment), e.g., Internet (Google/wikipedia), asking their friends for answers, copying...

>For example, in my own studies I have been examining the role of fantasy role-play

>in an e-learning environment compared to the same e-learning environment without >the fantasy role-play elements.

>How can one be sure, then, that the difference was not something else outside of

>the game? Is either condition significantly more effective that not using any game >whatsoever?

You are right, one can never be sure, hence we need to place limitation and delimitation parameters around our research.

>A second control group, that does not use any tool at all, can inspire confidence that >the interventions are having an effect.

Here, I believe you just place yourself into Clark's definition of a Media Comparison Study. Comparing a non-game classroom with two gaming/e-learning classroom is 'meaningless', by Clark's definition. Because the result tends to be 'no significant differences statistically'. It's hard to argue against Clark because his conclusion is based on meta-analysis of many people's research.

I am not interested in debating what is MCS, only to point out that there is a solid literature already in existence on this issue. Those who are interested should take a serious look... and avoid doing research that are comparing one class with games against a class with no game. (It work with medicine and clinical studies, not in education).

** Do a search on google using "clark media comparison", and read references no. 1, 3 and 5 to start learning about MCS.

Also: Hasting and Tracey, TechTrend 2004, http://www.ecoisonline.org/file.php/8/lesson_3/required%20reading/does%20media%20affect%20learning.pdf

I like your way of looking at it (for the performance difference/improvement section). Its the control study (third group) that I do not think will help because it sounds just like media comparison studies (of which "control" group is a staple).

How about let us look at the question using the following scenario? Let's say we are using Portal 2 in a research and want to measure performance improvement of players. What (generic) performance metrics would you suggest?