Depends on the domain, task, and goals. Can you be more specific? Take a look at Think-aloud protocols, and Task Analysis methods and research used in HCI (the human factors engineering literature, see work by Bonnie John). If you can decompose the cognitive and task processes of experts, you have a chance of understanding the design elements required (factors) to build a successful simulation and then empirically test it. Basically, you need to fully understand the process, as it is thought about, and how it is executed by the expert. 

Dear researcher,

In implementation of learning method, each has the requirements to be completed. There are three main steps: planning; action; and observation with evaluation. So, below, it is available for strategies to measure teaching effectiveness.

Kind Regards.

Hi, do you mean effectiveness regarding behavioral change or psychomotor skills?

You need to design experiments to evaluate different methods of teaching and learning using different simulations in different teaching and learning environments and analyse the students progress using assessment tools

Any methods effectiveness should be evaluated by its expected outcomes. The type and level of expected outcomes is crucial. Are you using simulation because student cannot  enact the experiments by themselves (because it could be too dangerous, they may lack the needed resources and guidance, simulation may shorten time, etc.)? Or are you using the simulation to raise situational awareness and consciousness about the nuisances of "real" complex healthcare systems? Measuring the effectiveness of simulations that are acting as replacements is relatively easy: use as a guide the methods that you would be using for obtaining the same outcomes.

Evaluating the effectiveness of simulations used to raise situational awareness and consciousness is a different thing. You will probably would like to identify prior to the simulation the things or aspects that you expect people to be aware of and then create a pre-test and a post-test to measure change. And you can also use the Most Significant Change (MSC) method to detect unexpected outcomes. 

Dear Dr Wang 

Please feel free to search on my profile in Research Gate...you will find some interesting ideas of what you are asking for.

With kind regards

NP

Thanks for all the suggestions!  I am part of the faculty team who are actually implementing simulated scenarios, and too often, the scenarios are focusing on specific skills or following medical orders. I want to maximize its potential to achieve better student expeiences and outcomes.  Dr Miftachul Huda, thanks for your sharing! Dr Nikolaos Pellas, you've done a lot in viutual simulalted learning.    Dr Elizabeth Vega, thanks for suggesting me to visit IANSCL.    Dr Maria C. R. Harrington, you are right about the Think-alond process, I am working with a American lady who helps us to design the case, it is really helpful to hear different voice.     Dr Sokratis Nifakos, we want to focus on hgher layers of learning, such as critial thinking.  Dr  Samina Rajper, we want to focus on the learning process of students. Thank you!

HI, I am coming quite late on this discussion but it is a topic of particular interest to me as I have done some research on the effectiveness of scenario-based simulation in healthcare education. Rather than the simulation on its own, the debriefing and reflection plays an important role in the learning and comprehension for the learners, particularly when it come to decision making and critical thinking. Whatever research method you adopt, it needs to encompass the debriefing element, and not include debriefing with a control group so you could actually measure the contribution of that particular phase of learning... some work has been done in that area but there are many factors such as the type of simulation "facilitation" and the debriefing model used. Best Wishes with your research.