Design problem- If we have 3 UI concepts: A, B and C and we would like to quantify which one is better out of the three, can heuristic evaluation be used? Constraints: only low fidelity prototypes are available and context is based on participants' envisioning power. Also, how large the sample should be to prevent inter-rater bias?
We have used HE as a summative evaluation method always. It will be challenging to use it as formative evaluation method with low-fidelity prototype. It's Expert-based method and 3-5 evaluators can participate. Some of the heuristics such as Real-world match, Error prevention, Efficiency of use and Minimalist design, may be applied on prototype for evaluation. All the best !
Thank you Ganesh. How about the effect of providing context while heuristic evaluation is done? Is it okay if we ask participants to imagine themselves in a scenario or an actual context should be provided?
Typically, heuristic evaluation will use experts (those skilled in the system or at evaluation, or both). Therefore, I think the participants will be able to exercise some modicum of imagination (pretending the fidelity is higher). However, if people typically rely on high-end graphics to navigate the system, I believe you could possibly get results poorer than expected.
It is a good option to use HE mainly if you have low fidelity prototypes, an inexperienced user might not be as recommendable, in our study we worked with 5 evaluators, which is within the recommended range (3 - 5) and very good results were obtained. be experts you can expect from them a lot especially for the conditions of the prototypes that you mention
Thanks everyone, just to add another discussion topic, let us say if I explain the system to all users, can I reasonably say that all users can act as an expert for HE? Or in other words, who qualifies as an expert?
Dear Mr. Miglani,
I have participated twice in heuristic evaluations on non-functional prototypes. Even though, in both cases, the links were not programed yet, it was possible to analyze structure, taxonomy and cognitive aspects just by checking visual pages representing a 5 depth layer structure. But all pages were high fidelity prototypes. The 5 evaluators found more than 200 usability problems in total.
In cases where the project is still on sketch prototype phase, I think a better direction would be to either use think-aloud protocol or cooperative evaluation. The results will be less numeral for comparison between A-B-C, but will bring more depth insights on users' mental model and expectations.
This is a recent article about using Think-aloud Protocol:
Conference Paper Affordances and Gestural Interaction on Multi-touch Interfac...
But Villanueva has a whole dissertation about it too.
Best,
Adriano Renzi
Hi,
it's a common misunderstanding that HE is an expert technique. Reviewers need to be briefed about the heuristics, but in principle anyone qualifies. Of course experts are better, in particular domain experts (you need accounting knowledge to review an accounting app). It's more important to have multiple reviewers, as each one catches only a subset of issues.
You may also consider Cognitive Walkthrough (https://experience.sap.com/skillup/debugging-design-cognitive-walkthrough-with-build/; also works on paper),
or, to work with end users, Click-Thru Interviews (https://experience.sap.com/skillup/usability-on-the-fast-track/).
- Badges
- Science topic
- Similar topics
- Geoscience
- Asia
- India