Through multiple empirical studies, I have collected user needs for an ICT intervention. During this study, I intend to design a prototype and then evaluate the prototype to check whether user needs are captured in the proposed design.
What is the most suitable approach? Quantitative, Qualitative or mixed?
Are we evaluating the features of the prototype or evaluate the user requirements?
You may use - SUS, PSSUQ, net promoter score, USE questionnaire etc.
There are many research techniques you can use to test UX of a prototype. Qualitative research often suffice to bring valuable insights based on users interaction with the prototype. The important thing is to bring people that are representative of the persona for the project.
Think-aloud protocol can be performed with 4-6 people and already bring very interesting data from users' perspective. But due to covid times and remote research I've been finding that cooperative evaluation can be more suited for remote research.
Here is an example of using think-aloud protocol:
Usability and Interaction Evaluation on Breakfast Delivery Mobile App: Users’ Experience Expectations
DOI: 10.1007/978-3-319-94947-5_57
And here is an example of cooperative evaluation within an extensive set os research methods:
Desate Nó: Management System App for Buildings
DOI: 10.1007/978-3-030-20040-4_51
I hope these can help.
All the best,
Adriano
Thank you so much for your valuable input Adriano Renzi Subhankar Banerjee
Adriano Renzi and Subhankar Banerjee, do these mechanisms help to evaluate whether the proposed features in the prototype satisfy the user need? or they check the interaction and the overall usability of the system
Hello Manoja Weerasekara , even though think-aloud protocol is considered by some authors as a widely used method for usability testing of software, interfaces, websites, and (instructional) documents. It is possible to perceive the satisfaction of users by his/her expression, frustrations, frictions and comments. An interview or Likert scale could be added after the Think-aloud procedure, focused on user's satisfaction with the experience.
The basic principle of this method (Think-aloud protocol) is that potential users are asked to complete a set of tasks with the artefact tested, and to constantly verbalize their thoughts while working on the tasks.
From all possibilities of prototype testing, the Think-aloud Protocol is the most
suitable choice to help observe usability and interaction problems from users’ point of view. According to Villanueva [4], the technique consists of a researcher observing 1–4 users doing specific tasks within a controlled environment. The user’s actions and thoughts are to be described verbally aloud by him/herself on real time. The researcher records the user actions by written notifications, video or voice recorder.
The participants are to perform a simulation of tasks using the prototype. The Think-aloud Protocol has to be executed individually and the steps of the task to be closely observed with participants verbalizing actions, thoughts and confusion: do users understand what is the app? Do users cognitively understand icons and actions to take? What is the visual perception of users regarding colors, types, size, icons and visual impact? Is the process an easy and smooth experience? How do users explore the options?
Nevertheless, I've been feeling that for remote research, cooperative evaluation can be more useful e easier to execute.
Manoja Weerasekara
Usability testing only can compare a product with another. Higher score means better usability. But it cannot evaluate whether the proposed features in the prototype is satisfying the users.
You can ask few questions related to user satisfaction. May be as an open ended questions or a likert scale.
You also may use and correlate NET promoter score and NASA task load index with above-mentioned scales/questionnaire, if .