RAA 1: A Comparison of Usability Methods for Testing Interactive Health Technologies

Jaspers, M.W.M. 2009. “A Comparison of Usability Methods for Testing Interactive Health Technologies: Methodological aspects and Empirical Evidence.” International Journal of Medical Informatics 78 (2009): 340-353.


Jaspers  identifies two major types of expert evaluation: heuristic evaluation and cognitive walkthrough. Heuristic evaluation is a relatively efficient usability evaluation method with a high benefit-cost ratio, but requires high skills and usability experience of the evaluators to produce reliable results. Heuristic evaluation is widely used since it generally requires less resource than testing with users. But it is fundamentally different from usability testing. The cognitive walkthrough is a more structured approach than the heuristic evaluation with a stronger focus on the learnability of a computer application. The limitation is the required level of detail of task and user background descriptions for an adequate application of the latest version of the technique.


User-based testing methods include user performance measurements, log-file and keystroke analyses, cognitive workload assessments, satisfaction questionnaires, interviews and participatory evaluation. Among these methods, the participatory evaluation can give the clearest indication as why a certain interface aspect poses a problem to a user or how to improve the interface. For example, through talking or thinking aloud or in a retrospective interview, the participants can reveal the underlying causes for system usability problems encountered and participatory evaluation thus has more reliability. Scholars such as Jaspers regard “think aloud” as a very direct method to gain deep insight in the problems end users encounter in interaction with a system (340). Many also suggest that test participants need to be diverse but representative of the envisioned users.


Heurstic evaluation have a higher benefit-cost ratio, but it require higher skill set in usability experience. The cognitive walkthrough  is more structure than Heurstic but need to have high level detail of task and background. And the think aloud technic is great way to get in-depth detail on problem which user are facing.

Continue discovery, research and hand on experience in this area of technology is a must.


Author: eddyzhuo

Think about your day, seen like brake into two part: online -- offline. And then think about your time when you are shopping in the grocery stores, you will look what is better with a better price. Same thing apply to surfing online, we should be able to identify what is good and bad website, use good product that worthy your time, instead only being passive. Hello my name is Eddy, a master of computer graphics technology student in Purdue University. Let us use the web, think about it, discuss on it, and create a better and friendlier user experience WWW.

2 thoughts on “RAA 1: A Comparison of Usability Methods for Testing Interactive Health Technologies”

  1. So, is this an essay where they make arguments based on existing evidence? Am I missing something, or is it correct that they did not collect any new data for this study? This is not clear in your summary.

    1. This article is more like a walk through and comparison of many literature on these three methods: 1.the heuristic evaluation, 2. the cognitive walkthrough, and 3. the “think aloud” method. It did not collect any new data. Thank you for pointing it out.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s