Supplemental Resources for Designing E-Learning: How to Respond to the Results of a Usability Test

the studio of
saul
carliner
content development
for the workplace

resources for
designing e-learning

Chapters: 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | Home | Glossary | Links | Order the book

You are here: Ideas and Issues in E-Learning --> Supplemental Resources for the Book Designing E-Learning --> Chapter 7 --> How to Conduct a Usability Test of the Prototype --> How to Respond to the Results of a Usability Test

How to Respond to the Results of a Usability Test (Part 3 of 3)

After the test, as participants complete the test scenarios, usability engineers review the observation logs and identify potential problems. The engineers separately log the potential problems. Usability engineers also log the problems identified through the debriefing interviews.

After the last participant finishes the test, the usability engineer goes through the log of potential problems and reviews them. In most cases, users can achieve the main and supporting objectives, but resolving certain issues that arose can help other users achieve those objectives more easily. The usability engineer identifies the issues and rates their importance as follows:

A	Showstopper. If you do not resolve this problem, learners cannot accomplish the intended goals of the learning program. An example of a showstopper is missing content. A problem identified by all of the participants is a likely showstopper.
B	Urgent. Learners are not likely to correctly complete a lesson. An example of an urgent problem is an illustration telling learners to press button A , even though the written instructions specifically state button B. This confused learners in the test. A problem identified by most or all of the participants is likely to be urgent.
C	Nice-to-do. Nice-to-do problems fall into two main categories. In the first, users will likely be able to accomplish the goals of the learning program if you do not resolve this problem, but you can increase their satisfaction if you do. In the second, one or two users might have offered a suggestion but others might not agree with it. A problem identified by just one or two participants is likely to be a nice-to-do problem.

The usability engineer presents a report of the findings, with a list of problems and their classifications. You would then prepare an action plan, which indicates how you intend to address each of the problems identified and into which draft you will incorporate the changes.