Evaluate

Fifth project milestone: Evaluate – Cognitive Walkthrough and User Test

Test the low-fi prototype for usability bugs (problems, faults, weaknesses) by performing both a brief cognitive walkthrough and a small informal user test, using the task examples supported by your prototype.

 Step 1: Cognitive walkthrough (in-class activity)

  • Note that while the walkthrough is typically performed by the interface designer and a group of his or her peers, you should consider having one or more of your team members perform the walkthrough together with two or more classmates from another team (e.g., swap team members for the walkthrough).
  • Walkthrough Oral Report: Toward the end of the class, each team will briefly summarize what you learned in your walkthrough, good and bad. Your assessment should focus on your task examples.  If you found nothing wrong for a given task, (i.e., your interface is perfect) then outline the ways in which it was perfect (e.g., “Our cognitive walkthrough showed that users can do X, Y, and Z without errors or confusion.”).
  • You should leave the working class with a list of minor issues that you will address based on the feedback. (There is not sufficient time to address major issues/start from scratch.)

Step 2: Informal user test

  • Recruit 3-6 representative participants (depending on how easy it is for you to get participants) to a) evaluate both your low-fi prototype and your conceptual model, and b) validate your task examples.
  • Develop a simple user testing protocol that include a short list of the goals you have for the evaluation, a simple script for how you will introduce the prototype to the users, the tasks that you will ask them to do, and any interview questions that you will ask users.
  • Run participants through your protocol. It is best to have at least two team members at each session: one for managing the session and the other for taking detailed notes. (In a real project, you would video record and transcribe these user testing sessions in order to replay them with the entire team for more detailed analysis. There is not enough time to do this in this course.)
  • Summarize what you learned from the user testing, the strengths and weakness of your design; assess these according to how critical they are (show-stopper, major, minor).

Deliverable: User Test Report

  • Briefly describe your protocol and participants to the extent that another team could replicate what you did.
  • Describe the outcome of the user test, including the strengths and weakness of prototype based on feedback from users.
  • Reflect on how well your conceptual model was received, and whether you were able to validate your task examples.
  • Length: 3-4 pages, 11-12-point font, 1.5 spacing
  • Submit a pdf of the report, filename: “Evaluate-usertest-report-<team name>.pdf”