Problem 1: Qualitative Usability Evaluation (in-lab)

Part 1:  (5 minutes) Select one of the following tasks to for your usability evaluation:

• Enabling two-factor authentication (2FA) at a popular social media platform of your choice.
• Sending money to a new contact using e-transfer using a mobile banking app of your choice.
• UBC CWL password change process.

Part 2 (1 hour 10 minutes): Evaluate the User Experience

Steps:

1. (20 minutes) Watch first 16 minutes of the introduction to qualitative evaluation that explain Cognitive Walkthrough, Heuristic Evaluation, and Persona Based Evaluation.
2. (20 minutes) Collectively develop two personas for your target user, considering their likely tech experience.
3. (15 minutes) Each member of the team performs individually a Persona-based Cognitive Walkthrough of the task  selected in Part 1. Everyone on your team should use the same task but can select any of the two personas you identified. All members of your team can perform this step concurrently.
   • Discuss findings of each team member.
4. (15 minutes) Each member of the team performs individually a Heuristic Evaluation (using 10 Usability Heuristics for User Interface Design) of the task  selected in Part 1. Everyone on your team should use the same task. All members of your team can perform this step concurrently.
   • Discuss findings of each team member.

Part 3: (40 minutes) Write a 2-5 page report that includes the following:

• Cognitive Walkthrough
  • (2 points) Two personas for your target user.
  • (2 points) All the requirements of the cognitive walkthrough (e.g., task description, list of actions to complete the task, etc.)
  • (4 points) The outcomes of the Cognitive Walkthroughs, including
    1. The answers to the four “will users” questions for each action to complete the task, and
    2. Any potential issues members of your team identified (i.e., anything that could prevent the user from completing the task correctly in reasonable time).
• Heuristic Evaluation
  • (4 points) The outcomes of the Heuristic Evaluations, including any potential issues members of your team identified (i.e., anything that could prevent the user from completing the task correctly in reasonable time).
• (4 points) Your recommendations for how the user experience with the task could be improved to alleviate the issues you identified. Separate recommendations that came out of the Cognitive Walkthrough vs. Heuristic Evaluation.

Problem 2: Quantitative Usability Evaluation (at-home)

A technology company has asked UBC researchers to improve the design of the User Interface (UI) of one of their security-related products. The researchers have come up with a new UI design and have performed controlled experiment to compare its usability to the original UI. In their experiment, they recruited 11 participants and asked them to perform 2 separate tasks using both the new and the original UI. They measured three metrics in their tests, the results of which are presented below:

1. Efficiency / Completion Rate (how many participants were able to compete each task successfully with each UI)
   

   Task	Original UI	New UI
   Task 1	11 participants succeeded	10 participants succeeded
   Task 2	11 participants succeeded	11 participants succeeded

2. Speed (how long it took each participant to complete each task with each UI). The results are provided in this csv file (the unit of the numbers is seconds).
3. User satisfaction (the total score each participant gave to the ease of performing each task with each UI, using the After Scenario Questionnaire (ASQ)). The results are provided in this csv file.

Write a 1-2 page report explaining:

• (3 points) Explain the following:
  • Hypotheses tested in each of the above experiments.
  • Independent and dependent variables for each of the above experiments,
  • Whether it was between-subject or within-subject study.
• (4 points) Comparing the mean of each metric for each UI, explain if you can identify any significant difference between the two UIs, in any of the metrics.
• (4 points) Assuming the data are normally distributed, conduct t-test analysis and explain whether there is any statistically-significant difference between the two UIs, in any of the metrics.
• (4 points) Given that the UI is for a security-related product, which UI would you recommend for performing each task? Justify your answer.
• (2 points) If the UI was for a non-security product, would you change any of your recommendations? Explain why.
• (4 points) Based on your analysis, could you make a general recommendation on which UI should the company choose? If so, explain which one and why? If not, what other experiment would you need to perform to be able to make a definite recommendation?