Type of assignment: This is a group assignment. This assignment must be done in same groups as the term project. Only one submission per project group will be considered.
Points: The maximum number of points for this assignment is 60, which will be prorated accordingly after all assignments are posted. Weight of each problem is in parenthesis.
A. Malware Analysis (20 Points): Pick one of the following malicious software (malware):
2. Hello Kitty
- Post a public message (one message per group) on Piazza indicating which malware your group has picked.
- If another group has posted an earlier message, indicating that they had picked the same malware as your group, go back to step 1 and pick “free” malware.
- Analyze the design of the picked malware, i.e., its structure, behavior, mechanisms it uses for selecting victims, infecting them, hiding itself and its authors, etc.
- Identify the aspects specific to your malware that can be used to detect or prevent it.
- Based on your analysis, come up with short, medium and long term countermeasures (i.e., protection and detection techniques) against the threat posed by this malware.
- Identify which of the principles for designing secure systems have not been followed by the developers of those systems whose vulnerabilities allowed the malware to become effective.
- Use no more than 4 pages to write a short report documenting your findings obtained in steps 3–6. Use figures and tables, if necessary to achieve better impact of your report. Note that although you can use existing reports in crafting yours (given proper citation), you are expected to perform some analysis of your own, as well.
B. Usability Evaluation:
B1. Qualitative Usability Evaluation (20 Points): Perform a cognitive walkthrough of the task of “obtaining a COVID-19 vaccine card through BC government website“. In doing so, assume that the general users of the system have high-school-level education with no particular computer expertise. Write a 2-3 page report that includes the following:
- A description of the persona you developed for the target users.
- All the requirements of the cognitive walkthrough (e.g., task description, list of actions to complete the task)
- The details of how you conducted the cognitive walkthrough session(s), including the number of participants and their backgrounds, the length of the session(s), and the general procedure that were followed.
- The outcomes of the cognitive walkthrough sessions, including (1) the answers to the four “will users” questions for each action to complete the task, and (2) any potential issues you identified (i.e., anything that could prevent the user from obtaining the vaccine card).
- Your recommendations for how the process of obtaining the vaccine card could be improved to alleviate the issues you identified.
B2. Quantitative Usability Testing (20 points): A technology company has asked UBC researchers to improve the design of the User Interface (UI) of one of their security-related products. The researchers has come up with a new UI design and has performed silent observer A/B testing to compare it to the original UI. In their experiment, they recruited 11participants and asked them to perform 2 separate tasks using both the new and the original UI. They measured three metrics in their tests, the results of which are presented below:
- Efficient / Completion Rate (how many participants were able to compete each task successfully with each UI)
Task Original UI New UI Task 1 11 participants succeeded 10 participants succeeded Task 2 11 participants succeeded 11 participants succeeded
- Speed (how long it took each participant to complete each task with each UI). The results are provided in this csv file (the unit of the numbers is seconds).
- User satisfaction (the total score each participant gave to the ease of performing each task with each UI, using the After Scenario Questionnaire (ASQ)). The results are provided in this csv file.
Write a 1-2 page report explaining:
- Comparing the mean of each metric for each UI, explain if you can identify any significant difference between the two UIs, in any of the metrics.
- Assuming the data are normally distributed, conduct t-test analysis and explain whether there is any statistically-significant difference between the two UIs, in any of the metrics.
- Given that the UI is for a security-related product, which UI would you recommend for performing each task? Justify your answer.
- If the UI was for a non-security product, would you change any of your recommendations? Explain why.
- Based on your analysis, could you make a general recommendation on which UI should the company choose? If so, explain which one and why? If not, what other experiment would you need to perform to be able to make a definite recommendation?