Author Archives: roger luk

Update 6c: Revised Supplementary Experiment Materials

We added a script so that our studies could be conducted in a consistent manner.

Script

Minor changes were made to the consent form to fit the context of our study, such as adjusting the time the experiment should take and what the participant should expect to do.

MedFiconsentform

We made separate questionnaires for each one of our interfaces and also a post study one. Questions were also reworded for clarity.

GulfOEQuestionnaire

We compiled a list of all the comments on both videos for the participants to use during the search for the comments.

ListofComments

Update 3e: Proposed Goals of Experiment

The list of potential goals for our evaluation are as follows:

  1. How error-prone is the system and how clear are the steps to perform a task? We could measure how much users hesitate or make with the design in accomplishing a set of tasks. This value could be compared against a set threshold of number of hesitation points and number of errors.
  2. How quickly can a user perform a task? We could measure the time taken for a user to perform a task, as an indication of the efficiency of the system. The time taken can be compared against predetermined threshold times.
  3. What do people think of the new system compared to existing platforms? We could measure this using a likert scale to quantify the user’s willingness to use the design compared to a neutral threshold.

 

Ranking of Importance:
3,2,1

Item 3 is most important because people need to be willing to use the system in order for it to be a useful system to develop.

Item 2 is next most important because efficiency is related to how usable the interface will be- if the task takes too long to perform, then the interface will suffer.

Item 1 is least important since although users may hesitate at certain steps, the interface is meant to be forgiving and the consequences of hesitation or making an error is minimal.

 

Ranking of Testability:
2,3,1

Item 2 is the easiest to test because we can measure times and compare the number obtained with the threshold times to complete a task.

Item 3 is more difficult because we are relying on a user’s opinion, which might not always be completely truthful. However, if we collect their opinions using a Likert scale, it will be easier to analyze these results.

Item 1 is the most difficult because it could be hard to determine when a user hesitates.

Based on these ideas, we have decided to focus on goals 2 and 3 for our evaluation, especially since idea 1 ranked lowest on importance and testability.