After completing Milestones I to IV, we reflected on our design process and had several points to consider:
- Our design and evaluation focused on an aspect of a collaborative video platform but not the entire system. The original vision of our design was to create a collaborative video platform, to help users watch educational videos and seek useful guidance when a concept was unclear. At times, it was difficult to realize the relationship between low level details of our design and the high level design goals. For example, timing users completing the video task could show us if interface type affects how users complete tasks in a video. Similarly, comment search time could be considered as a metric of how easy it is for users to seek help. In hindsight, we could have focused on testing different aspects of the system, that would help reinforce the relationship between the low level details and the high level design goals.
- When developing the medium-fidelity prototype, we felt restricted by our ability to implement it in Javascript and HTML. Specifically, we wanted the segments to be labelled on the video which we could not implement due to our limited experience with the Youtube player API and time constraints. The visual appearance and graphic design of the website also suffered due to our lack of experience designing websites.
- One of the things that was surprising when conducting the study was that users’ abilities varied greatly. Although piloting indicated that the difficulty of the tasks was reasonable, during the actual experiment, some participants struggled greatly with the task while others found it easy. As a result, user performance variation should be considered more carefully when designing a study.
- Lastly, we could have performed another field study to see how people would interact with our segments and annotation system. We found that when designing our experiment, we had to make many assumptions about how annotations would be used and the type of content that they would hold. Having a better idea of this would have allowed us to develop a stronger and more valid experiment.