Blog Update #7a – Final conclusions and recommendations:
Overall, we believe that the project drove our interface towards using sound design concept based on the elements we uncovered in our field study. However, our experiment could have yielded better results to support our hypotheses and validate the prototype interface. We have identified several aspects of the experiment that require further revision in order to remedy the statistically weak results we see. Nevertheless, the project produced interesting information that could help guide future work in the field. We gained insight into the user mentality and how different information types are considered as crucial for the map/chat aspects of the interface when planning for trip activities, thus providing clear connections of affordance between key elements that our intended users look for. We believe that improvements in the prototype in terms of robustness and overcoming technical limitations would greatly help take our research to the next level.

For recommendations on future research and to validate our interface, we believe some minor adjustments are needed, but the overall approach has been validated. As we have seen with previous research in the field, our hypotheses and overall tests are on the right track. However, it is important to extend the capabilities of our medium-fidelity prototype and revise certain aspects of the experiment. Overcoming technical issues with chatbot behavior/intelligence, application freezes, and dynamic updates would help produce more accurate data. Researching the relevant factors for expertise may also yield helpful information that aid in the experimental design.

Blog Update #7b – Reflection on your design and evaluation process:
Upon reflecting on the design process, we believe that the most significant ways in which the design concept and actual interface changed under the influence of the user was getting the map to be the main focus of the UI. Previously, we had wanted to only use the chat for everything. However, users have pointed out the importance of map information, which shifted our focus to include proper map implementation. Some of the biggest surprises we have seen include the fact some users almost exclusively use the map, even when the chat is also available. Another is that a user ended up spending a long time with the chat because they wanted to test the limits of the AI. For the most parts, the methods we chose (think-aloud task followed by semi-structured interview questions) were very effective. This also includes the way we recorded data using coding sheets, screen capture, camera, and logs. However, it may have been interesting if we decided to account for trip planning expertise by looking at pair/group planning and adjusted the task execution to test for that dynamic. A particularly unhelpful method we used was audio recording, since the notes we took were very detailed, the recordings seemed superfluous.