Blog Update #6a – Pilot Test:
In our pilot test, we found out that more preset/canned responses were necessary for the range of questions that the user could ask in the chat. Our previous repertoire of responses only covered the basic greetings, museum information, and travel directions. We expanded it to include other topics such as public transportation, pricing, restaurants, etc. We felt that this would help cover the limitation we have in a static map view with hard-coded markers. We also found out that the participant may respond better if the chat interface was not entirely AI. With recommendation from our TA, we decided to change the concierge to be a human/bot hybrid. As a result, whether or not the participants we looked for are more easily convinced that we used a full AI was no longer applicable. Furthermore, we decided to move our experiment setting to that which emulates the bustling and active environment of a hotel in order to help the participant immerse in the task scenarios. We hoped that this would generate more accurate results than a lab setting, and decided to designate areas in the NEST and nearby cafés for holding the experiment.

Blog Update #6b – Experiment Abstract:
We investigated the use of a chatbot to act as a responsive concierge for users planning trip activities on the go. It was our expectation that this service would provide customized and well-informed responses to users and help improve efficiency. Our experiment aimed to compare two interfaces, our Locl Bot prototype and traditional resources like Google Maps, with 5 participants in order to evaluate the the pros and cons of each interface via the task of finding a museum to visit in a foreign city. The results collected from time spent, satisfaction, and frustration ratings showed that there was no significant difference between our prototype and the traditional alternative. This was likely due to statistically weak data from the small sample size and various technical issues that may have affected the results. In the future, we hope to overcome these limitations in design with improvements and to validate our hypotheses.

Blog Update #6c – Revised Supplementary Experiment Materials: