Blog Update #7 – Final Recommendations & Reflections

Blog Update #7a – Final Conclusions and Recommendations

 

When comparing proximity versus geolocation methods, we had found that both methods were valid approaches when implemented for social gathering applications, more specifically for connecting musicians with each other. The geolocation approach was found to be more user-friendly by the transfer effects from other applications. On the other hand, we had theorized that proximity could create bonds between users. However, the result remained inconclusive due to the deviated direction of the experiment design.  To summarize, it was concluded one was not better than the other for populating users in a social application and was found to primarily depend on user preference when choosing between the two methods. This could have been due to our experiment design implementation, which could have handled the execution in a more controlled manner, like the implementation of two prototypes where one would have implemented geolocation as a control and the other proximity, instead of using a currently commercially available application as the control.

Overall, even though our findings were inconclusive, we found proximity methods to be a valid approach when used for populating a list of users, though adjustments are needed if future experiments are to determine whether proximity is a better approach when compared to geolocation. Current applications in the market have similar use cases for proximity and geolocation for populating a list of people in social applications, however, proximity based-methods were not found to be implemented in social music applications. Further investigation needs to be done to determine whether or not proximity is a better approach in establishing bonds with other users as opposed to geolocation. Conclusive results between the two approaches can help in the development of other future or current FriendFinder applications since the two approaches are not limited to only social music applications.

In an effort to investigate whether the proximity approach would help to establish bonds between users, the next logical step to take would be to run a more in-depth study involving more participants, while having users use the proximity-based application for a prolonged period of time. This will help users feel a sense of realism when developing connections in the application while providing a better approach for studying how a head counter (how many times a user passed by another user) helps users connect with other people.

 

Blog Update #7b – Reflection on Your Design and Evaluation Process

 

Throughout the course of the design process, our actual interface design and approach remained relatively constant. Many aspects of our design were inspired by commercial applications already in existence which validated the interface design and concept in advance. The results gathered from our initial field study supports this fact. Initially, we created two designs for the profile page in the application which were amalgamated into one following the initial field study, but only minor changes were made to other aspects of the interface. Certain features allowing the user to gather a better understanding of the musician ‘at a glance’, such as the music sample and skill level were brought to the forefront of the application based on feedback from the first field study. Conceptually however, the design choice of utilizing a proximity based system to connect musicians with each other remained constant throughout.

A major pain point heading into the development of the medium fidelity prototype that was not foreseen was the ability to dynamically generate the list of matches in the application that the study participant had presumably passed at some point. The introduction of the walk around the building in the second field study was then introduced to convince the participant they had just recently passed other users that were also in the application. Ultimately it was difficult to convince participants these encounters had just occurred and detracted from the reality of the experiment.

Every method applied in this study were effective in some capacity in helping us to validate our hypotheses. A semi-structured interview was carried out initially to help us explore the problem space. Observation and semi-structured interviews were later used to evaluate the usability of our design concept and prototypes. However, the methods chosen for our evaluations would have been more effective in a long-term study with participants using the application throughout the course of a week to two weeks. Also, tasks assigned to the participants to carry out could have been more open-ended and less structured to allow participants to choose the users they personally enjoyed.

Interviews, field study, pilot study, and observation were found to be the most useful methods used in the studies. Semi-structured interviews were conducted twice throughout the course of the project and allowed us to gather insightful information about our participants which we found to be of the most value. Observation and field studies were used once in the project for the purposes of evaluating our medium fidelity prototype. Through observation, we gained a much deeper understanding of the weaknesses and strengths of our interface that we had not seen before. Lastly, the pilot study was another useful method that we used for pre-determining the flaws of our interface before putting it into the real field study. On the other hand, the questionnaire from our initial field study was found to be the least helpful, mainly because the results did not really aid much in the studying of our participants. Overall, many methods we applied were crucial to the success of this study.

Blog Update #6 – Experiment Abstract and Materials

Blog Update #6a – Pilot Test

Pilot Study

Overall the pilot studies went as expected, there were no major changes made to the protocol. However, we did implement a few minor changes based on our discovery from the pilot studies. For example, we rephrased some of the questions from interview and questionnaire script to be more specific since we found that some questions were redundant and confusing to the participants. Second, we added a few things to our medium fidelity prototype, such as age, gender and more description in the profiles as a way to eliminate biases and differences that may influence the result of the experiment between BandFriend and Band&Jam that we were not accounting for in our measurements. The time recorded for the pilot studies were shorter than anticipated, but we expect the real field studies and experiments to be longer. Lastly, we did receive useful feedback from the pilot study participants regarding the instructions that were provided to them. There was initially some confusion on how the applications differed between Band Friend and Band&Jam as the proximity versus geolocation functionality was not explicitly and clearly conveyed. Changes were made to the briefing given to participants in order to better convey the differences between proximity and geolocation before starting the interaction with the applications.

 

Blog Update #6b – Experiment Abstract

Abstract

Through observing how musicians interact with each other, we conduct a field study to analyze how musicians connect with each other for the purposes of forming a band or jamming together and reveal the differences in how geolocation versus proximity-based systems assist in interacting amongst musicians. Fundamentally, we compare two applications, one that utilizes geo-location and other utilizing a proximity-based approach. We conclude by analyzing the differences in user’s performance time and preferences over the two location-based functionalities and its influences on their behaviors and comfortability towards other users in the application.

 

Blog Update #6c – Revised Supplementary Experiment Materials

Revised Supplementary Experiment Details

The questionnaire questions provided to the participant were changed from milestone III to reflect the different experiment procedure and goals. The interview questions asked were also changed to pertain to this experiment. Aside from the general beginner questions asked in the questionnaire related to the background information of the participant, no questions were carried over from the previous milestone.

Provided below is the link to the revised supplementary material (Questionnaire and Interview Questions), pertaining to the details above.

https://drive.google.com/file/d/0BwiLLoppv9ngbS15SjhNQjV0bms/view?usp=sharing

Blog Update #5 – Medium Fidelity Prototyping

Blog Update #5a Rationale of prototyping approach

 

One interface prototype was chosen to be developed to be able to compare to an existing application in the market. Because of this one-to-one comparison, we chose to develop one prototype. To scope the implementation of the prototype, we chose to vertically implement only the functionality that will be utilized by the participants in the field study. Features such as creating a profile, messaging other users, or editing the setting of the application, while being very important for a full implementation of the app, were horizontally implemented as they were not features directly used by the participants. Functionality that was chosen to be vertically implemented was carefully evaluated by analyzing our field study tasks and procedure. From that, it was determined that functionality related to exploring other musicians profiles was needed to be vertically implemented.

 

To limit the scope, we chose to simulate the proximity feature of the application by pre-populating the list of matches in the application for the user. While this list would be dynamically generated in a full implementation when the users would cross paths, we simulate this behavior by still having the participant walk around to generate this list of matches. From the user’s perspective, this functionality looks identical to them within the app.

 

In terms of appearance, since we will have the user give a rating of preference using a Likert scale, we chose to create a prototype that appears “market ready” to avoid any influence that appearance may cause. By utilizing Axure to create the prototype, this allows us to create a visually appealing prototype while also taking advantage of our previous experience with Axure from CS 344.

 

Blog Update #5b Prototype Demonstrations

 

Link to video of prototype: https://drive.google.com/open?id=0B-DflbkztNycVDFuZUFFamkwcFk

 

Blog Update #4 – Experiment Design

Putting the script and interview questions on the top of the page for easier access:

Interview Script and Questions:
https://docs.google.com/document/d/1_foU-aBvDJ906hsayNaUU5svqDjUmoN7hVkyJBFb7wU/edit?usp=sharing

 

 

Blog Update #4a Revised goal(s) of experiment:

As mentioned in our proposed goals, we will be focusing on the first two goals, which are:

  • Goal 1: Does the interface help the user establish or feel a sense of trust with other users from the information provided in the profile page, private message, and mutual friends.
  • Goal 2: Are the users able to use the application to connect with suitable jam buddies for each other?

 

In order to narrow down our focus, and attain specific insight from our data, we revised our experiment goal to be more specific as follow:

  • Based on the information provided in the profile page interface, the user will establish a sense of trust in looking for a new jam member, and this would develop further into connecting with the new jam member through sending a message or adding as a friend.

 

To test if our application succeeded in reaching this goal, we will be comparing our application to an existing application, BandFriend. Both applications support the task that allows the users to express their trust and connection with new members, but we believe that our application, Band&Jam, would have faster user’s selection time on new members, and have higher user preference from our selected workflow.

 

Blog Update #4b Experiment method

 

Participants:

We will have five participants who are over the age of 18 years old. The participants will all be musicians or play an instrument, albeit at different skill levels to provide some variety amongst the participant pool. We will recruit our participants through personal connections such as friends and friends of friends.

 

Conditions:

The participants will complete the given task on the existing application, BandFriend, that is available on the market now, and our application, Band&Jam.

 

Tasks:

We have provided tasks for users to interact with our system. First, users will walk around with the app after going through the tutorial of the app, the system will mock the data and populate the app with other users which they have passed by. Second, users will be given a pre-populated list of user matches based on proximity encounters, attributes, and mutual friends. Third, users will then be asked to explore the interface and choose 5 users in which they want to jam with the most. Fourth, users will be presented with another existing app(BandFriend) and do steps one to three again. Lastly, a semi-structure interview will follow by the end of the tasks where users will be asked some questions about the two apps.

 

Design:

The formal experimental design used will be a 2 sample (2 conditions) experiment based on the comparison of one sample mean with a constant. 5 participants will partake in the 2 sample experiment and will be conducted using within-participant design. The sample mean we will measure using our new design relates to proximity and its influence on how comfortable/quickly a participant took to add friends. The constant measure will not include proximity but will be an application currently used in the real world. The application we will be using as a constant is known as BandFriend. The comparison of performance data and performance requirements between the new design and BandFriend will determine whether our new design, focused on proximity, meets key design requirements. Performance data pertaining to speed and user experience will be obtained using Likert scales and video recordings as measurement tools.

 

Procedure:

Procedures of the experiment can be sectioned into three major components as following:

 

We will begin by briefing the participants on the process of the experiment, including the purpose of walking around, the tasks they will be expected to follow, and the prototype they will be working with. We will not disclose the entire experiment to the participant at this point, but enough information that they will be able to follow the tasks. Then, we will present the consent form to the participant. After signing the consent form, we will show our prototype to the participant and proceed to make our way to the HCI lab following our predetermined route.

When we have made our way to the HCI lab, we will first let the user navigate the two applications on their own. The order of whether BandFriend or Band&Jam will be used first will be randomized to cancel out any learning effect. At the same time, we will be noting any usability problems on the application. Subsequently, we will ask the participant to choose five musicians on the each application based on the information that is present on the application.

By performing these tasks, we wish to measure the preference level between BandFriend and Band&Jam and the execution time of the choosing task in order to find out whether our application has a better approach to the problem.

Lastly, we will be conducting a brief semi-structured interview. The interview will focus on the reasoning behind the selection of jam partner, what information will help in the decision-making process, and whether there are any previous unvocalized usability issues present on the prototype.

 

Apparatus:

The Apparatus that will be required for this experiment is our medium fidelity prototype and BandFriend on a mobile device, a Camera for video recording and photo taking, and laptops or notebook for note taking purposes.

 

Independent and dependent variables:

We will be using two separate interfaces as our independent variables. One interface will be the prototype of our application, Band&Jam, and the other will be an app in the market currently in the market named BandFriend. Our dependent variables will relate to the user interactions with each app. Firstly, the users will be timed on how long it takes them to find five members to form a band in each app and will represent our first dependent variable. The second dependent variable will be a qualitative measurement of the user’s preference between apps using a Likert scale. To summarize:

 

  1. Independent variables:
    1. Application interfaces – BandFriend VS Band&Jam

 

  1. Dependent variables:
    1. User performance time to pick their jam members (quantitative measurements)
    2. User preferences between apps (qualitative measurements)

 

Hypotheses:

Task performance time

  • H0: User takes similar amount of time to select 5 jam members on both app – Band&Jam and BandFriend
  • H1: User takes lesser time to select 5 jam members for Band&Jam than BandFriend

 

User preference

  • H0: Users prefer both designs equally
  • H1: Users prefer Band&Jam  more than BandFriend

 

Planned statistical analyses:

For our statistical analysis, we will be using t-test with (p = 0.05) to determine the significance of our values. We have chosen this analysis as there is only a single factor comparison for our experiment with two conditions. We will also analyze on qualitative data, such as Likert scale answers.

 

Expected limitations of the planned experiment:

There are several limitations that we encountered when designing our experiment. First, the number of participants are limited to five due to time and resources constraint. We do not have the time to recruit too many participants to create a larger scale study environment.  Second, places or locations are limited to UBC main hall, we are not able to conduct the experiment at many different locations, again, because of time and resource constraint. We believe that by running the experiment at various locations is highly recommended in the future if given enough time. Lastly, the list of matches is also limited to only certain pre-populated data. We have tried to eliminate all possible factors that could potentially bias our experiment. However, given the nature of our platform, the list of matches has to be faked since it is impossible to have carried out the intended idea (matching based on proximity) with only Medium fidelity prototype along with other limitations.