Blog Update #7a – Final Conclusions and Recommendations

When comparing proximity versus geolocation methods, we had found that both methods were valid approaches when implemented for social gathering applications, more specifically for connecting musicians with each other. The geolocation approach was found to be more user-friendly by the transfer effects from other applications. On the other hand, we had theorized that proximity could create bonds between users. However, the result remained inconclusive due to the deviated direction of the experiment design.  To summarize, it was concluded one was not better than the other for populating users in a social application and was found to primarily depend on user preference when choosing between the two methods. This could have been due to our experiment design implementation, which could have handled the execution in a more controlled manner, like the implementation of two prototypes where one would have implemented geolocation as a control and the other proximity, instead of using a currently commercially available application as the control.

Overall, even though our findings were inconclusive, we found proximity methods to be a valid approach when used for populating a list of users, though adjustments are needed if future experiments are to determine whether proximity is a better approach when compared to geolocation. Current applications in the market have similar use cases for proximity and geolocation for populating a list of people in social applications, however, proximity based-methods were not found to be implemented in social music applications. Further investigation needs to be done to determine whether or not proximity is a better approach in establishing bonds with other users as opposed to geolocation. Conclusive results between the two approaches can help in the development of other future or current FriendFinder applications since the two approaches are not limited to only social music applications.

In an effort to investigate whether the proximity approach would help to establish bonds between users, the next logical step to take would be to run a more in-depth study involving more participants, while having users use the proximity-based application for a prolonged period of time. This will help users feel a sense of realism when developing connections in the application while providing a better approach for studying how a head counter (how many times a user passed by another user) helps users connect with other people.

Blog Update #7b – Reflection on Your Design and Evaluation Process

Throughout the course of the design process, our actual interface design and approach remained relatively constant. Many aspects of our design were inspired by commercial applications already in existence which validated the interface design and concept in advance. The results gathered from our initial field study supports this fact. Initially, we created two designs for the profile page in the application which were amalgamated into one following the initial field study, but only minor changes were made to other aspects of the interface. Certain features allowing the user to gather a better understanding of the musician ‘at a glance’, such as the music sample and skill level were brought to the forefront of the application based on feedback from the first field study. Conceptually however, the design choice of utilizing a proximity based system to connect musicians with each other remained constant throughout.

A major pain point heading into the development of the medium fidelity prototype that was not foreseen was the ability to dynamically generate the list of matches in the application that the study participant had presumably passed at some point. The introduction of the walk around the building in the second field study was then introduced to convince the participant they had just recently passed other users that were also in the application. Ultimately it was difficult to convince participants these encounters had just occurred and detracted from the reality of the experiment.

Every method applied in this study were effective in some capacity in helping us to validate our hypotheses. A semi-structured interview was carried out initially to help us explore the problem space. Observation and semi-structured interviews were later used to evaluate the usability of our design concept and prototypes. However, the methods chosen for our evaluations would have been more effective in a long-term study with participants using the application throughout the course of a week to two weeks. Also, tasks assigned to the participants to carry out could have been more open-ended and less structured to allow participants to choose the users they personally enjoyed.

Interviews, field study, pilot study, and observation were found to be the most useful methods used in the studies. Semi-structured interviews were conducted twice throughout the course of the project and allowed us to gather insightful information about our participants which we found to be of the most value. Observation and field studies were used once in the project for the purposes of evaluating our medium fidelity prototype. Through observation, we gained a much deeper understanding of the weaknesses and strengths of our interface that we had not seen before. Lastly, the pilot study was another useful method that we used for pre-determining the flaws of our interface before putting it into the real field study. On the other hand, the questionnaire from our initial field study was found to be the least helpful, mainly because the results did not really aid much in the studying of our participants. Overall, many methods we applied were crucial to the success of this study.

Blog Update #6a – Pilot Test

Pilot Study

Overall the pilot studies went as expected, there were no major changes made to the protocol. However, we did implement a few minor changes based on our discovery from the pilot studies. For example, we rephrased some of the questions from interview and questionnaire script to be more specific since we found that some questions were redundant and confusing to the participants. Second, we added a few things to our medium fidelity prototype, such as age, gender and more description in the profiles as a way to eliminate biases and differences that may influence the result of the experiment between BandFriend and Band&Jam that we were not accounting for in our measurements. The time recorded for the pilot studies were shorter than anticipated, but we expect the real field studies and experiments to be longer. Lastly, we did receive useful feedback from the pilot study participants regarding the instructions that were provided to them. There was initially some confusion on how the applications differed between Band Friend and Band&Jam as the proximity versus geolocation functionality was not explicitly and clearly conveyed. Changes were made to the briefing given to participants in order to better convey the differences between proximity and geolocation before starting the interaction with the applications.

Blog Update #6b – Experiment Abstract

Abstract

Through observing how musicians interact with each other, we conduct a field study to analyze how musicians connect with each other for the purposes of forming a band or jamming together and reveal the differences in how geolocation versus proximity-based systems assist in interacting amongst musicians. Fundamentally, we compare two applications, one that utilizes geo-location and other utilizing a proximity-based approach. We conclude by analyzing the differences in user’s performance time and preferences over the two location-based functionalities and its influences on their behaviors and comfortability towards other users in the application.

Blog Update #6c – Revised Supplementary Experiment Materials

Revised Supplementary Experiment Details

The questionnaire questions provided to the participant were changed from milestone III to reflect the different experiment procedure and goals. The interview questions asked were also changed to pertain to this experiment. Aside from the general beginner questions asked in the questionnaire related to the background information of the participant, no questions were carried over from the previous milestone.

Provided below is the link to the revised supplementary material (Questionnaire and Interview Questions), pertaining to the details above.

https://drive.google.com/file/d/0BwiLLoppv9ngbS15SjhNQjV0bms/view?usp=sharing

Blog Update #5a Rationale of prototyping approach

One interface prototype was chosen to be developed to be able to compare to an existing application in the market. Because of this one-to-one comparison, we chose to develop one prototype. To scope the implementation of the prototype, we chose to vertically implement only the functionality that will be utilized by the participants in the field study. Features such as creating a profile, messaging other users, or editing the setting of the application, while being very important for a full implementation of the app, were horizontally implemented as they were not features directly used by the participants. Functionality that was chosen to be vertically implemented was carefully evaluated by analyzing our field study tasks and procedure. From that, it was determined that functionality related to exploring other musicians profiles was needed to be vertically implemented.

To limit the scope, we chose to simulate the proximity feature of the application by pre-populating the list of matches in the application for the user. While this list would be dynamically generated in a full implementation when the users would cross paths, we simulate this behavior by still having the participant walk around to generate this list of matches. From the user’s perspective, this functionality looks identical to them within the app.

In terms of appearance, since we will have the user give a rating of preference using a Likert scale, we chose to create a prototype that appears “market ready” to avoid any influence that appearance may cause. By utilizing Axure to create the prototype, this allows us to create a visually appealing prototype while also taking advantage of our previous experience with Axure from CS 344.

Blog Update #5b Prototype Demonstrations

Link to video of prototype: https://drive.google.com/open?id=0B-DflbkztNycVDFuZUFFamkwcFk

Putting the script and interview questions on the top of the page for easier access:

Interview Script and Questions:
https://docs.google.com/document/d/1_foU-aBvDJ906hsayNaUU5svqDjUmoN7hVkyJBFb7wU/edit?usp=sharing

Blog Update #4a Revised goal(s) of experiment:

As mentioned in our proposed goals, we will be focusing on the first two goals, which are:

• Goal 1: Does the interface help the user establish or feel a sense of trust with other users from the information provided in the profile page, private message, and mutual friends.

• Goal 2: Are the users able to use the application to connect with suitable jam buddies for each other?

In order to narrow down our focus, and attain specific insight from our data, we revised our experiment goal to be more specific as follow:

• Based on the information provided in the profile page interface, the user will establish a sense of trust in looking for a new jam member, and this would develop further into connecting with the new jam member through sending a message or adding as a friend.

To test if our application succeeded in reaching this goal, we will be comparing our application to an existing application, BandFriend. Both applications support the task that allows the users to express their trust and connection with new members, but we believe that our application, Band&Jam, would have faster user’s selection time on new members, and have higher user preference from our selected workflow.

Blog Update #4b Experiment method

Participants:

We will have five participants who are over the age of 18 years old. The participants will all be musicians or play an instrument, albeit at different skill levels to provide some variety amongst the participant pool. We will recruit our participants through personal connections such as friends and friends of friends.

Conditions:

The participants will complete the given task on the existing application, BandFriend, that is available on the market now, and our application, Band&Jam.

Tasks:

We have provided tasks for users to interact with our system. First, users will walk around with the app after going through the tutorial of the app, the system will mock the data and populate the app with other users which they have passed by. Second, users will be given a pre-populated list of user matches based on proximity encounters, attributes, and mutual friends. Third, users will then be asked to explore the interface and choose 5 users in which they want to jam with the most. Fourth, users will be presented with another existing app(BandFriend) and do steps one to three again. Lastly, a semi-structure interview will follow by the end of the tasks where users will be asked some questions about the two apps.

Design:

The formal experimental design used will be a 2 sample (2 conditions) experiment based on the comparison of one sample mean with a constant. 5 participants will partake in the 2 sample experiment and will be conducted using within-participant design. The sample mean we will measure using our new design relates to proximity and its influence on how comfortable/quickly a participant took to add friends. The constant measure will not include proximity but will be an application currently used in the real world. The application we will be using as a constant is known as BandFriend. The comparison of performance data and performance requirements between the new design and BandFriend will determine whether our new design, focused on proximity, meets key design requirements. Performance data pertaining to speed and user experience will be obtained using Likert scales and video recordings as measurement tools.

Procedure:

Procedures of the experiment can be sectioned into three major components as following:

We will begin by briefing the participants on the process of the experiment, including the purpose of walking around, the tasks they will be expected to follow, and the prototype they will be working with. We will not disclose the entire experiment to the participant at this point, but enough information that they will be able to follow the tasks. Then, we will present the consent form to the participant. After signing the consent form, we will show our prototype to the participant and proceed to make our way to the HCI lab following our predetermined route.

When we have made our way to the HCI lab, we will first let the user navigate the two applications on their own. The order of whether BandFriend or Band&Jam will be used first will be randomized to cancel out any learning effect. At the same time, we will be noting any usability problems on the application. Subsequently, we will ask the participant to choose five musicians on the each application based on the information that is present on the application.

By performing these tasks, we wish to measure the preference level between BandFriend and Band&Jam and the execution time of the choosing task in order to find out whether our application has a better approach to the problem.

Lastly, we will be conducting a brief semi-structured interview. The interview will focus on the reasoning behind the selection of jam partner, what information will help in the decision-making process, and whether there are any previous unvocalized usability issues present on the prototype.

Apparatus:

The Apparatus that will be required for this experiment is our medium fidelity prototype and BandFriend on a mobile device, a Camera for video recording and photo taking, and laptops or notebook for note taking purposes.

Independent and dependent variables:

We will be using two separate interfaces as our independent variables. One interface will be the prototype of our application, Band&Jam, and the other will be an app in the market currently in the market named BandFriend. Our dependent variables will relate to the user interactions with each app. Firstly, the users will be timed on how long it takes them to find five members to form a band in each app and will represent our first dependent variable. The second dependent variable will be a qualitative measurement of the user’s preference between apps using a Likert scale. To summarize:

1. Independent variables:
   1. Application interfaces – BandFriend VS Band&Jam

1. Dependent variables:
   1. User performance time to pick their jam members (quantitative measurements)
   2. User preferences between apps (qualitative measurements)

Hypotheses:

Task performance time

• H0: User takes similar amount of time to select 5 jam members on both app – Band&Jam and BandFriend
• H1: User takes lesser time to select 5 jam members for Band&Jam than BandFriend

User preference

• H0: Users prefer both designs equally
• H1: Users prefer Band&Jam  more than BandFriend

Planned statistical analyses:

For our statistical analysis, we will be using t-test with (p = 0.05) to determine the significance of our values. We have chosen this analysis as there is only a single factor comparison for our experiment with two conditions. We will also analyze on qualitative data, such as Likert scale answers.

Expected limitations of the planned experiment:

There are several limitations that we encountered when designing our experiment. First, the number of participants are limited to five due to time and resources constraint. We do not have the time to recruit too many participants to create a larger scale study environment.  Second, places or locations are limited to UBC main hall, we are not able to conduct the experiment at many different locations, again, because of time and resource constraint. We believe that by running the experiment at various locations is highly recommended in the future if given enough time. Lastly, the list of matches is also limited to only certain pre-populated data. We have tried to eliminate all possible factors that could potentially bias our experiment. However, given the nature of our platform, the list of matches has to be faked since it is impossible to have carried out the intended idea (matching based on proximity) with only Medium fidelity prototype along with other limitations.

a. Further updated task examples:

SUMMARY OF REVISIONS:

We have revised the task examples to be more fluid with each user. Adding more personality into the task examples pertaining to each individual task example user. With this revision, we hope to tackle the “list” format that was addressed during the MSII review.

TASK EXAMPLE 1:

Jeffrey is a 2nd year UBC student who loves to play music as a hobby, especially the guitar. He would like to play music for fun with new people and make new friends who also play music in the upcoming school year but is not actively looking. He didn’t make many new friends in his first year because he lacked time to join clubs and time to meet other new students. Mainly because he wants to meet other students on his own leisure and be notified when people around him are into music playing, like playing in a small band or having small jam sessions. He walks to UBC every weekday and knows that some of the students he passes and sees in his classes must play music. Jeffrey knows there are multiple ways of meeting new people in a community like UBC but like 1st-year, he lacks time to commit himself to clubs and smaller communities within UBC. So, he has tried talked to his friends from highschool indicating his willingness to play in jam sessions without much preparation beforehand. However, he has not received any interest from his friends because his friends, who live in different parts of the city, are either too busy or not willing to go out of their way to play at UBC. Though he has tried contacting his high school friends, being a UBC student he would rather prefer meeting other musically talented UBC students, people he would find trustful and similar to himself, in regards to personality and music skill level.

TASK EXAMPLE 2:

Alexander, a jazz bassist in his twenties who is more on the introverted side, is looking to find other jazz musicians to play with because he finds it a great way to improve his own skills. He has asked around in his friend circle and unfortunately the only friend he knows that is a musician plays the clarinet. It’s not that his friends don’t play music, it’s more so that they don’t particularly enjoy playing jazz music. In his longing to play with other jazz players, Alexander browsed classified sites such as Craigslist to connect with other like musicians but hasn’t been able to find posts about jazz players in particular. He has also thought about posting an ad on Craigslist, but he doesn’t feel comfortable posting information about himself for the whole world to see, especially since random strangers that he has no connections to will have access to his ad. Oddly enough, Alex has posted self-info about himself on Tinder but has only met up with people he had a handful of mutual friends with that helped establish a sense of personal connection. Although he is shy and doesn’t reach out to strangers in public very often, once he feels comfortable with someone he can be quite social.

TASK EXAMPLE 3:

Sharon is a 4th-year SFU student majoring in chemistry who is quite musically talented. She plays in small jam sessions casually with her roommates and has previously played in formal bands that performed in gigs and coffee shops. She has long since wanted to form another band but for the long-term, hoping to perform at local coffee shops and small gigs. Sharon can play a variety of instruments like electric guitar, electric bass, and drums, but she is most comfortable playing the piano as she has been playing piano for 10 years. Currently, she plans on playing with her 2 roommates who are excellent guitar players in various genres, and is specifically looking for a drummer for her new band but is open to other players that play different instruments from themselves. However, she’s not particularly comfortable meeting new people online because of all the scary stories she’s heard, so she is looking for someone who is from her school with previous band experience. Sharon had tried other alternative methods of recruiting new band members such as asking friends and friends of friends but has yet to have any luck finding a suitable drummer fitting to her particular tastes.

b. Low-fidelity prototype(s) demonstration:

Demonstration of the prototype can be seen https://www.youtube.com/watch?v=X8i_GbZPj40

c. Additional information about the prototype:

Our low fidelity paper prototype mainly supports the first and second task example (See task examples in section a). More specifically, Jeffrey, from task example 1, and Alexander, from task example 2, both want to find people to play with but were unsuccessful with the current platforms. They tried using Craigslist and joining social clubs to find other musicians. However, the use of Craigslist lack a sense of trust and real life situations tend to be more stressful.

Through our low-fidelity prototype, we intend to investigate if our proximity design is feasible and which of our profile layout better suits the purpose of our application. We suspect that users might have difficulties understanding the proximity aspect-passerby counter, map, and catalog without further explanation provided. We hope the result of our walkthrough will help us come up with an intuitive design to the problem. We have two profile interfaces to test if a transfer effect when viewing the profile page, or if a new way of representing the information is more suited. For our first profile interface, we focus on transfer effect by incorporating a similar architecture to facebook. It allows users to interpret information on profile page without unnecessary heading and titles. Our second profile interface focuses on prioritizing important information to our application. Based on our findings from our field study, we found a music sample to be an important element when determining user compatibility and as such, emphasized this in the second profile interface. Therefore, our second profile interface features a music sample at the top of the paper followed by user information with headings.

d. Walkthrough report:

After running through cognitive walkthroughs of our task examples 1 and 2, we found that the flow of our application was quite intuitive. Our participants were able to easily move through the pages of the app with relative ease and understanding. For example, our profile page and messaging system are components that are found in other popular social media apps and allow users to utilize transfer effects. The music samples on the user’s profile was a popular and useful widget, pointed out by multiple participants as a helpful way to determine if they would be a suitable jam buddy.

Conversely, we also learned the negatives of our application. Our opening screen in the app, a catalog of their current ‘matches’, confused participants and did not convey what the purpose of the page was. We also found that our profile page was missing the availability of the other user to explain when they were available to jam, and also the confusing use of a pop-up map with pins showing where the two users had passed each other. While the messaging system was easy to use, it was pointed out that it was missing a way to distinguish new messages from ones that had already been read, and lacked a way to initiate a new message with a user that had not been messaged yet. It was brought up that a splash screen could be used to explain many of these features and avoid the confusion that was apparent from the lack of an explanation.

Overall, we learned that participants found our prototype easy to use and found that our task examples were well supported by our interface. The participants were able to complete the task with no to minimal errors. However, they did encounter confusion when using our interface: First, both participants were confused about the term “catalog” which was a page for a list of people nearby. Second, was the map and the counters on the profile page. The counter was a button that takes the user to a map where it shows the location of which they passed each other. In the task example, Jeffrey was looking to form a band within a time period, participant reported that there should be details on the profile page about when they are available to jam to support this aspect of need. Participant two also suggests to have sample music on each list (messaging list, friends list, catalog), with the addition of having a list of sample music with titles for each file, if applicable.

e. Proposed goal(s) of experiment:

• Goal 1: Does the interface help the user establish or feel a sense of trust with the user by the information provided in the profile page, through the private message, and mutual friends.

• Goal 2: Are the users able to use the application to connect with suitable jam buddies for each other?
• Goal 3: Do users keep in touch with each other after using the application?

Although we have identified three goals that pertain to our interface, goals one and two will be pursued while goal three will not. Trust has become a key theme in this interface and because of this, we have identified goal one as the most important aspect to study as it relates to the ability of the interface to instill a level of trust in the user. Goal two is somewhat of an extension of goal one; finding a suitable jam buddy does include trusting the other user, but also includes personal attributes of the matched user such as the instrument they play and their preferred genre of music. Both of these goals are also testable with the same study. Goal three, while being important, will not be pursued as it requires user usage of the app to be tracked over a large period of time which is out of scope for this course.