Experiment Goals
From Blog #3:
- To find out what aspects about a buddy would make people feel safer and more willing to choose them over others
- To find out how people perceive safety when walking in a group or by themselves, and how this perception manifests physically (eg. in the form of shortened walking time when walking alone)
- To consider limitations of the Safewalk service and the feasibility of extending its functionality
Revised Goals:
- To find out what aspects about a buddy would make users pick them over other choices
- To find out the differences in how people perceive their level of safety when walking with a SafeBuddy or by themselves
- To find out whether the users’ expectations based on the SafeBuddy profile aligned with their actual experience during the trip
The major focus of our previous experiment goals from the Blog Update #3 was not on the SafeBuddy application, but rather a more general experience of commuting and using SafeWalk . We revised the experiment goals to cover the cases that are directly related to the users and the application of our design. In order to do so, we have revised our goals to narrow the scope by solely focusing on the experiences of the SafeBuddy. Specifically, our goal is to evaluate the effectiveness of our design in increasing the perception of safety while commuting.
Experiment method:
Participants
We plan to recruit a total of 5 new participants for this experiment, primarily through convenience sampling via direct face-to-face or online conversations. These participants will all be current students at UBC who are familiar with smartphones. In addition to these participants, each team member who does not personally know the participant in the study will also be a part for the buddy matching, and also for the field experiment if the member is chosen during the buddy matching. This design was chosen as it allows a greater selection and variety of buddies for the participants to choose from.
Conditions
We plan to compare two main conditions in this experiment within subject: how the measures of feeling safe differs when one walks with someone else, versus when they are walking alone. By doing so, we aim to discover the extent of the impact of providing users who currently travel alone with a SafeBuddy. In order to do this, we will ask participants to go on two separate walks that are similar in lighting and distance. However, one walk will be of them walking by themselves, while the other will be walking with a buddy that they chose during the buddy matching phase.
Tasks
The following are the different tasks that participants are expected to complete in our experiment:
- Given a selection of different profiles of SafeBuddies, rank the top three.
- Participants will be given profile cards with a picture, name, number of mutual friends, and a short two sentence self description of each buddy (that we will have participants provide). See Appendix 1.2. for example of a profile.
- Given a set path, walk alone from a designated starting point to the destination
- Participants will be told to keep the SafeBuddy application on them, with a map implemented to show and track their location and route in real time.
- Give a rating of perception of safety from 1-5
- Participants will be given a simple paper survey to rate accordingly. See Appendix 1.1.
- Given another set path, walk with your chosen safety buddy from a designated starting point to destination.
- This path will be different that the one in Task 2, but will be similar in other factors as discussed below.
- Give a rating of perception of safety from 1-5
- Again, see Appendix 1.1.
- Final interview with the experimenters
Note that both set paths provided will be very similar in lighting, distance, and number of people walking around. We plan to organize our walks around 6-7pm, by which the sun will have set and thus natural lighting will be relatively dimmer. Both routes will be set on the UBC campus and will have fewer people walking on it in an attempt to simulate a more authentic “late night” feeling. Also, the order of tasks 2 and 4 will vary for each participant to control for any participant bias for walking first alone and then with a buddy, or vice versa.
Design
The experimental design will be 2-condition, within subjects to identify how the subjects’ perception of safety change based on the use of the SafeBuddy application. The independent variable we will be modifying is the presence of a SafeBuddy. The results will not be significant when accomplished using a between-subject experimental design since we want to discover how individuals’ feelings change when walking alone versus walking with a SafeBuddy, not how the perception of safety might vary between different types of users. Additionally, individual differences such as individual preference and prior experience walking through the route in our subjects might change their perception of safety between the two walks. A within-subject experiment would allow us to better identify and analyze these cases.
Procedure
The overall procedure is designed to be completed within two 30 minute sessions for each participant, for a total maximum of one hour per participant. We will run the experiment over two different days for each participant. On the first day, the participants will be asked to rate their top 3 SafeBuddies based on the SafeBuddy profiles provided. Once all the participants complete choosing their top 3 SafeBuddies, the experimenters will pair users that included each other as the top 3. In the second session, these users will complete the remainder of the experiment by walking a route alone, walking with their paired SafeBuddy, and the post-experiment interview. We will provide a short questionnaire containing the Likert scale questions to the participant after each walk for them to evaluate their perception of safety. After both the walks are completed, we will interview the participant for further information to answer our experiment goals.
Apparatus
Matching Part
The participants will be placed in a room with the experimenters and the interface prototype, along with appropriate note-taking materials (Appendix 1.4) which will be used to record the participant’s answers regarding their process of evaluating the best prospective SafeBuddies.
Walking Part
The participants will walk two different predetermined routes on the UBC Campus around 7pm. There will be a experimenter at the destination point and an experimenter at the source point of the routes. The experimenter at the destination will have the questionnaire on a sheet of paper and a writing utensil. These routes will take around 10 minutes each.
Independent and dependent variables
- Dependent Variables
- Perception of safety based on 5 pt scale
- Satisfaction with buddy selection after trip (likert scale i.e. disagree -> agree)
- Time of travel (given by phone)
- Choice of SafeBuddy
- Independent Variables
- Walking with a buddy versus walking alone
- Route
- Time of day
- Pool of SafeBuddies that can be chosen
Hypotheses
H0: SafeBuddy application decreases or has no effect in the perception of safety
H1: SafeBuddy application increases the perception of safety when walking with a SafeBuddy
H2: SafeBuddy application increases the perception of safety when walking alone
H3: SafeBuddy application increases the perception of safety when walking with a SafeBuddy and when walking alone
Planned statistical analyses
T-test for:
- Perception of safety
- Travel time
Mean/Median/IQR for:
- Satisfaction rating of SafeBuddy choice
Expected limitations of the planned experiment
The major concern for this experiment is that the experiment must be safe for the participants but also needs to recreate any natural unease and fear that the participant would normally feel when commuting. Since the experiment is designed to be safe by having multiple experimenters watch over the participants and takes place when it is relatively early and there may still be a degree of sunlight, the participants could feel disproportionately safe during their walk. To control the experiment, the same routes will be chosen for all participants. Although this attempts to limit undesired variations created when some participants walk in a route that is “scarier” than other participants, it will not eliminate variations caused by the individual differences between participants. Some participants may be more familiar with the route than other participants, altering their perception of safety. Another individual difference that cannot be accounted for is the participants’ personalities. The two participants’ comfort levels with each other could negatively affect their experience of walking with a SafeBuddy even if the participant normally enjoys walking with a companion more than walking alone.
Supplemental experiment materials:
- Interview Questions (1.3)
- Observation Sheet for Matching portion (1.4)
- Questionnaire for rating (Can be part of interview) (see appendix 1.1)
- User Profiles of each participant and group member
Appendices:
1.1.
Perception of safety survey given to participants at the end of their routes.
https://drive.google.com/open?id=1xF_1JSKeSRQ_4OQsjYS9ZqJmoL3R903ZZZktZWJpczY
1.2. Example of a buddy profile template
https://drive.google.com/open?id=0B0o8btIcsGf6dFhCaWhxdVlPaDQ
1.3: Interview Questions
- What affected your choice of the SafeBuddy?
- Do you think the SafeBuddy profile gave you a realistic expectation of your SafeBuddy? Why or why not?
- What information on the SafeBuddy profile was not helpful in choosing a SafeBuddy?
- What information was missing that would have aided you further in choosing a SafeBuddy?
- Do you think walking with a SafeBuddy helped you feel safer? Why or why not?
- Do you think the SafeBuddy application, or something similar is something you would use in the future regularly?
- How did you interact with the SafeBuddy application during the walk?
- Did you feel safe walking alone? Why or why not?
1.4. Observation Sheet for Buddy Matching
| Profile | Number of Times Looked At | Patterns in viewing Buddy profiles | Notes |
| 1 | |||
| 2 | |||
| 3 | |||
| 4 | |||
| 5 | |||
| 6 | |||
| 7 | |||
| 8 | |||
| 9 | |||
| 10 | |||
| Total time taken to choose a buddy | |||
| Final Rank of Buddies
(1, 2, 3) |
|||
1.5. SafeBuddy satisfaction survey given to participants at the end of their routes.
https://drive.google.com/open?id=0B0o8btIcsGf6V3hYcGc0MlBFS0U