Blog Update #7a: Final conclusions and recommendations

Conclusions:

Overall, although we did not manage to see statistically significant results, both qualitative and quantitative data from our study suggests that community-related elements are important motivators for participants when deciding to donate to a charity. Donation events with segmented progress bars and community-related elements displayed a higher mean level of engagement and trust. Many participants stated that seeing their friends who have also donated was an important requirement for their own donation, and also played an impact on the amount they decided to donate. This hints that our design was able to meet its primary goal of creating an engaging donation experience to motivate donor behaviour.

However, due to time and experimental limitations, we were only able to evaluate a subset of the functionalities in our design; we were not able to assess the impact of establishing clear goals and subsequent feedback in encouraging increased user engagement and donations. We can only conclude that the trends from our data showed that the segmented progress bar and community-related elements may be useful to motivate donation behaviour. Interestingly, we also found that both elements may cause a feeling of competition or social comparison to arise, and that the colors of the segmented progress bar are distracting. These areas can potentially be explored further.

Recommendations:

We believe that our logical next step would be to perform several much-needed minor adjustments, but that our overall design approach was also validated. Adjustments to the progress bar and improvements of aspects such as anonymity are needed, but the community-related elements of our interface to improve donation motivation was validated through our prototype.

However, there are mixed findings on the emphasis on individual contribution as well as social comparison. Regarding these findings, future studies should investigate the effectiveness of segmented progress bar with more aesthetically pleasing colors, such as single colored segments, or low saturated colors, to reduce visual fatigue. Another issue that is worth studying is the scalability of segmented progress bar. Larger scale donation events would likely have unrecognizable individual contributions and overly crowded segments, which might lead to confusing donation experience. Moreover, a major concern for many users has to do with the social comparison arising from the segmented progress bar and community elements. Some users liked it, whereas others reacted negatively to it. More investigation should be done on ways to mitigate the negative aspects of competition, while continuing to cultivate increased intrinsic motivation to donate.

Blog Update #7b: Reflection on your design and evaluation process

Beginning our project with a field study was a great way to start. We were able to get insight about the current practices used by charities to motivate donors, and the causes that drive a potential donor to contribute, which then directed our design. From the field study, we learned that participants trust a charity more when family and friends have also donated, and the sense of community is also a motivating factor as the impact made by a community will be much bigger. By conducting interviews with users, we were able to uncover these motivators, which would not have been reflected in our design without the field study.

Starting our design with low-fidelity prototyping before moving to medium-fidelity prototyping worked well. We were able to find confusion points early on that could have been potential confounds in the evaluation of our interface designs later. However, when moving on to our medium-fidelity prototype, we made certain visual and interface decisions that we found ended up surprisingly influencing our evaluation (e.g. the colorful segmented progress bar). In our paper prototype, the segmented progress bar had less saturated and a more toned down color palette, meaning that any issues were missed from piloting. However, when we moved on to the medium-fidelity prototype, we changed the colors to become more saturated and more colorful – leading some users to feel distracted and confused by the array of colors presented. From this experience, we realized the importance of such seemingly small decisions – even on the visual aspects of our interface. More careful and systematic piloting would have helped to avoid these issues again in the future.

Regarding our evaluation of participants’ motivation to donate, we thought that the approach we had taken was, in general, the right direction, but certain details could have been changed to improve the validity of the study. Creating a scenario for participants where they were given money to allocate to different donation events worked well, as participants were able to definitively decide which interfaces they liked better, and donate more to those. Although the task was limited in that the given budget was not real money, and that people were forced to donate some amount of money, we still found that this allowed us to better focus on just the effect of the interface factors we were interested in. We could have, however, been more careful with the content of the donation events we used – although there was no statistically significant effect from the content, we found that people were more motivated to donate towards natural disasters such as earthquakes (something we also found in our field study). As a result, we should have, in hindsight, briefly piloted the content of each donation event as well, and remove events where there might be a potential confound.

Blog Update #5a: Rationale of Medium Fidelity Prototype Approach

Our prototype will be a website, built using HTML/CSS. Our experiment only requires one prototype that can be toggled to simulate all 4 conditions.

Horizontally, we decided to cover the following features:

• Users already have their profiles created and linked to social media accounts
• Filter function
• Search function

We decided to Wizard-of-Oz’d users’ profiles to avoid the tedious creation process, and to Wizard-of-Oz’d the filter and search function because our experimental goals don’t emphasize on that. Since it is impossible for us to link participants’ social media accounts to our website, friends of participants will be Wizard-of-Oz’d.

Vertically, we decided to focus on two areas – the community-related elements and a segmented progress bar that highlights individual contributions:

• A list of donation events that users can view by clicking the “next” button
• A segmented progress bar in which a segment represents the donation by a donor
• When user hover over a segment of the progress bar, a pop-up shows up, with a thank you note to the donor
• Friends of user will be shown under the progress bar and when the user hover over his friend’s profile, a pop-up shows up, with a thank you note to his friend
• A draggable bar that user can adjust the amount of money to donate
• An input box for user to input the donation amount
• When user click the “Donate Now” button, he is directed to the payment page
• After the donation process is completed, a pop-up shows a thank you letter to the user
• The progress bar on the detailed event page is grown to show the contribution of the user after the donation process is completed

We decided to implement the whole donation process because we want the user to experience the donation process with community-related elements and segmented progress bar to test whether these elements will motivate the user to donate more money.

Event details and donors’ contributions will be fake data. The payment process will be Wizard-of-Oz’d and payment information will be filled out in advance to avoid the tedious form-filling process. We included these elements because we want to make the donation process as realistic as possible.

The overall appearance is not a first priority in our prototype as long as the layout is clear and doesn’t distract participants. However, the appearance of the progress bar is argued to be important. Key to the progress bar is the clear and intuitive visualization that informs the impact and recognition to each individual contribution.

We decided to build the website using HTML/CSS because of our group skills and since our prototype requires many interactions on the interface, it is faster to build the prototype by using HTML/CSS.

Blog Update #5b: Prototype Demonstrations

Video Demonstration: 

Blog Update #4a: Revised Goal(s) of Experiment

To narrow down the scope of our experiment, we decided to only focus on goals 1 and 3 from Blog Update #3. We felt like these goals were ones that were not only simple to execute, but also had the most potential to produce interesting results. Aside from that, only minor wording changes were made for clarity and brevity.

1. To test the effects of community-related elements (e.g. Facebook friends who have also donated) on promoting donation behavior.
2. To test the effects of a segmented progress bar, or one that highlights individual user contributions, on promoting donation behavior.

Blog Update #4b: Experiment Method

Participants

10 participants will be recruited through convenience sampling. As a result, it is anticipated that our participants will be university students ranging from the ages of 19 to 25. This sample of participants is not completely representative of the entire target population (adults of varying socioeconomic statuses), given that most students are of the same socioeconomic status.

Conditions

In this study, we will investigate four different interfaces. Each of the interfaces have the same underlying structure, but with various features of interest toggled on or off to investigate their effect on donation behavior.

Regular Progress Bar + No Community Elements

This interface represents a standard donation interface which uses a regular progress bar as a measure of progress towards reaching some target monetary goal.

Regular Progress Bar + Community Elements

This interface retains the same standard progress bar as the first interface, but adds on community-related elements. Under the progress bar, there is a list of Facebook friends who have also donated to the same cause. More information such as brief description of friends and amount of money contributed appears when mouse is hovering over the profile pictures.

Segmented Progress Bar + No Community Elements

Instead of a regular progress bar, this interface features a “segmented” progress bar, where each segment in the progress bar represents an amount of money donated by a donor. The length of the segment is proportional to the progress made in terms of monetary amount towards the target goal. Users can hover over each segment to see the name of the donor and the amount donated. However, there are no community elements present in this interface.

Segmented Progress Bar + Community Elements

This interface contains a segmented progress bar, as well as the community-related elements as described above. When participants hover over the icon of their friends below the progress bars, it reveals a popup box indicating which segment of the progress bar their friend contributed to, and the amount of money they donated.

Tasks

Participants will be asked to evaluate and interact with four different donation interfaces (corresponding to the conditions above). They will be given a budget of $200 to specifically give to various causes or charities (see Section 4c for detailed task descriptions). After going through the four donation interfaces, they will be asked to allocate this money to each of the donation events, however they like. They can give as little or as much as they want (within $200) to each of the donation events. Each participant goes through the donation events in the same order, but the pairing of interface conditions to each donation event is counterbalanced with Latin squares to mitigate order effects and the potential confound of a particular donation event’s content.

Design

2 x 2 within-subjects factorial design, with 2 levels of community elements (inclusion of Facebook friends or no inclusion) x 2 types of progress bars (segmented or regular).

Procedure

Here we detail the basic template of our experiment procedure:

1. Participants will be introduced to our study and asked to read and sign the consent form.
2. Experimenter will set up experiment equipment with permission of participants, including: audio recording device, screen recording, and noting participant id.
3. Experimenter will walk participants through a donation task description, which gives participants a situation to donate using a allocated amount of money. (More details in attached instruments)
4. Participants will be presented with the experiment interface, which consists of 4 donation events each with different interface setups based on experiment conditions: each of the interfaces has the same underlying structure, but with various features of interest toggled on or off to investigate their effect on donation behavior.
5. After finishing exploring each interface, participant will be asked to donate some portion of money in the budget and answer a short interview question about their impression of the interface.
6. After looking through all interfaces, participants can decide if they want to change the amount of money that they allocated for donation to each of the 4 charities.
7. A follow-up questionnaire will be provided to collect information about participants’ demographic information, preferences, engagement and trust levels on each donation events.
8. A short interview will be given to collect more qualitative data based on feedback of participants’ questionnaire.

Apparatus

For convenience, the location of our experiment will take place in study rooms in the Computer Science building. The participant will be presented with a laptop computer that contains our interface prototype, as well as a piece of paper detailing the task description for easy reference. The laptop screen will be recorded to capture users’ interaction with the prototype. No location or hardware constraints were imposed for this experiment, as the key interaction will occur in the computer’s web browser.

Independent and Dependent Variables

Independent Variables:

All independent variables are within-subjects factors.

• Community-related elements (inclusion of Facebook friends or no inclusion)
• Progress bar type (segmented or regular)

Dependent Variables:

• Amount of money allocated to a donation event (quantitative)
  • Decided by participant after evaluating and interacting with each donation event’s interface.
• Motivation to donate
  • Measured by user rankings of donation events in terms of preference (qualitative).
  • Measured through 5-point Likert scales about engagement and trust (quantitative).

Hypotheses

H1: Participants will donate more money to interfaces with segmented progress bars compared to regular ones.
H2: Participants will feel more motivated to donate to interfaces with segmented progress bars compared to regular ones.
H3: Participants will donate more money to interfaces that include community-related elements compared to those without them, except for when there is no segmented progress bar.
H4: Participants will feel more motivated to donate to interfaces that include community-related elements compared to those without them, except for when there is no segmented progress bar.

Planned Statistical Analyses

We plan to conduct an Analysis of Variance (ANOVA) to test the difference in means of donation amounts across our interface conditions. For our Likert scales, we plan to simply present descriptive statistics, as they are meant to be supplementary to results from the ANOVA and the rest of our qualitative data. Presentation order of interfaces, as well as donation event content will be considered as potential confounding factors.

Expected Limitations

Due to the limited time and resources available to complete the project, our proposed study has several expected limitations. First, the experimental task lacks external validity, as it is somewhat unrealistic, given that the budget assigned to users is simply a number, and not their own real money. In addition, because we intend to prompt users to donate some amount to each of these donation events, motivation to donate is, at a base level, forced in the experiment. This was a decision made to help us more directly investigate the effects of our interface factors.

Second, a potential limitation is in exploring donor motivation, as it is difficult to measure objectively. Given the scope of the experiment, we rely on the amount users decide to donate and their self-report on various Likert scale measures to do so. However, we suggest looking at literature on donor motivation to find more established methods of evaluating motivation to donate.

Third, another limitation could be the potential confound of the content associated with a particular donation event. The content or cause associated with a donation event could be a major factor in affecting how much money a user might want to donate. Although we have made efforts to try to mitigate this, it still remains as a limitation of our study. We felt that using the same content across interfaces sacrifices too much external validity for our experiment.

Blog Update #4c: Supplemental Experiment Materials

Consent Forms

Study Instruments

Blog Update #2a: Next Steps

The next steps of the development will be generating low-fi/medium-fi prototypes and determine a proper experimental design to explore our findings further. Given the limited time and resources, we will focus on some of the key findings from our study. According to our interviews, all participants cited that internal factors such as empathy are the most important factors of motivating them to donate. External factors are relatively trivial and some external factors such as people who are pushy or intrusive may even hinder them from donating. Most participants tend to donate to causes that really touch them such as the difficulties of people in need. Therefore, we will be focusing on spurring people to donate by appealing to arousing their emotions. Additionally, there is a tendency for people to donate for causes they care about or mostly related to their personal experience. Our system will also focus on matching people’s interests with opportunities for donations.

We found that the lack of trustworthiness is a barrier that hinder people from donating. And most participant evaluate the trustworthiness of an charitable organization according to their familiarity and transparency of information. Additionally, one thing in current practices our users complained the most is that organizations do not always feedback, and we found providing feedback would promote long-term donations. Therefore, Our system will aim to motivate donations by improving the exposure of the donation information and providing feedback.

Moving forward we plan to explore different approaches/design alternatives to help us to address findings from our field study. We will explore different ways of arousing emotions of users and different formats to provide information and feedback to users. Hopefully we will help to address motivating long-term donations.

Blog Update #2b: Revised Task Examples

Summary of Changes:
Raymond:
– We changed the background of Raymond to be a more realistic person that is willing to do donations since we didn’t have a chance to interview any rich entrepreneurs that are also interested in donations.
– We pointed out the importance of feedback and the progress from charity after donors’ donations which will increase donor’s motivation.
Bella:
– We changed to point out the reason of Bella’s personality of caring others.
– Focused on the trustworthiness between individual and charitable organization.
Bob:
– Since we want to focus on increasing the motivation of donation behavior, we have changed this task example from a charity to a donor.
– We pointed out the fact that some donors are only motivated to donate to people who are related to them, and they need charity to remind them of the donation events that they are interested to donate.

Task Example 1: Raymond
Raymond is a university graduate, he is 27 years old and has worked as an Engineer for 3 years. Raymond cares about people and he intends to donate some portion of his income to help those who are in need. However, he is concerned about where his money goes to and the impact of his donation. He is a frequent donor and is especially motivated when he sees the stories of the donees and how hard their situation are. Most of Raymond’s donations go towards organizations he just so happened to stumble upon – for example, from street posters, or from social media content on the Internet. However, some of these organizations are really busy, and often don’t have enough feedback on the progress afterward or personal touch to their donors, so Raymond feels a bit more isolated and less connected that reduces his motivation to donate. He wants to establish a long-term donor-to-donee relationship with these organizations and wishes that there was some way that he could keep up to date with how he can help, and how his contributions have made a difference to those in need.

Task Example 2: Bella
Bella is a 20-year-old college student. She has an awareness about “helping others is a virtue” which is influenced by her family. This let her becomes a very charitable person that has much concern about people who need help. In the past, if she saw any organization was doing fundraising on the street and those charity workers came to her for donation, she was always willing to donate money to them. However, many news showed that some organization doing fundraising was just a fraudulent trick. She felt conflicted as the way she contributed was surprisingly not her original intention, and these destroyed her trustworthiness on those organizations. This changed her way to do donation. She only wants to donate to the charitable organizations recommended by her friends and family.

Task Example 3: Bob
Bob is a 35-year-old accountant and has moved to another country due to his job assignment. He is a person who only cares about the things or people who are related to him. Everytime he sees a charitable organization doing fundraising and asks him if he can donate some money for people in need, it does not motivate him to donate since he does not know who the donees are and he feels that it is not part of his business. In fact, sometimes he feels annoyed at those charity workers, as they are sometimes pushy and get in his way. Last month, he saw a news report about a natural disaster that happened in his hometown and there was an organization doing fundraising for helping those people in need. This disaster is related to his hometown, so it caused him to have a motivation to do the donation. He decided to do the donation after he get home from work. However, he totally forgot to do the donation when he got home, and

Blog Update #2c: Prioritized list of requirements

a) Absolutely must include
These requirements represent the major factors that can motivate donors to donate and are the most common concerns of donors after they make a donation. In addition, these requirements require more user testing and refinement than other requests.
– System includes ways that can appeal to the emotion of donors during the whole donation process.
– System includes a way to categorize different kinds of donation based on people’s interests.
– System includes a way for donors to know where their money goes to and the result of their donation.
– System includes an easy to use and clear flow when going through the process of donating.

b) Should include
This requirement is also very important to donors according to our field study but are slightly less important because they require a longer time for user testing to observe the effect.
– System includes a way for donors to establish a long-term donor-to-donee relationship with charity.
– System includes an easy way to remind the donation event that the donors are interested in.

c) Could include
This requirement is less important because donors tend to donate to charities that has established for a long time and/or are recommended by their family or friends, which does not require much user testing.
– System includes a way to show their trustworthiness to donors.

d) Could exclude
This requirement is a great addition to our system but is currently out of scope.
– Algorithms that can suggest categories of donation that are potentially interested by the donor.

Users: Must include
– Individuals who are (at least, somewhat) interested in donating to charities
– Individuals with enough knowledge to navigate a web interface
– Individuals who are able to pay by the payment methods provided

Users: Exclude
– Technologically illiterate people
– Illiterate people
– Individuals who are not interested at all in donations

Blog Update #2d: Design Alternatives

Design Alternative I: Donation Events

Description:
We envision a system that revolves around different types of “donation events”. These donation events are essentially fundraising campaigns organized by various charities with the goal to raise a certain amount of money for a specific cause. On the main interface of a donation event, there’s a large video which is used to introduce the donation event and arouse user emotion. We might also envision having some kind of progress bar to indicate the amount of money donated thus far.

Users have the option to create an account and set up a personal profile. Their profile stores information about categories of donation events that they are interested in, so that these can then be recommended to a specific user, personalizing the process.

Pros：
– Video is likely to be a stronger form of communication to arouse people’s sympathy
– Donation events give users a goal to reach and spur excitement around a cause.

Cons:
– Usefulness of system is dependent on number and diversity of events.
– Requires charities to host events and to include videos and relevant information.

Design Alternative II: Donation Gallery

Description:
In the main page of our system, we will show posters about the stories of donees who are in need, similar to that of a poster/photo gallery. Users can swipe to see more posters and click the “Details” button which direct them to the videos of donees if they are interested to know more about it. At the bottom of the posters, we will also show videos of donees to attract the attention of users.

If the user does not have a user account, he can create one and choose the categories of donation that he is interested. The latest posters and videos shown on the main page will be personalized by the categories chosen by the donees. When users click the video, he will be redirected to another page that has a huge “Donate Now” button to encourage one to donate. It makes it easy for the user to know the next step to do if he want to donate. After a user has donated, he can check his message box to know about the update of his donation. News feed with photos and/or videos will be shown to tell the donor about the result and impact of his donation.

Pros:
– Catch user’s attention by using pictures and videos of donees on the main page
– Use of videos to appeal to user’s emotion
– Provide feedback to donors about the result and impact of their donation

Cons:
– Users can only choose a category to donate according to the posters/videos posted
– Require charity to film video only
– Nowhere to see detail information about the charity.

Changes to Project Direction:

The original project direction focuses on increasing donor engagement with donation centres in need; we broaden the focus of this project from engagement to the motivation of donor behaviour, hoping to build a more equal community through interaction design. Although engagement is certainly a part of donor motivation, we thought that there was more aspects to motivation than simply engagement that we wanted to explore in our project.

Design Direction:

We will keep the overarching design direction from the original proposal: to create a personalized and engaging donation experience to motivate donor behaviour. We will focus our design efforts specifically on these two parts of the interface:

1. Donors’ Personal Profile
   1. We intend to explore ways to visualize and keep records of user donations, a method used a lot in helping form habitual behaviours. This could take the form of badges, leveling systems, etc.
   2. To encourage prosocial behaviour, we might incorporate records on common social media platforms, such as LinkedIn or Facebook – bringing direct, extrinsic rewards to donors for their donations.
2. Charity Profile
   1. Allow charities to specify donation goals for specific items (e.g. $100 towards purchasing 5 chickens for a village in Sudan).
   2. Provide updates on the impact of users’ donations (e.g. photos, graphs or statistics showing positive impact, etc.) to encourage long-term donation sustainability and donor engagement.
   3. For the two functionalities above, ensure that they are easy to use and efficient, enabling charitable organizations to add goals and provide updates in a timely manner.

In this project, we hope to evaluate this proposed system in terms of its effectiveness in motivating donor behaviour. These design directions are flexible and likely to change after conducting our field study, but in general the system will take on the form of a website or web application, as the platform is not critical to our objectives in this project.

Task Examples:

Task Example 1: Raymond

Raymond is an affluent board member of a real estate company. He generates considerable income every year. Feeling grateful and generous, he intends to donate some portion of his income to those that need it more. Most of Raymond’s donations go towards organizations he just so happened to stumble upon – for example, from street posters, or from social media content on the Internet. He finds it difficult to establish a long-term donor-to-donee relationship with these organizations, to see the results of his donations and the impact that it had. Occasionally, he visits the people he sponsors. However, as a busy man, Raymond doesn’t always have time to visit. He wishes that there was some way that he could keep up to date with how he can help, and how his contributions have made a difference to those in need.

Task Example 2: Bella

Bella is a 20-year-old college student. She is a very charitable person that has much concern about people who need help. The appreciation from people she helped also motivated her to help more of the others. She used to spare change to homeless people to help them get food. However, many news and studies showed that most homeless end up spending the money on narcotics and drugs. She felt conflicted as the way she contributed was surprisingly not her original intention. In this case, she changed her way of helping others; instead of sparing change to the homeless, she decided to donate money to charitable organizations. However, just like with her situation with the homeless, she struggles with who the money she donates to these organizations actually goes to, and whether or not it actually leads to a positive impact. As a result, Bella feels discouraged from donating money anymore.

Task Example 3: Steve

Steve has worked as a Donor Relations Coordinator for a charity for 3 years. His job is to be the main point of contact for most mainstream fundraising involving members of the public, to develop relationships with donors and to encourage them to donate. Recently, a news report showed that a number of charities were found to spend more than 80% of donations to the organization’s operating expenses. This has negatively affected the public’s motivation to donate. However, the charity that Steve is working for is a trustable organization. Since there has been a drastic decrease in donations for the charity, Steve’s manager asked him to think of solutions to increase donors’ confidence to the charity and encourage them to donate. Steve realizes that it would help if donations were made more transparent, and donors could see exactly how their money is being used to make an impact (instead of unrealistic amounts going towards operating expenses). He wishes that there was a system in place that would help improve transparency with the least amount of additional overload on their organizational practices, as their charity is already very overworked and busy.