Experiment Abstract and Materials

Pilot Test

We ran two sets of pilot tests, in which the full experiment was run on two participants. Both participants were immediate family members within the same household. Through the pilot testing, we were able to reveal the functional issues of our prototype in regards to completing test trials. Namely, we discovered that some frames did not transition appropriately based on the user’s input, affecting the response times and general feedback with the prototype. 

After conducting the surveys of each section in the experiment, we also found that it was easier to conduct a semi-structured interview using the questions from the survey rather than have the participants directly write their answers down. Two issues we found were that some questions required further explanations and that some questions relied on a specific response from the participant. One example of the latter issue was that one question asked “What factors made you think one page was more trustworthy than the other?”, but one participant rated both pages equally. As a result, we shifted to using the coding sheet as a ground for a semi-structured interview to allow for more adaptable questions.

Experiment Abstract

We introduce MealUp, a mobile app for food sharing and community building within apartments and similar building complexes. MealUp is designed to facilitate easy sharing for those providing and receiving food and encourage a better sense of community. 

In our experiment, we hope to discover:

1. The level of information in postings to maximize trust between users
2. Whether a close mapping between building layout and interface layout supports stereotypical search tasks

After conducting our experiment, we found no significant results between high and low level of information for both food posts and profile posts. Although, the level of information is more important to users in profile posts than food posts.

Additionally, we found that the list layout shows to be significantly faster for both visual search tasks. Participants also preferred the list layout in regards to enjoyment and satisfaction, but the apartment layout has a more community aspect to it.

Revised Supplementary Experiment Materials

None of the materials we used in our experiment has changed from MS III, such as consent forms or evaluation instruments. However, due to COVID-19 Pandemic and self-isolation rules, all the material was delivered online to participants.

Rationale of Medium Fidelity Prototyping Approach

Our prototype consists of three parts: a showcase of screens and interactions (mostly horizontal in scope), prepared trials in which users choose amongst trustworthy food postings, and prepared trials in which users search amongst active postings/people. The horizontal implementation serves to motivate why users might accomplish the latter two tasks, but the bulk of the prototyping effort was to support evaluating design alternatives for these tasks.

We chose this approach in order to validate certain ideas we proposed in our low-fi prototype. We wanted to test (1) what form the information on a food listing or profile should take in order to maximize a user’s trust and (2) whether making the physical layout of the building salient in the app is a good practice in terms of both efficiency and preference. As such, we decided to forego prototyping features such as text field entry or realistic progression through order creation (which would involve taking pictures of food).

To address (1), we decided to allow users to cycle through scrollable profiles and food listings in order to make more direct comparisons. As this task is largely one of visual appraisal, it was important that the features of these screens (e.g. user pictures, bios, food pictures) appeared to be realistic and differed only in information we intended to manipulate. To address (2), we fleshed out two different versions of a home screen. The ‘list layout’ was created to condense info as much as possible for efficiency’s sake, whereas the ‘apartment layout’ was made to reflect the spatial layout of the apartment. We would like to advance the latter as a way to better support thinking of the building in terms of floors, doors, and neighbours inside them. We imagine stereotypical ‘apartment life’ tasks (e.g. grabbing food from the first floor on the way out) might be best supported in this layout. With that being said, it introduces a lot of clutter. To this end, we prototyped multiple trials for two different search tasks within each layout. As we also aim to ask about preference, we decided both layouts would have realistic food and profile pictures. The apartment layout would also be displayed over a small building to better communicate what exactly users were scrolling through.

We chose to prototype in an iPad frame in Figma in order to best reflect the sort of touchscreen interaction we expect of the final interface (ideal for timing tasks) without significant effort programming for iOS or Android. This tool best supported mocking up screens and overlays for the horizontal prototype, generating believable pictures using plug-ins, displaying profiles in a slideshow-y manner for experiment 1, and mocking up rearranged test trials with time delays for experiment 2.

Prototype Demonstrations

The following is a video demo showcasing features implemented:

To serve our experimental goals, we have separate playable pages in the Figma prototype consisting of sequences of trials. These can all be played in different orders to easily implement counterbalancing. In experiment 1, users tap through a food post with a description and list of tags (left) into one with a longer description and comprehensive list of ingredients (right).

Users also tap through profiles with notable differences in bio length, app experience, profile picture, mutual friends list, and trophies.

In experiment 2, users are asked to find and tap a specific food or browse for and tap a hungry person on a given floor. We prototyped a progression through trials on each layout for each task. New targets are provided in each task in order to factor out memorization, and rearranged in order to cover a reasonable horizontal span of the lists (left) and horizontal and vertical span of the building (right).

The practice trial for each interface begins with a blinking overlay indicating scrolling directions (visible in video). Trials are separated by a timed instance of the following screen:

Revised goals of experiment

• Evaluate what features of a posting can help increase trust for a potential consumer
• Evaluate whether users prefer having an interface that allows them to scroll through a list of available items or an interface that shows all the users registered for the apartment with their current status
• Evaluate which of the above interfaces performs more efficiently at browsing both specific food postings and neighbors

One takeaway from our field study was that participants who have little interaction with their neighbors generally have a low level of trust with one another. Our first goal is to help us understand what kind of information should be shared to increase trust between users. We will be evaluating this goal through analyzing multiple features such as, attaching pictures of the food in the posting versus not attaching a picture, viewing an user with a completed profile versus a partially completed profile. 

Our second goal is to help us better understand how to design our main page in order to display important visual information. A simple design option we have is to list only items and persons that are active. Although this option considerably cuts down on info onscreen, we feel that this layout hinders the ability to visualize their community in a salient way. As such, we aim to evaluate the subjective preference of the list interface against an interface that maps more closely to the building. 

Our third goal will look at whether there is a difference in efficiency between a layout that organizes building community by room versus one that displays just active listings and active persons. This will help to inform us about any possible tradeoff in efficiency brought on by emphasizing community in a way that does admittedly introduce more clutter.

Experiment method

Participants

We aim to recruit 12 participants overall, taking a purposive sampling approach to recruitment. A representative sample of the population living in apartment buildings (or similar co-living complexes such as condos or townhouses) will be chosen. Those living in student dorms will be excluded, as findings from our field study mostly indicate either a lack of interest or lack of capacity for this kind of food sharing in temporary residences such as dorms.

Conditions

Our study will consist of two experiments, each with two tasks. All 12 participants will complete all four tasks. 

 

Experiment 1 will be comparing two different components of our interface: food posts and user profiles. For each component, we will be assessing the level of trust the participant perceives given the amount of information (Low, High) contained in each. 

The main difference between the low and high level of information category of the food posts is that the high food post contains a longer description, an ingredients list, and a separate dietary notes section. Additionally, the description also mentions that the user is looking to make friends and willing to share or trade food. For the low food profile, the dietary notes section is mixed in with other hashtag labels.

 

As for the profiles, the main difference between the low and high level of information profiles is that for the high profile, the user uses a real picture, contains the user address, has an awards/trophy section and shows mutual friends.

Experiment 2 will compare the efficiency and overall satisfaction between two distinct layouts for the main page of our interface. The two layouts that we will be evaluating are the apartment view and the list-view layouts. 

 

For the apartment view, the layout will be designed to mimic the layout of an apartment, specifically being organized in floors and enabling the user to browse other users on a specific floor. Additionally, each user may visually indicate that they are looking for food, marked with a “hungry!” (red box in picture below) indicator next to their profile, or post a food dish that is available that is indicated with a picture of their food (green box in picture below).

The list view will be simpler and straightforward, featuring two headings that indicate both users who are looking for food and food postings that are currently available.

Experimental Tasks

Experiment 1:

Meal Selection Process Task – this task consists of two parts: comparing between a high/low level of information for food posts, and comparing between a high/low level of information for user profiles. After each part, the participant will be asked to respond to a survey after each comparison.

Experiment 2:

Person-targeted task – For each of the two main page layouts, the participant will be prompted to find a person on a specific floor that is looking for food. For each trial, the screen will feature different people and food listings that the participant will have to search for. Once each trial is completed, an intermediate screen will show for 3 seconds that will prompt the participant for the next trial. An example prompt would say “Find any person living on the 3rd floor who is looking for food”.

Food-targeted task – For each of the two main page layouts, the participant will be prompted to locate a specific food listing. Each listing will be specified only by their name. Like with the person-targeted task, the screen will feature different people and food listings that the participant will have to search for. This task will also utilize an intermediate screen. An example prompt will say “Find the food listing for tomato soup”.

The ordering between these two tasks will also be counterbalanced between participants, but the participant will always complete the food listing task and user profile task before starting the person-targeted task and the food-targeted task. 

Design

Our study consists of two 2×2 within-subjects factorial designs. The first experiment consists of two levels of food postings (Low, High) and two levels of profiles (Low, High). The second consists of two levels of apartment layout (spatial vs list) and two different tasks (person-targeted search vs food-targeted search).

Procedure

Actual procedure may vary in order due to counterbalancing of conditions. The following is one example: 

Experiment 1

1. Participant will have 2 minutes to to read through a basic user profile and task description.
2. Participant will perform the meal selection process task.
3. Participant will complete a survey.

Experiment 2

1. Participant will have 3 minutes to familiarize themselves to both the spatial-view and list-view layouts of the application.
2. Participant will perform the person-targeted task after one practice trial.
3. Participant will perform the food-targeted task after one practice trial.
4. Participant will complete a survey regarding their experience with both layouts. The survey will help to gather information about which interface was more desirable. 

Apparatus

The experiment will be conducted in different project rooms around the UBC campus. The basic profiles and task description will be printed and given to the participants, and the actual tasks will be conducted on a provided tablet.

Variables

Independent variables

Experiment 1

• Profile (Levels: Low, High – within-subjects)
• Food Posts (Levels: Low, High – within-subjects)

Experiment 2

• Layout (Levels: Spatial, List – within subjects)
• Search Task (Levels: Person-centered, food-centered – within-subjects)

Dependent variables

Experiment 1

User Trust Level – using a Likert scale to record data and determine the level of trust the participant has with different food posts and user profiles (with different levels of information). The participant will fill out their preferences after the completion of the first task.

Experiment 2

Average time to complete – for each task, the average amount of time between trials will be calculated. The time for each trial will be recorded in a spreadsheet and verified by reviewing the video recording session.

User satisfaction – after completing both tasks, the participant will report their satisfaction with a brief survey that includes six Likert scale questions, and a question asking about which layout they overall prefer.

Control Variables

Experiment 1

Order of interfaces – will be counterbalanced

Order of information level – will be counterbalanced

Experiment 2

Order of layout – will be counterbalanced

Order of search task – will be counterbalanced

Hypotheses

Participant Trust Level

H1: Higher levels of trust for user profiles with higher levels of detail will be reported than those with lower levels of detail.
Null: There will be no difference in the reported values for trust between the two user profiles.

H2: Higher levels of trust for food posts with higher levels of detail will be reported than those with lower levels of detail.
Null: There will be no difference in the reported values for trust between the two food posts.

 

Layout

H3: There is a difference in the time spent in performing (search tasks) between the two layouts.
Null: There will be no difference in the time spent in performing (visual search tasks) between the two layouts

H4: There will be a difference in the reported desirability between the two layouts.
Null: There will be no difference in the reported desirability between the two layouts.

Planned statistical analyses

For experiment 1, we will be using ANOVA (Analysis of Variance) to test the difference of means between ratings, information level, and profile vs food post. 

For experiment 2, we will use analysis of variance (ANOVA) to test the difference in means between search time and the overall satisfaction between the two layouts. We will mainly be looking at whether the layout has a main effect, but we will also determine if there is an interaction effect between layouts and tasks. The ordering of the tasks and of the layouts will also be considered.

Expected limitations of the planned experiment

For experiment 1, our task would not be a realistic situation since the participant will not be going through the normal flow of the interface to get to these pages. Additionally, the participant will not be able to message the other party to develop trust between each other, which may affect the findings of our analysis, since actual users of our interface can develop trust in other ways than just the amount of information given.

In regards to experiment 2, while it is reasonable to assume that a user will want to search for both food and other users who are looking for food, we did not provide a way to filter information or perform the task based on a criteria. For example, it is likely a user would want to find meals that are suitable for a vegetarian diet. Another way to filter through postings could be to provide times in which a person will specifically be looking for food, allowing a user to plan ahead on when they will desire food. However, since a visual search forms the basis of these activities, it will still provide useful evidence for which layout is most efficient and desirable. Future research should be conducted on how to best implement some of the features listed above.

Supplementary experiment materials

Experiment 1 – Profile and Task Description

Experiment 1 – Survey

Experiment 2 – Survey

Consent Form

Call For Participation Form