a. Final conclusions and recommendations

Overall, our initial studies suggest that a family archiving system would be a valuable interface for future use. However, based on the results of our experiment, it is evident that further analysis must be done before continuing on any design changes as the experiment showed no difference between the two alternatives and we are not confident that we can truly accept the null hypothesis. Although Design B appears to be faster (based on mean calculation) and preferred by users, initial calculations also show it is more error prone than Design A. We take this observation with caution since our experiment did not conclude definitively that there was a difference. Due to these initial findings, we believe that our hypothesis might still be supported with a larger participant pool. Additionally, this suggests that a family tree representation we initially thought to be too complicated, might also be a valuable area of study especially when considering participant feedback wherein hierarchy and structure are given importance.
That said, we recommend also focusing the analysis on other aspects of the interface such as the timeline representation for events, which we were not able to do during the course of this experiment. We also feel that changing the appearance and layout design of the interface based on some valuable feedback from the questionnaire to avoid similar errors before performing further analysis.

b. Reflection on your design and evaluation process

We found that user involvement significantly changed the focus of our interface. Initially, we included lineage in the concept of our application but based on our field study, we felt that it would be beneficial to focus more on sharing and storing archives. Focusing on the representation of the family relations for our design experiment was a direct result of this finding. We felt that to properly support sharing of family archives, we also need to properly represent their relationships.

Interestingly enough, we found no difference between our two design alternatives, which we feel is due to the resource limitation of the experiment (e.g. time, the number of participants.) That said, we were still surprised to find Design B appears to be more error prone than Design A based on initial calculations. However, this might also be due to some appearance and layout issue that participants pointed out during the experiment. Despite these challenges, we feel that using HTML/PHP for our prototype was the best way to create our prototype.

Our evaluation process could have been improved by coming up with more stable metrics to determine what constituted valid user errors. We also could have made more tasks that involved more user decision making to create a much clearer distinction between the two interfaces. Perhaps it would be a good idea to see if maybe users would prefer to do their own graphical representation of family relations on just be able to link specific nodes of their representation to other users to which they are related. There is a lot of ground to cover in this largely untapped section of social media.

Our design was pretty simplistic, though that was mostly due to a skill deficit than anything else. The idea itself is nice in theory (i.e. give users the flexibility to define their own relations and relational groups such as grandma and grandparent respectively), but the execution proved to be much more difficult with all its moving parts and laborious data entry. Keep in mind that a lot of archiving is fairly laborious so it would be good to have some sort of minimization on the task of data entry. Our design gave some defaults to prompt users to think about certain relations and how they might fit family members into those customizable constructs, but it largely relied on the cognitive ability of a user to remember all their family ties. Maybe it would have been better to streamline it by making a user take a survey that clearly lays out to what extent they know their family and then only prompts them to enter data about members they know they have and edit them later if necessary.

Overall there were some failings in the design that we notice now, but the design served the purpose it needed to which was to get closer to a good consensus on how users like to represent their families.

The pilot test showed that users could finish the experiment tasks correctly. However, there were still a couple issues:

• There are some bugs in our interface such as some buttons were not working. Corrections were made to be sure they worked in our formal experiments.
•  Some of the instructions in our protocol were not clear. When we created the instructions for the two alternative tests, we decided to state it more specifically (i.e. instead of ‘create a tree’, we can have ‘add your grandpa, add your grandma, add your …’).
• The questionnaire had some duplicate questions, we deleted some of the questions that were not clear.

The pilot test gave us some idea of the real experiment would look like. Our users took 15 minutes to do the pilot test on average and this is also what we expected the real experiment average would be.

Number of participants : 2 , One female, One male

The following criteria should pass in order for the pilot to be marked as success :

Experiment Abstract 

The experiment was conducted with number of participants N = 8. Each participant was asked to systematically go through each design to create a family tree. The instruction for the task was precise to the name of each family members intentionally to reduce the possible noises that could arise from user errors and time factor that they may take to remember the family members. For each participants the following data was measured :

1. Time to complete the task using design A
2. Time to complete the task using design B
3. The number of encountered errors using design A and design B

The data then were combined in a table to be analyzed in R. Based on our findings the total number of errors made using each design was as following:

Design A Errors: 4

Design B Errors:7

Which indicated the design B was more error prone.

However the total mean time for design A was less, which indicated that the time required to complete the task with design B is less than design A.

Mean Completion time Design A : 7.007

Mean Completion time Design B : 6.270

ProcedureForTheParticipants

Questionnaire

CodingSheetforObservationandTimeKeeping

HCIMSIVConsent.doc

A. Rationale of prototyping approach 

i.

The prototype has pretty bare minimum functionality horizontally, so our scope is vertical on the ‘My Family’ portion of the interface. It allows the user to create a family tree based on their relations. We are choosing this vertical scope because we want to determine which interface users find easier and more preferable when building their family tree.

ii.

The simulated functionality:

– create a family tree, the user should be able to add family members onto the tree, and view the tree.

– create a event, and add people involved, date, photos and stories on to it.

– the ability to view a person’s profile by clicking on his/her icon

– users should be able to view events that are organized in a timeline.

– users should be able to share their creations to other media websites (e.g. Facebook, twitter, etc)

Many of the functionalities would be hard to implement for now, thus wizard-of-ozd is necessary. Functionality like organized event in a timeline is one of the ‘impossible to implement’ by using Axure. Thus we will have the timeline ready for the users, instead of the application generate.

We do not have real data for our testing, all family members that are on the tree and other information will be fake data. However, when the users start using the system, they may enter their real family members’ information.

iii.

Based on our experimental goals and the stage of the study, we decided to focus more on functionality as opposed to appearance. Additionally, since we are measuring user satisfaction in a post questionnaire, we felt that participants would be more open to criticize the design if the prototype does not appear complete.

iv.

We do not have any physical elements in our interface.

v.

We decided to use HTML/PHP.

Here is the link to our prototype: http://www.ugrad.cs.ubc.ca/~l1c0b/444/index.php

B. Prototype illustration

Figure 1. The home screen requires the users to either log in or sign-up. The tutorial video is present on the home page for beginning users. Video is an effective medium to let users quickly explore the application and learn the basic features.

Figure 2. The sign-up procedure requires the user to enter an E-mail, Username and password. Avoiding modern SNS applications from the requirements keeps easy accessibility for the less computer literate users.

Figure 3. The login screen requires Username and password.

Figure 4. The dashboard screen has three main features of the application. The wording of the name may not effectively describe its feature; therefore, images are another feature to prevent from misleading the users.

Figure 5, 6. My Family screen has the two designs that will be used for the experiment. Adding a button to alternate between the designs may need to be observed cautiously as it may affect the users if used beforehand. However, it is easy to compare the two.

Figure 7. The result of adding a member to a family.

A. Revised goal(s) of experiment

After discussing the progress with our TA we decided that our experiment goals were accurate, therefore we won’t make any changes to our primary design goals.

B. Experiment method

i. Participants

As previously practised successfully we will use the “Call for participation” approach. Our pool of potential participants will include UBC students. While we are aiming for the total number of participants to be 12 we are confident that with 10 participants as the lowest threshold the experiment can be concluded. The participants will be combination of female and male between the ages 18- 65. Each member of the group will recruit 2 participants, ideally one female and one male. Other characteristics of our participants will be:

• They should have basic computer literacy. We will divide them into two levels of proficiency with computer applications and measure how comfortable each group are with the design. The distinguishing factor in determining proficiency will be previous experience with similar applications.
• In order to achieve rich data results we will aim to increase the total number of participants if possible. The more participant we recruit the richer our results will be.

ii. Conditions

We will compare two types of designs to create a family tree in terms of speed, accuracy and user preference. The participants will go through both design alternatives of a single page by following the completed design of the application two times. Design A will give users the option to create their family tree as they wish. They can use add button to add a new family member and edit them, while Design B will be more focused on filling out the blanks with the information of their family members. The degree of freedom in Design A is more relative to Design B.

iii. Design

The experimental design is a 2 factor mixed design. There will be 2 within-subject independent variables (design A and design B) and 2 between subject variables (self-reported knowledge of family tree interfaces).

iv. Procedure

– Participants will be asked to create a family tree

– Participants will be asked to create and view events

– Participants will be asked questions related to the two different design

– Participants will be asked to fill out questionnaire and interview questions

v. Apparatus

Our main medium to showcase the designs will be a mid fidelity prototyping tool. The experiment will be conducted in Axure, where each of five experiment conductors have equal access to. The official setting will have the following features:

• Quiet with as little distraction as possible.
• Windows 10 environment with prefered resolution 1366×768 (the most common screen size for website design based on https://techcrunch.com/2012/04/11/move-over-1024×768-the-most-popular-screen-resolution-on-the-web-is-now-1366×768/)
• Standard mouse and keyboard
• Time will be measured by windows Alarm and Clock app.

vi. Independent and dependent variables

Independent Variables

For this experiment, we have one within-subject independent variables (design A and design B) and one between subject variables (self-reported knowledge of family tree interfaces). The Design Types are differentiated by groupings with Design A having grouped relations when creating a tree as opposed to Design B which only lists family relations. We also have three dependent variables measured as follows:

Dependent Variables

1. Speed : We will measure the speed using standard time from beginning to end (when a tree has been created) of the task being performed by a participant . Observation and timekeeping (stopwatch) will be our prefered method.
2. Accuracy : Accuracy will be measured by counting the number of errors the users make in using the designs. Errors will be assessed based on the outcome of the task (i.e. is it the correct tree?) using a common checklist.
3. User Satisfaction and Preference : Users will fill up a short questionnaire which measures their subjective opinions on the efficacy of both designs. If necessary, follow-up questions will be asked based on participants’ answers to questionnaires.

vii. Hypotheses

H0: There is no difference in user performance (time and error rate) when entering family history data using either designs A or B  

H1: Design B is faster in terms of user performance

H2: Design B is less error prone in terms of user performance

H3: Design B is the user’s prefered design

Measuring the goals in the predefined scope:

1. Speed : We will measure the speed using standard time in terms of delays in seconds using either of designs. Observation and time-keeping will be our preferred method.
2. Accuracy : Accuracy will be measured by counting the number of errors or lapses the users make while using the designs. Also the number of corrections users make afterwards could be indicator of accuracy.
3. User Satisfaction and Preference : Users will fill up a short questionnaire which measures their subjective opinions on the efficacy of both designs.

viii. Planned statistical analyses

Based on our hypothesis and variables, we plan to perform a two-way ANOVA. The ANOVA will be used to test for significant interaction between the design type and self-reported ability with respect to speed, error and satisfaction.

viiii. Expected limitations of the planned experiment

Some limitations may occur during the procedure of selecting participants, as the experiment expects the participants to have basic computer literacy; however, as participants are self-judging their computer literacy, their judgement may be different from the expectations of the experiment. Also, some of the visual features of the prototype may not be relatable for participants who have different cultural background where visual display of a family may have different model.

Supplementary experiment materials (Click to Download)

Consent Forms

Questionnaire

CodingSheet

A) Task examples 

Based on the data we gathered from our initial study, participants expressed little desire in finding their lineage but instead focused more on organizing and sharing family events within each other. With this information, we decided to use our task examples #1 and #3 only, since our second task example focused too much on family lineage. We use these two task examples as #1 focuses on the collection of family information while #3 focuses on the sharing of this data, which is what participants were inclined to do based on the study. Note that we keep the idea of a family tree in the task examples as we wanted to still show the relations between family members – especially those involved in a particular event – although it is not as robust or complex as accurate family trees.

B) Low-fidelity prototype illustration 

Link to the Narrated Video of the Low-Fidelity Prototype: https://youtu.be/0ys1i5Wod6Y

C) Description of prototype

We decided to focus our interface around event-sharing since our field study yielded a lot of rich data about users wanting to capture events as opposed to lineage. Lineage was still considerably important for users to orient themselves within their family stories, but events were more important for making archiving an interesting activity altogether. As such, we have chosen to support task examples #1 and #3 from our Blog Update #2 post. Task example #2 will have some support, but the main focus of that example is straight reporting on family lineage in a tree format so it does not line up with our focus. Task example #1 focuses on sharing of information corresponding to family members and task example #3 focuses on collecting information to preserve for later review. We designed our prototype with this in mind.

The scope of our prototype is pretty horizontal with basic functionality across all parts of our prototype (in our video, the flow shows all parts of the prototype: entry, dashboard, my family, my events, and all events). We wanted to focus on making the design intuitive to the user (i.e. the user should know what they are doing, what should happen when they do something, and how to execute appropriate tasks intuitively). As such, we do have some alternative design screens within our prototype and we plan to create an experiment to determine which of the screens is more intuitive.

D) Walkthrough report 

For our first task example, our user can sign in and start to build her family by clicking on the ‘Create My Family’ icon. Family tree creation is simple and clear. During the walkthrough, we figured this may also be a little bit vague for users so we might change this button to hide ‘Create’ if the user already has a tree created. The creation form is straightforward and the view of the tree is intuitive and easy to follow because of the labelling. For the events page, we have a simple ‘add a new event’ page. The timeline/chronological view makes it easy for users to search for the event they are interested in more conveniently.

The interface satisfied the basic needs in the second example, the user can create her family tree, and upload photos and stories. The user may also have the same problem as the first task example, but overall the users can achieve their goals without confusion. That said, we are not supporting this task example fully.

For the last task example, our walkthrough showed that Mark can create his family tree, upload family photos, and archive family events. This is also like the first task example, but in this task example, the user would also like to share what he built with other people. We do not really have a ‘share’ feature yet in our prototype, we considered the link account method (so that people can link their account to other family members) but it may be better to have one button that let users to share their creations (like a link sent by email or even share on other media platforms). Since this is one of the task examples we are supporting, our walkthrough leads us to conclude that we need a sharing feature at bare minimum. We also liked the ‘People Involved’ feature in our event page because it makes it easy to know who got involved in each event. Instead of creating the same event in every member’s account, people can have the same event page for the event they participated together.

E) Proposed goal of experiment

At this stage, we decided to focus on identifying a better design for a representation of family relations. The goal of our experiment is to identify if there is a meaningful difference in user performance and satisfaction when using either of the two design templates we present in our video.  In our video, we show these alternate designs side by side and explain their differences.We hypothesize that design B (with groupings) is superior to A (without groups) in terms of speed, accuracy, user satisfaction and preference. Our hypothesis include :

H0: There is no difference in user performance (time and error rate) when entering family history data using either designs A or B  

H1: Design B will be faster and less error prone than Design A in terms of user performance

Measuring the goals in the predefined scope:

1. Speed : We will measure the speed using standard time in terms of delays in seconds using either of designs. Observation and time-keeping will be our preferred method.
2. Accuracy : Accuracy will be measured by counting the number of errors or lapses the users make while using the designs. Also the number of corrections users make afterwards could be indicator of accuracy.
3. User Satisfaction and Preference : Users will fill up a short questionnaire which measures their subjective opinions on the efficacy of both designs.

A) Recommendations

Based on the conclusions from the field study we conducted, we recommend focusing the development of the application on sharing functionalities of media archives that will allow users to build their own family histories, rather than searching or looking up family history and ancestries through census records or related genealogy records. This is based on our findings that although participants are curious with their genealogy and background, the main reason for reviewing archives is nostalgia. As such, participants enjoy sharing these archives with several other family members. Participants also expressed knowledge of family history up to their grandparents, so we want users to be able to contribute these knowledge between family members as opposed to collecting information from other people.  

Additionally, most participants what their ancestors looked like, which is why we recommend focusing on media archives that  are more visual.  

We also found that users prefer their current method of sharing system, which is electronic-based, due to the ease of the storing and sharing capabilities. However, participants also expressed that they have lost information as there is no clear organization when using laptops or USB devices. As such, we suggest a navigable family tree where the archives will be attached to. This way, users will be able to locate information in a specific way.

Finally, since most participants use web-based systems such as social media and email to share these archives, we believe using a browser-based interface is the best way to represent the system.

B) Updated task examples

Revised Task Example #1:

Leslie wants to map out her family lineage. She discusses family ties with her grandfather. As her grandfather tells her about each person in the family, he tells interesting stories about them and tells her how they are related. Leslie draws out a family tree on paper starting with her grandfather and grandmother and branching down to her nieces and nephews. She wants to keep track of important photos and documents of each person on the family tree, so she sorts photos and documents into piles corresponding to the members of the family on the family tree.

Revised Task Example #2:

Maggie is a student who is taking History in high school, and the teacher assigns a project to make a family tree for each student’s family. Maggie, only knowledgeable about her parents’ lives, asks her parents about some of her grandparents’ interesting historical moments that they know of. Her mother tells her some of the obvious facts like their hometown, age, etc. After hearing the stories, Maggie becomes curious and strives for more interesting stories that her grandparents would like her to know about. As Maggie’s family lives far away from her grandparents, she does not often call her grandparents and she is afraid that it may be awkward to ask. After pondering for a while, she decides to only present the obvious facts on the family tree project.

Revised Task Example #3:

Mark’s family is going back to their home country for a family reunion since migrating to Canada years ago. As a part of the event, he wants to showcase their family history and ancestries. He compiles all his family records and photos from his own collection. Since he does not have all the information he needed to form the tree, he ask elders in his family about family history. After he got all the information he can get, he draw a family tree on a piece of paper  based on his findings and his collection, so that he can show it to the new generation and reminisce with family members. He plans to take pictures and make photocopies, so he could share his collection to other family members as a remembrance of the event. He also would like to be able to send the pictures of the collection to family members who are unable to join the event.

We changed the first and third task example as per TA feedback. The first task example needed to be more complete in its task example. It was not clear what the person was using to complete the task. The field study helped us understand why this realistic task example is not supported very well. There are plenty of issues that can arise from a family tree project that involves collecting documents, artifacts, and photos of family members that grow exponentially as the tree widens with children. Considering that one person might not actually have all the information about a particular family member, the family tree will probably be doomed to never be as complete as it could be if the task were supported better. The third task example has the same problems, from the field study, we discovered that people are not so familiar with their family history, mainly up to grandfather and grandmother generation. Thus, it will be very hard to form a complete family tree.

C) Prioritized list of requirements

Absolutely must include

• A family tree
• Profiles of each family tree member
• Image archives for each family tree member
• Sharing between family tree members
• Browser based interface

We want there to be some sort of navigable hierarchy of information having to do with lineage. Just as well, each family member needs to be associated with information about them in support of the task. Sharing is also important as a collaborative environment leads to a more complete picture of the family. We chose a browser based interface to allow for a larger screen and more control over interface elements.

Should include

• Curation permissions responsible for particular family tree nodes

The information should have a person in charge of organization and censorship. If it is the person who the node is about, all the better. Without curation, the tree could become offensive to involved parties (i.e. maybe you do not want your family to see a certain picture of you).

Could include

• Sharing between friends and outsiders

The idea with possibly including sharing is to foster more collaboration or to just allow for exploration from outside users. For example:  http://pocahontas.morenus.org/poca_desc.html is a website that shares the lineage of Pocahontas down to a living descendent. This information may be of historically interesting value to outsiders.

Could exclude

• Accessibility features (visual impairment, hearing loss, etc)
• Mobile features and functionality

Accessibility complicates our interface with the need to support many varying disabilities. For the purposes of our early interface fidelities, we will not bother with that kind of complexity. The mobile features can also be ignored in a lieu of a more robust system with a larger graphical interface.

Users to Include

• Individuals old enough to own an email account
• Individuals with enough knowledge to navigate a web interface
• Individuals who are interested in exploring or curating information about their lineage
• Individuals who want to submit information to build their family tree
• Individuals who are invited to explore others’ lineages

Users to Exclude

• Technologically illiterate people
• Curation abilities for children below the age of 13
• Individuals who are uninterested in their own heritage

The interface will not support the technologically illiterate because teaching users to become literate with technology requires too much outside the scope of the project. Children are chosen for a similar reason in that they can not actually sign up for email, so it is a reasonable assumption to exclude them from being curators.

D) Design alternatives

Alternative #1

The general idea of this proposed design is following the same path as Google Doc and OneDrive with more strict sharing system .The difference though, would be the application UI is curated to be more informal and User friendly especially for people with less computer literacy. Each group or family has a personalized space to themselves and from the index page the user have access to different categories. Each category or folder can be edited but can’t be removed. Inside each category users can create event based folders. This type of design is privileged to be quality tested as “working” in real life but the downside is it would not be so creative. Another downside is that the users are power users, and the files could still be disorganized after folder creation.

Alternative #2

The idea of this design is to have a navigable family tree that is always focused on a particular leaf to reduce confusion. In the diagram, Greg Mander is focused and so his parents, spouse, and issue are the subcomponents of the visual interface. Clicking on a new leaf or navigating to a sibling changes the focus. Focused leaves can be investigated. The idea with the investigate icon is to give users access to an almost Wikipedia-esque view of a particular family tree member. At the bottom of the page (represented by the bottom ripped image), there is an archive that stores information about a person. The maintenance of these pages would be handled by a curator, either the person who the page is about, if they are alive and interested, or a family member.

The upside to this design alternative is that it offers an intuitive way to explore family lineage since there is a lot of transfer knowledge from other interfaces. The downside is that there are lots of complex family relationships that need to be handled. In the diagram, Kimberly Mander has a child that was not born out of the marriage with Greg Mander. That is an example of a way we could handle a complex relationship, but consider the possibility of multiple spouses or multiple parents, adoptive or biological. The system would be directly susceptible to extreme situations.

The original idea was to create an application to help share huge libraries of family documents, photo albums, or any kind of data within a family tree.

1. In order to narrow the scope, we reduced the load to only basic information, scannable documents or pictures (history objects), with documents being really anything a user might find pertinent about a particular leaf (node) on a family tree.
2. Another change was to include an actual family tree with family members as ‘leaves.’ Family members can vet history objects attached to their own nodes or nodes of post-humous relatives, or attach a history object to another leaf in a family tree.
3. In order to support manageability, we also decided to handle information overload by narrowing the interface to focus on parent leaves and their child leaves as a vertical relationship, and parent leaves’ siblings as a horizontal relationship. At any one time while using the interface, the focus will be on just parents and their children with options to navigate horizontally or vertically.

Task Example #1

Leslie wants to map out her family lineage. She discusses family ties with her grandfather. As her grandfather tells her about each person in the family, he tells interesting stories about them and tells her how they are related. Leslie makes family tree starting with her grandfather and grandmother. She wants to make notes on the tree about her grandfather and attach a picture of him. She does the same for her grandmother and all subsequent family members until she has a tree mapping all her aunts, uncles, cousins, parents, and siblings.

Task Example #2

Maggie is a student who is taking History in high school, and the teacher assigns a project to make a family tree for each student’s family. Maggie, only knowledgeable about her parents’ lives, asks her parents about some of her grandparents’ interesting historical moments that they know of. Her mother tells her some of the obvious facts like their hometown, age, etc. After hearing the stories, Maggie becomes curious and strives for more interesting stories that her grandparents would like her to know about. As Maggie’s family lives far away from her grandparents, she does not often call her grandparents and she is afraid that it may be awkward to ask. After pondering for a while, she decides to only present the obvious facts on the family tree project.

Task Example #3

Mark’s family is going back to their home country for a family reunion since migrating to Canada years ago. As a part of the event, he wants to showcase their family history and ancestries. He compiles all his family records and photos, creates a family tree to show to the new generation and reminisce with family members. He plans to create copies of his collection so he could give copies to family members as a remembrance of the event. He also would like to be able to give copies to family members who are unable to join their reunion.