Update #6 – Experiment Abstract and Materials

A. PILOT TEST

After performing 2 pilot tests of the experiment, our team decided upon a few minor modifications for the experiment protocol and study instruments.

We initially planned on video recording all 10 of the interviews, however we realized that this was not feasible due to time constraints. Instead we video recorded 5 interviews, and in order to make up for the lost data, we also did screen recordings of the entire experiment.

The open-ended questionnaires were too easily dismissed by both the pilot participants, we believe this occurred because of sheer laziness. As a result, we decided to ask participants to elaborate verbally. This seems to have provided far more rich feedback.

The final study instrument which required modifications was the coding sheet.  After performing both the pilots, it was very apparent that the coding sheet was seriously lacking.  We decided to completely remake the coding sheet with more appropriate sections and fields.

Finally, we realized that our interface was not clearing the text fields in between posting stories. This bug was resolved before we began running actual participants.

 

B. EXPERIMENT ABSTRACT

Consuming and sharing news is an essential part of today’s social media landscape. In that setting, numerous issues arise around the credibility of the news, as online activities converge around entertainment and advertising. It is difficult for news consumers to have consistent access to reporting that is factually true, valuable, and unbiased. We propose a mobile app for engaging with real-time political news, allowing the users to interact with information that is crucial for transparent democratic governance. We asked our participants to share a story with the prototype of our app and with a popular existing news app, and found no significant difference in task completion time. No significant differences in user satisfaction were observed. We also found that peer moderation can be an adequate mechanism for filtering out inappropriate stories. We conclude that our proposal is viable and suggest several crucial improvements from a prototype to the actual implementation.

 

C. REVISED SUPPLEMENTARY EXPERIMENT MATERIALS

Leave a Reply