Author: tzu hsu chu

Task #6: Examination and Anti-racism Speaker Series Alignment Through Making — Pedal Perspectives, Navigating Bike Culture Through Gender, Race and Justice

Brief

The purpose of the task is to critically examine the role of equity, diversity and inclusion within the context of makerspaces, while incorporating decolonization and anti-racism frameworks through the examination of the content in MET’s Anti-Racism Speaker Series. Design a maker challenge or provocation that fosters an environment of equal participation, cultural understanding, and social justice within the makerspace community.

Choose one Anti-racism Speakers Series presentation or podcast to frame the challenge/provocation. Make note of key ideas, theories and research presented in the series. This challenge/provocation should actively promote the participation of individuals from diverse backgrounds, including those historically marginalized and under-represented in the field. Integrate cultural sensitivity and awareness into the design process, ensuring the activity is respectful, accommodating, and accessible to all participants. Use the following templates to design the challenge/provocation.

Inclusive Maker Challenge Template

Inclusive Provocation Template

Find the full Word document here:

Pedal Perspectives – Navigating Bike Culture Through Gender, Race, and Justice

Overview of Provocation

This provocation is designed to encourage students to explore the local bike scene and explore how community, identity and inclusivity intersect in cycling culture.

In Pedal Perspectives, students are encouraged to read some bike zines, visit their local bike shops, meet local riders and perhaps join a biking event, and consider the ways that these spaces are — or are not — accessible and welcoming. How do gender, race and identity shape our experiences in these spaces?

While cycling has surged in popularity as a mode of transport, fitness and community engagement, scholarly research on bike culture remains scarce. Traditional academic circles often overlook the nuanced, lived experiences of cyclists, especially those from marginalized groups.

In conjunction with the Anti-Racism Speakers Series podcast Pervasive Racism and the 2SLGBTQIA+ Community, guest speakers Alex DeForge and Meera Dhebar talk about the lack of meaningful representation, the impact of stereotypes, and need for more intentional and inclusive effort to reflect diverse identities. Both speakers emphasize that critical literacy is the key to empower individuals, and to understand the diversity and depth of the bike community, we need to explore beyond mainstream publications and delve into more underground and personal spaces – like zines.

“The history of lack of women, LGBTQIA+, and BIPOC representation in cycling reveals their marginalization and exclusion, especially in cycling media and sport. And it speaks to a bigger cultural issue of how marginalized groups are continually ignored and misrepresented. We cannot shy away from telling our stories in cycling just because it doesn’t suit the narrative the cycling industry sells.” (Cyclista Zine, 2019)

Zines are booklets (like magazines) that are used to share information, knowledge or experiences with others. They are often made with pen, paper, and collage techniques. They can also be made on computers using digital publishing software.

These DIY publications offer a more authentic look at the voices, struggles and triumphs within biking subcultures, often addressing topics such as gender identity, race, accessibility and social justice that are otherwise absent in academic discourse. By engaging with zines, we can gain more personal and grassroots perspectives that shape biking culture, and illuminate the vital, under-represented voices that mainstream studies miss.

Through guided reflections and interactive exploration, and creating your own zine, this project encourages students to investigate how their own backgrounds and identities relate to the biking community, in hopes to start a conversation about inclusivity, equity and what it means to belong and participate in a community.

Materials and Resources Required

    • Device (computer, tablet, phone) to take notes or document reflections
    • Pencil, pens, paper, magazines, flyers, art/craft supplies etc.
    • Optional: access to a bicycle

Inclusive Maker Provocation Instructions

Part 1: Pre-activity Reflection — Understanding your own positionality in bike culture

Think about your current experiences and knowledge when it comes to biking and bike culture. By reflecting on these questions beforehand, you can better understand your personal lens and become more aware of how your own experiences and identities shape the way you engage with biking spaces and community interactions.

Some questions to consider:

    • What is your personal history with biking? Do you view it more as a form of transportation, recreation, fitness or something else?
    • How often do you engage in biking activities, and in what kinds of spaces (i.e. recreational trails, urban commutes, group events)?
    • Have you experienced barriers to accessing biking spaces, whether economic, social, physical, etc.? How have they shaped your relationship to biking?
    • Do you see people who share your identity and background in biking spaces you frequent? How does this affect your sense of belonging?
    • Are there stereotypes or assumptions within biking culture that impact on how people perceive you or that shape how you perceive others?
Part 2: Participate —
Visit your library’s zine section
    1. Visit your library’s zine section, and browse the selection of bike-related zines
    2. Select a few zines to read, pay attention to the different voices, styles and stories represented
    3. Take notes on reoccurring themes, especially in relation to gender, race, accessibility or personal experiences in the bike community

Some questions to consider:

    • Who is the author and how does their background and identity impact their experiences within the bike community?
    • How do the perspectives in the zines compare with the mainstream cycling media or bike shops?
    • What issues or themes appear frequently across the zines? What is something novel or surprising that you’ve noticed about the zines? What are some themes that you can relate to personally?
    • How do the zine creators express their identities and experiences through their zines? Are there recurring challenges or barriers that they discuss?
    • How do the zines influence your understanding of the biking community? How does that affect your perspective of your understanding of the bike community?

If you do not have a zine section at your local library, consider reading through online eZines related to bike culture, here are some recommendations:

    • Cyclista Zine Library : a collaborative zine made from intersectional feminist and DIY culture lens. It is run by zinester Christina Torres and celebrates intersectional feminism through zines, live events and workshops.
    • Microcosm Publishing Bike Zines : should you have the funds and means, check out some bike zines Microcosm Publishing, a zine publisher based in Portland, Oregon.
Part 3: Create — Record and share your experiences in a zine

    1. Follow the instructions above to make a simple one-page zine as a base to document your reflections from the previous activities.
    2. Embellish your zine with illustrations or make a collage with my arts and crafts supplies that you have available.

Some additional resources: 

The Creative Independent – How to Make a Zine

Critical Questions for Consideration

    • How do you feel about entering spaces that may be new or unfamiliar? Are there any assumptions or expectations you hold about bike shops, biking events, or zine culture?
    • In what ways are you open to letting this experience challenge or expand your existing views on biking culture, inclusivity, and community dynamics?
    • In what ways could this provocation be adapted to other subcultures or community activities where marginalized voices are underrepresented? How might this broaden students’ understanding of inclusivity in various societal domains?

Background/Additional Information

Inclusivity Focus

This challenge considers the EDIDA framework with emphasis on exploring biking spaces in relation to the student’s own positionality, encouraging students to access who these spaces are designed to serve and who may be excluded or underrepresented.

The activity also includes diverse ways to engage with the biking culture and community (see Extension section below) – whether it is visiting a bike shop, attending an event, or exploration via zines — students with varying physical abilities, comfort levels and preferences can participate meaningfully.

Lastly, I have purposefully included zines as a source of learning as it engages with the decolonization framework by valuing non-traditional, grassroots forms of knowledge production and distribution. Zines often represent marginalized voices and disrupt dominant narratives, offering insight to biking culture and underrepresented voices often missed in academic literature.

No Tech, Low Tech, High Tech Options

Zines are quintessentially low-tech, DIY methods of sharing knowledge and experiences, easily made with pen and paper, and a photocopier if you would like to share and distribute your zines!

Some higher-tech options for sharing similar kinds of DIY content include:

    • Blogs/Personal Website
    • Digital Zine/eZine
    • Social Media
    • Podcast/ Youtube Channel

Extension

Choose an activity below and follow the guiding instructions to help reflect on your experience. These activities are made to include a broad range of involvement and participation, and are mindful of certain barriers, such as access to a bicycle, different levels of mobility and skill levels, and meant to expand in the different ways participation can look like.

Nonetheless, these activities and questions aim to deepen your understanding of the diverse, often overlooked experiences within biking culture and help you explore how your own perspectives interact with these communities. Enjoy the journey!

Visit a local bike shop
    1. Choose a bike shop near you and make a visit. As you explore space, pay attention to how the space is organized and who the shop seems to be designed to serve.
    2. Observe the shop’s environment, the types of bikes and gears available, and any information displayed about events or cycling groups.
    3. If possible, strike up a conversation with an employee to learn more about the shop’s role in the local biking community

Some questions to consider:

    • Who does this bike shop seem to cater to? Are there specific demographics that seem prioritized in this set up, product offerings or marketing?
    • What aspects of the shop make it feel inclusive or exclusive? Consider factors like pricing, layout and accessibility.
    • How does the shop engage with local community? Do they advertise any events or promote social causes on their bulletin boards?
    • How did you feel going into this space, what about it made it feel welcoming or intimidating? Did you feel comfortable being in this space?
Participate in a bike event
    1. Find a local bike event that is open to all skill levels, like a group ride or a community bike day.
    2. Participate in the event, look around and make note of the people involved, the event’s purpose and the general atmosphere
    3. Notice how newcomers are welcomed, and whether accessibility (in terms of skill level, gender, background) is addressed during the event

Some questions to consider:

    • What kinds of people attended this event? Are there particular groups that seem well-represented or under-represented?
    • In what ways was the event accessible or inaccessible to people with different needs or backgrounds?
    • How did the event organizers foster a sense of community or shared purpose? Were there any efforts made to address diversity or inclusivity?
    • How did you feel about participating in this event? How does your own identity and experience shape your sense of belonging or connection? Would you participate in a similar event again?

Provocation #2: 2SLGBTQIA+ Community Inclusion

Brief

The intention of the provocation is to build awareness and acceptance of 2SLGBTQIA+ individuals and communities in order to create a welcoming and inclusive environment for all.

Part 1
  1. Visit the ASEAN SOGIE CAUCUS website
  2. Based on what you’ve read, what are your thoughts?
Part 2
  1. Watch the following Youtube video:
  2. Based on the video, what are your thoughts?

 

Using the analytical and application synthesis portion of the chart above , what critical questions come to mind that you wish to investigate further?

Critical Questions for Reflection
  1. Compare and contrast the website to the video
  2. How might the website and video be used to honor the intersectionality of culture, gender, race and 2SLGBTQIA+ communities through making?
  3. What are your key takeaways from the provocation?
  4. How can you extend this provocation?
  5. How might this provocation be extended to create an inclusive Makerspaces challenge?

 

Task #4: Race, Gender, Identity and Intersectionality in the Maker Environment

Brief

Design a critical and cohesive overview of race, gender, identity, and intersectionality in the maker environment. Introduce others to representational problematics in makerspaces and communities.

Choose your means of presentation so long it is multimodal.

Project: Inclusive Makerspace Twine Game 

This is an interactive text-based story surrounding a group of diverse makers that challenged the space’s gender norms and definition of “making” in the local makerspace the Iron Forge.

There are two main aspects of inclusivity that I focused on, specifically on gender and culture.

The story draws upon themes mentioned in the lecture by Kimberly Martin on   Centering Gender: A feminist analysis of makerspaces and digital humanities centres  especially the notions of the difference between “Making vs Crafting” and how that often is intertwined with the idea of “New Domesticity” when it comes to “softer” forms of making with sewing, knitting, weaving, etc. having it be dismissed as mere “craft”.

Another theme that really resonated with me in that lecture was the idea of “invisible labor” of women when it comes to expectations within makerspaces and the stereotypical gender roles affecting how work is organized into private and public spheres that in turn shape how we categorize tasks into “men vs women’s” work.

The narratives I have included are completely fictional and made with the help of ChatGPT. I wanted to utilize AI in this project as I was curious to see what stories it creates and the underlying biases and stereotypes that arises when it comes to inclusivity and diversity in makerspaces.

Below is an excerpt that I did not use in my story, but was given to me after prompting.

As the weeks passed, the dynamic between the men and women evolved. The men, who had been skeptical, began to respect the women’s leadership, learning new techniques and embracing projects they had never considered. Meanwhile, the women appreciated the technical knowledge the men brought, weaving it into their more holistic, creative projects.

By the end of the month, the makerspace had become a true melting pot of ideas, where gender was no longer a barrier to creation. Men and women worked side by side, designing everything from wind-powered kinetic art installations to smart home devices that responded to voice commands. They learned from one another, breaking down stereotypes and building something even stronger: a community that thrived on diversity and collaboration.

The women had taken charge, not by forcing the men out, but by expanding what was possible in the space—and in doing so, they had created a makerspace that belonged to everyone.

It felt like it was perpetuating the gender binary, even though there was one non-binary character in the story, and it felt like it was lumping “non-men” into one category of “women”. The erasure of non-binary-ness was somewhat surprising to me, and it played into a conundrum that I had when I was creating this project. Namely, the challenge of  wanting to “break gender stereotypes” but needing to lean into and “exaggerate” the stereotypes in order to highlight that gap. I felt like that was the case when I was reading some of the papers, like by talking about under-representation first entailed the re-enforcement of gender stereotypes!

In terms of culture, I think the story reflects the kinds of cultural assumptions nested within the creation tools used in makerspaces, especially the recent tendency for integration of Arduinos, circuit boards, e-textiles to add the “Art” into STEM. Many papers seems to place an emphasis on empowering girls to participate more in science and computational activities, but it feels like there is less of a push to empower boys to lean into their artistic side. Seo and Richard (2021) mentioned that bidirectionally responsive design (BRD) activities were shown to “dissipate gendered preconceptions, and the learners across gender equally saw value of crafting, coding, sewing and design”, which might be a start, but I am curious to investigate more into such design-thinking and the more practical aspects in a broader context.

Ideally, I would have liked to include characters that are based on lived experiences, and to make sure that the representations of the characters are reflective of people with such positionalities that the characters themselves occupy. Potential development would be by asking friends with culturally relevant maker experiences to be part of the storytelling.

Should I have more time, I would have liked to explore more into disability and accessibility aspect of makerspaces in terms of content and subject matter.

In terms of the making of this artifact, I would have liked to find ways to embellish the story with more visual elements, such as photos of the makerspace, the characters, the kinds of projects they are making, etc. to create a more immersive and visual experience.

 

References

Martin, K. (2017, November 28). Centering gender: A feminist analysis of makerspaces and digital humanities centers [Video]. Media Collections Online. https://media.dlib.indiana.edu/media_objects/47429d523

Norris, A. (2014). Make-her-spaces as hybrid places: Designing and resisting self constructions in urban classrooms. Equity & Excellence in Education, 47(1), 63-77. https://doi.org/10.1080/10665684.2014.866879

Parekh, P. (2024). Girls’ reluctance and intersectional identities in STEM-rich makerspaces. Education Sciences, 14(6), 628.

Seo, J., & Richard, G. T. (2021). SCAFFOLDing all abilities into makerspaces: A design framework for universal, accessible and intersectionally inclusive making and learning. Information and Learning Sciences, 122(11/12), 795-815.

 

Task #1: Critical Investigation into Inquiry and Liberatory Design Thinking in the Maker Environment

Brief

Design an interactive and rich presentation that addresses the following questions:

    1. What is a makerspace?
    2. What does it mean to have a maker mentality?
    3. Why are makerspaces becoming more popular worldwide?
    4. How are constructivism, inquiry-based learning, and liberatory design-based thinking fundamental theories and pedagogies of makerspaces?
    5. How are future-ready skills and competencies developed through making?
    6. How are makerspaces being utilized to solve complex and critical problems?
    7. Section for free-choice topics or additions

 

Provocation #1: Notice and Name

Brief

Choose a news article from the given options and ask yourself questions to identify what perspectives/peoples/voices are missing. How can these issues be solved without considering perspectives? When creating inclusive makerspaces, it is the missing perspectives that need attention and emphasis.

Ontario megachurch’s former victim’s advocate concerned after role goes to person with ties to denomination

The article title first and foremost points out the obvious problem – to have the first point of contact for sexual assault survivors be someone connected to the church’s denomination brings up a huge conflict of interest and questions whether or not this advocate is acting in a survivor’s best interest.

Another layer that may create a challenging environment for survivors to come forward, is the tension between religious doctrine and experiences of (sexual) abuse, especially if the advocate is not independent from the church’s teachings.

If we are not talking about the article in a self-contained way, and a more generalized way… is that religious institutions may already potentially reinforce colonial power dynamics on Indigenous peoples, LGBTQ+ and other historically oppressed communities and further perpetuate experiences of trauma.

Task #2: Makerspace and Inclusivity

Brief

Part 1

3-3-1 Method:

  • 3 sentence summary for each article
  • 3 sentence bridge between the three articles highlighting commonalities, discrepancies, linkages, contesting views, etc.
  • 1 question for each article to extend critical thinking

Part 2

Present a summary of your learning from your 3-3-1 analysis using Plotagon

Part 1

Corsini, L., Jagtap,, S., Moultrie, J. (2022). Design with and by marginalized people in Links to an external site.humanitarian makerspaces International Journal of Design, 16 (2), pp. 91-105. doi:  10.57698/v16i2.07. 

This paper explores how marginalized people use humanitarian makerspaces to create projects that meet both basic and high-level needs, like belonging and self-actualization. This supports Max-Neef’s idea that personal fulfilment comes from satisfying multiple needs at once. Further research should focus on how the design process itself can empower marginalized people and lead to impactful outcomes.

Question: The article emphasizes designing with and for marginalized people, what could that look like in your own personal contexts?

 

Melo, M. (2020). How do makerspaces communicate who belongs? Examining gender inclusion through the analysis of user journey maps in a makerspace. Journal of Learning Spaces,  9(1), 59-68. 

This paper explores how gender identity affects participation in makerspaces. The available technologies often send gendered signals, making men feel more welcome than women. A deeper look at the environment and materials in the spaces may help promote greater gender equity and representation.

Question: What were your personal experiences with interacting with tools in makerspaces, were you intimidated by certain tools and technologies more than others?

 

Kye, H. (2020). Who is welcome here? A culturally responsive content analysis of makerspace  websites. Journal of Pre-College Engineering Education Research (J-PEER), 10(2).  https://doi.org/10.7771/2157-9288.1190 

This paper shows though makerspaces can help create more equitable STEM opportunities for marginalized students through collaborative learning, intentional efforts of integrating culturally responsive pedagogy (CRP) is needed to support both the academic success and student’s cultural identities.  More practical guidelines on CRP principles are needed to better ensure equity and diversity in makerspaces both in-person and online.

Question: In your personal contexts, what are some practical culturally responsive pedagogical changes that can be applied?

Convergences
All three papers highlight the need for more intentional design of makerspaces to promote equity and inclusion for marginalized groups. Without conscious efforts, makerspaces risk perpetuating existing inequities related to gender, race and socioeconomic status. Fostering inclusive environments require purposeful frameworks and guidelines to empower marginalized participants and support diverse identities.

Part 2 

The Maker In Me

Brief

Introduce yourself as a maker and reflect on your own professional philosophy and how you view yourself as a maker, as a critical and innovative thinker, and as a designer. You will align your own personal and professional philosophy for your learners within a maker environment. Your introduction will include:

    • A visual representation of your values, approach to making, and educational/leadership philosophy inspired by the course readings
    • Investigate multimodal means to design a creative and critical introduction
    • Include your goals for this course within your introduction

Some guiding questions:

    • How do you describe yourself as a maker, and your maker philosophy?
    • What is your current educational/leadership philosophy?
    • How does your philosophy fit into the maker movement?
    • What is your experience with Makerspaces?
    • What is your experience with EDIDA Frameworks?
    • Why are you taking this course? What are your expectations and goals?

 

GeoDASH: Exploring Predictive Policing Technology

As someone living in Vancouver for the past 9 years, the first thing I did when I saw the map was zoom into my neighborhood to see what has been happening in my surrounding areas. In addition, I also looked around the locations that I frequent.

After tinkering with the map, I decided to look read the GeoDASH FAQ page for more details on how the data was collected and visualized on the map. Overall, due to the sensitive nature and need to protect the privacy of those involved in the incidents, it seems like most of the information displayed can only be considered as proxy measures, as most of the actual locations have been rounded to the approximate block level.

With the lack of transparency in terms of how information is being collected, reported, and mapped, it is rather challenging to assess the validity of information presented on GeoDASH at face value without cross-referencing other sources, to gain a more comprehensive understanding of the crime trends in different neighborhoods.

Some inherent biases that may appear when interpreting the data on the map are intersectional in nature— using GeoDASH without additional understanding of geographic, socioeconomic, and demographic characteristics of Vancouver may result in misinterpretation of crime trends.

For example, geographically speaking, certain areas may have higher concentrations of crime due to population density, land-use patterns, or proximity to transportation hubs (e.g. central business districts of Downtown Vancouver area vs suburban residential area in Langely)

From a socioeconomic perspective, areas with higher socioeconomic status may have greater resources for crime prevention measures leading to more reported crimes (e.g. West Vancouver), compared to more disadvantaged neighborhoods where crime may be more prevalent but underreported (e.g. Downtown Eastside).

Vancouver being a diverse city with pockets of immigrant cultural hubs and communities, differences in law enforcement practices targeting specific demographic groups can also result in skewed representations of crime patterns and disparities of enforcement outcomes.

To conclude, interpreting GeoDASH information as someone not from Vancouver might be challenging at face value,  as there is a lack of nuanced understanding of the makeup of this city, which requires a lot of contexts to provide a more holistic and accurate interpretation of the presented data. Even for a Vancouverite, the historical background can be an insightful starting point to gain a better understanding of the city that I live in.

Learning Analytics Adventure: Factors that Impact Student Motivation and Completion of MOOCs

Abstract

Due to the massive number of potentially enrolled students, the open-access nature for learners to participate or not participate, and lack of physical space on online platforms,  massive open online courses (MOOCs) embody a particular learning experience independent of time and space (İnan & Ebner, 2020). With the heterogeneity of learners with varying learning motivations, this impacts whether or not learners complete the course. By identifying the characteristics that matter to learners, the learning experience of MOOCs can be improved and prioritized accordingly (Nanda et. al, 2021).

Based on the Exit Survey questions of edX-based UBC MOOC on Climate Change, two open-ended post-course survey questions were analyzed. A frequency analysis was conducted and visualized as a word cloud with learning analytic tool AntConc, revealing that learners found the video content most helpful, whereas the assignments, lack of flexibility in time and deadlines, and unguided peer evaluation were aspects that negatively impacted their learning experience.

Statement Question and Literature Review

Based on Nanda et. al’s (2021) study on analysis of open-ended feedback from MOOC learners, due to the heterogeneity of learners that enroll in MOOCs, it is important to identify the characteristics that are most important to the different learners and to prioritize them accordingly based on qualitative post-course surveys. In their study, they conducted latent Dirichlet allocation (LDA) topic model on 150000 MOOC learners from 810 MOOCs in different subject areas with the following three open-ended questions:

Q1) What was your most favorite part of the course and why?

Q2) What was your least favorite part of the course and why?

Q3) How could the course be improved?

From their qualitative analysis, the researchers identified characteristics that impacted learner’s experience in completion of the MOOCs: the quality of course content, accurate description of prerequisites and required time commitment in course syllabus, quality of assessment and feedback, meaningful interaction with peers/instructors, engaging instructor and videos, accessibility of learning materials and usability of the platforms.

Based on these studies, I would like to conduct a similar review on the edx-based UBC MOOC course on climate change, provide some data visualization and generate some discussion for further development of collection of data, and improvement of the course for future iterations.

Methods and Tools

For the scope of this project, I hope to center exploratory qualitative analysis from the Exit Survey as a starting point to further investigate learner data surrounding student demographics,engagement in course content, engagement with their peers within the course to extrapolate beneficial information and gain some insight as to what are the characteristics and aspects of the course that contribute to learner’s motivation to complete or drop out of the course.

First I familiarized myself with the quantitative and qualitative data from the MOOC, and operationally defined some of the different themes of student behavior and data that I wanted to investigate, namely student motivation, student familiarity (to subject matter), student identity/demographics, student engagement (broken down to four categories) and qualitative feedback.

Table of Themes
Motivation
Climate Entry Survey –       Q2.1 What are the main reasons for taking the course?

–       Q3.1 How many weeks do you plan to engage in the course?

–       Q3.2 How many hours per week on average do you plan to spend on this course?

–       Q3.3 How frequently do you plan to use course components?

–       Q4.2 How many MOOCs have you participated at least partially?

–       Q4.3 Think about the MOOC you were most engaged in, what best describes the level of engagement?

 
Climate Exit Survey

 

 –       Q2.1 Were your goals for taking the course met?

–       Q2.2 Are you likely to…

–       Q3.1 How frequently did you use each of the course components?

–       Q3.3 How many hours per week did you spend in this course?

–       Q3.4 What interactions did you have with course peers?

–       Q4.1 Which of the following components of the course were you very satisfied with?

–       Q4.2 How did you find the following modules?

 
Familiarity (to Subject Matter)
Climate Entry Survey

 

 –       Q.1.1 ~ Q1.3 Climate Knowledge

–       Q4.1 Prior to this course, how familiar were you with the subject matter?
Climate Exit Survey –       Q.1.1 ~1.3 Climate Knowledge
Student Identity/Demographics
Climate Entry Survey

 

 –       Q5.1 Which country were you born in?

–       Q5.2 Which country do you currently reside in?

–       Q5.3 Main languages you speak

–       Q5.4 English proficiency
Student Engagement
Person_course_day_cleaned.tsv General (time spent)

–       Avg_dt

–       Sdv_dt

–       Sum_dt

Video Engagement:

–       Nevents

–       Nplayvideo

–       Ntranscript

–       Nvideos_viewed

–       Nvideos_watched_sec

Forum Engagement

–       Nforum_reads

–       Nforum_posts

–       Nforum_threads

–       Nforum_endorsed

–       Nforum_threads

–       Nforum_comments

Problems

–       Nproblems_attemped

–       Nproblems_answered
Qualitative Feedback
Climate Exit Survey

–       Q2.3 What did you like most about the course?

–       Q2.4 What did you like least about the course?

Qualitative Feedback Analysis

 

I utilized text analysis software AntConc for frequency analysis and visualization via the word cloud function, as it would be the best way to understand the top 50 most frequent words from the text at a glance.

There were 200 responses for the “What did you like most about the course” question, and 216 responses for the “What did you like least about the course” question. To prepare the text for frequency analysis. Due to the smaller scope and sample size, I manually cleaned up the comments by normalizing to consistent lowercase, removing unicode characters and punctuation, removing common stop words, and then proceeded to stemming and lemmatization.

Findings

In the question “What did you like most about the course?”, below are some notable words that stood out.

61 students reflected that the “videos” were helpful to understand the course content. Alongside the word cloud with higher frequency are words with positive sentiment such as “good”, “clear”, “understand” and “well”.

On the other hand, in the question “What did you like least about the course”, below are some notable words that stood out in the word cloud.

In the word cloud, the words “assignment”, “time” and “peer review” were some that I would like to mention. Having gone back and read the comments in detail, I found that they were somewhat related to each other. Many of the students reflected that they struggled with the assignments due to deadlines and personal time constraints, in addition to the peer review component that was required for completion of the assignment.

From the comments on assignments, 4 students shared that they came into the course with less background knowledge on the subject matter of climate change, feeling overwhelmed with the assignment and found “having to write essays on climate science before establishing a knowledge base was more than a little daunting.” There were also comments on how the assignments were rather demanding due to deadlines and personal time constraints, and would prefer “shorter or optional assignments” rather than the current structure.

Lastly, the assignment had a peer review component, in which 18 students voiced their dislike due to various reasons. Some students stated that “the rubrics for peer reviews were limiting and not quite matching the assignment” and had no guidance on how to provide constructive peer feedback, with varying degrees of effort put into peer evaluation such that many were in agreement that perhaps “it is necessary for staff review”.

Discussion and Future Development

One main limitation of this exploratory analysis of the qualitative analysis is only being able to conduct it after the course has ended and the surveys have been taken, and not in real-time as the course was in progress.

Aligning with Eriksson et. al’s (2017) identified three main factors influencing MOOC dropouts, which included mismatch between learner’s perception and actual course content and design; learner’s ability to manage time; and social aspects of learner community feeling.

Based on the findings of the frequency analysis as a starting point, I hope to discuss further potential investigation of data for further improvement and development of the MOOC course for future iterations based on some of the themes from the course survey comments.

Course Component, Learning Materials and Resources

Besides the positive sentiment towards utilization of videos in course delivery, I think it would be interesting to look further into the general utilization of course components, learning materials and resources. Based on the self-reported utilization of course components in the entry and exit survey, in comparison to the actual generated data of course component utilization, it would be valuable to investigate the difference to identify student expectations of what resources they think they will use versus the actual utilization of available resources, and whether there needs to be more development of certain materials for future iterations of the MOOC.

Peer Interactions and Peer Evaluations

In continuation of the findings for the negative aspects of the course, I think it would also be useful to look into the self-reported interaction with peers, and the actual generated data of peer interactions. With the utilization of Social Network Analysis, utilization of NodeXL for peer interactions can provide more insight on the socio-constructivist potential of MOOCs, Building rapport with the peers might also ease part of the negative comments on peer reviews, in addition to providing clearer rubric and guidelines for providing peer reviews, as  “peer assessment techniques and exploiting peer support can revolutionize emergence of new pedagogical models in the MOOC approaches” (Yuan, Powell & Cetis, 2013).

Prior Experience with MOOCs and Time Management

In terms of students reflecting that there was lack of flexibility and deadlines, in addition to their personal time management and constraints, I believe Time Management should be a theme that I would further add to the Table of Themes. Some of the relevant questions that would help operational define this theme currently overlap with those in the Entry Survey for Motivation, and additional questions, such as whether or not they have full-time/part-time occupations, can be a proxy measure to how much time students may realistically have.

However, I believe some of these questions also overlap with another potential theme, namely Familiarity (with MOOCs) in terms of whether or not students have had previous pedagogical experience with MOOCs, assuming that those with previous experiences have a better idea on delegation of time and other metacognitive executive functioning. Having additional survey questions that target these themes might provide more points of data collection for generating a more well-rounded idea of the students motivations and capabilities.

Personal Reflection

One big challenge that I was faced with was learning to familiarize myself with the massive quantities of data that was generated from the MOOC course, and figure out ways to operationalize it in order to target what I wanted to investigate.

Another major challenge was the learning curve to learn how to use the different Learning Analytic Tools that were available. For Tableau, I had to go through their tutorial videos to understand how to input the data with each other to create the visualizations that I wanted. For AntConc, the tool itself was more straightforward due to the smaller scope of functions and settings, however, learning to format and prepare the comments for analysis was something new that I learned about. The dataset was manageable in terms of manually preparing for it, though should the scope increase, proper Python programming would be beneficial to complete the task. In retrospect, I believe NodeXL might have been a more robust text-analysis and social network analysis software to utilize for this project. Ideally, I would have liked to have more time to understand the data and the functions of the application to create more substantial visualizations that could provide more insight to the social interactions of students in the courses, especially when it comes to the socio-constructivist potential of crowdsourcing peer reviews and evaluation in MOOCs.

References and Literature

İnan, E., Ebner, M. (2020). Learning Analytics and MOOCs. In: Zaphiris, P., Ioannou, A. (eds) Learning and Collaboration Technologies. Designing, Developing and Deploying Learning Experiences. HCII 2020. Lecture Notes in Computer Science(), vol 12205. Springer, Cham. https://doi.org/10.1007/978-3-030-50513-4_18

Eriksson, T., Adawi, T., & Stöhr, C. (2017). “Time is the bottleneck”: A qualitative study exploring why learners drop out of MOOCs. Journal of Computing in Higher Education, 29(1), 133-146. https://doi.org/10.1007/s12528-016-9127-8

Khalil, Mohammad & Ebner, Martin. (2017). Driving Student Motivation in MOOCs through a Conceptual Activity-Motivation Framework. Zeitschrift für Hochschulentwicklung. 12. 101-122. 10.3217/zfhe-12-01/06.

Nanda, G., A. Douglas, K., R. Waller, D., E. Merzdorf, H., & Goldwasser, D. (2021). Analyzing large collections of open-ended feedback from MOOC learners using LDA topic modeling and qualitative analysis. IEEE Transactions on Learning Technologies, 14(2), 146-160. https://doi.org/10.1109/TLT.2021.3064798

Nawrot, I., & Doucet, A. (2014). Building engagement for MOOC students: Introducing support for time management on online learning platforms. Paper presented at the 1077-1082. https://doi.org/10.1145/2567948.2580054

Zhu, M., Sari, A.R. & Lee, M.M. Trends and Issues in MOOC Learning Analytics Empirical Research: A Systematic Literature Review (2011–2021). Educ Inf Technol 27, 10135–10160 (2022). https://doi.org/10.1007/s10639-022-11031-6

Evaluation of Learning Analytics Tool

Evaluation Tool and Motivation

For this assignment, I have decided to choose OnTask as the learning analytics tool of choice for analysis. I want to look into the ways that this tool goes above and beyond monitoring student behavior patterns for “early alert” by providing real-time feedback for students and suggestions for further improvement by recommending additional resources for reference. In addition, as it is not limited to one LMS system, it is a potentially useful add-on for a variety of contexts. I hope to reflect on the ways that this tool would be especially helpful in the context of the UBC OnTask Pilot Study for first-year physics course as an example.

I chose to utilize the Cooper Framework (Cooper, 2012)  as the guide to evaluate this learning analytics tool, as it takes on a more “descriptive, rather definitive approach which allows to deal with real-world complexity of how analytics is” (p.3). I believe this framework refers to “characteristics of learning analytic tools which may overlap and are assumed to be extensible and adaptable” (p. 3), which I believe is important when it comes to benefiting from the affordances of the tools, while thinking about best practices for utilization in different educational contexts.

OnTask Tool Functions (OnTask, n.a)

OnTask Project “aims to improve the academic experience of students through the delivery of timely, personalized and actionable student feedback throughout their participation in a course”. Some core functions of OnTask include:

  • Assessment of data about student activities throughout the semester and allows instructor to design personalized feedback with suggestions about learning strategies for students to adjust their learning progressively
  • LMS- agnostic and receives data from various sources (i.e. online engagement, assessments, student information systems, electronic textbooks, discussion forums,etc.)
  • Directs students to specific chapters/examples in textbooks, suggest additional readings/resources, enroll in required workshops/tutorials/labs, redirects to university support services

Cooper’s Framework – UBC OnTask Pilot Study

Based on UBC’s OnTask Pilot Project (Moosvi, 2019), the tool was used in first-year physics course PHYS1117 by instructor Simon Bates and co-instructor Mateus Fandino to engage with students more and add personalized feedback for each student based on their performance throughout the weeks. Using Cooper’s proposed framework (Cooper, 2012), I have analyzed this case study as below:

Analysis subjects, objects, and clients Analysis Subjects: first-year students in PHYS1117

 

Analysis Objects: first-year students in PHYS1117, specifically student performance within the course

 

Analysis Clients: course instructors
Data Origin ●     Data collected from LMS system Canvas (i.e. assignments, exams and assessments, student engagement data, etc.)

 

●     In-class attendance and participation

 

●     Laboratory sections
Orientation and Objectives Orientation:

Diagnostic in nature even though the pilot project is an exploratory one, as it aims to investigate whether personalized feedback can  improve student performance.

 

This study is also a reflective orientation, although the natural extension of the pilot project could be used to predict outcomes of future cohorts, in which a more diagnostic mode is expected.

 

Objective: to enhance student performance and outcomes based on mix of quantitative and qualitative measures
Technical Approach Assumption of technical approach being statistical and hypothesis testing in nature based on analysis of students’ previous course performance without personalized feedback from OnTask compared to students’ course performance with the personalized feedback from OnTask.
Embedded Theories Socio-constructivist approaches for students to co-construct their learning process via receiving and implementing feedback given by instructors

 

Based on some of the feedback from students, there was a general positive perception from students for utilizing OnTask to provide feedback on what to focus on readings, reminders for upcoming assessments and deadlines, areas of confusion and commonly made mistakes, reflection of previous and future coursework, as well as helping students gauge their own progress in the course. There was also an increase of student engagement and interaction overall due to the weekly personalized newsletters that the instructors sent out.

Reflection

The pilot study results are in alignment with the findings of Pardo et. al (2019) on a first-year undergraduate engineering course. Although the size effect of personalized feedback was not large on the midterm scores, the result still provided a significant and positive impact on student satisfaction with feedback and academic performance in the midterm exam (p. 136). The researchers associated the feedback as an important factor to support student success, and highlighted the need to establish a connection between student data and how to utilize it to provide high quality feedback. Suggestions on creating comment templates to tailor to a course, or for algorithms to match between comments and observed data were provided.

Some further research on better techniques to identify individual differences in student participation in learning experiences and students study habits were highlighted, as these missing pieces of information could potentially provide additional information for interventions with potentially high effects.

In terms of applying it to my personal context of foreign language teaching and learning, due to the smaller scale of students per class and the increased amount of qualitative assessments, there might not be as big of a difference between utilizing OnTask versus the instructor providing feedback directly. However, I still believe OnTask could be a great tool to identify common challenges that students might be facing based on their qualitative assignments, and can still provide insight on student performance in courses regardless.

References

Cooper, A. (2012) CETIS Analytics Series Volume 1, No 7; A Framework of Characteristics for Analytics http://publications.cetis.ac.uk/2012/524

Lim, L., Gentili, S., Pardo, A., Dawson, S., & Gašević, D. (2018). Combining technology and human intelligence to provide feedback and learning support using OnTask . In Companion Proceedings of the 8th International Conference on Learning Analytics & Knowledge (LAK’18), Sydney, Australia. Academia.

Moosvi, Firas (2019, June 20). Learning Analytics Project OnTask: A Case Study https://learninganalytics.ubc.ca/ontask-a-case-study/

OnTask (n.a.). https://www.ontasklearning.org/

Pardo, A., Jovanovic, J., Dawson, S., Gašević, D., & Mirriahi, N. (2017). Using learning analytics to scale the provision of personalised feedback. British Journal of Educational Technology, doi:10.1111/bjet.12592

 

Spam prevention powered by Akismet