Tzu Hsu Chu – Masters in Educational Technology at the University of British Columbia

The Maker In Me

Brief

Introduce yourself as a maker and reflect on your own professional philosophy and how you view yourself as a maker, as a critical and innovative thinker, and as a designer. You will align your own personal and professional philosophy for your learners within a maker environment. Your introduction will include:

- A visual representation of your values, approach to making, and educational/leadership philosophy inspired by the course readings
- Investigate multimodal means to design a creative and critical introduction
- Include your goals for this course within your introduction

Some guiding questions:

- How do you describe yourself as a maker, and your maker philosophy?
- What is your current educational/leadership philosophy?
- How does your philosophy fit into the maker movement?
- What is your experience with Makerspaces?
- What is your experience with EDIDA Frameworks?
- Why are you taking this course? What are your expectations and goals?

GeoDASH: Exploring Predictive Policing Technology

As someone living in Vancouver for the past 9 years, the first thing I did when I saw the map was zoom into my neighborhood to see what has been happening in my surrounding areas. In addition, I also looked around the locations that I frequent.

After tinkering with the map, I decided to look read the GeoDASH FAQ page for more details on how the data was collected and visualized on the map. Overall, due to the sensitive nature and need to protect the privacy of those involved in the incidents, it seems like most of the information displayed can only be considered as proxy measures, as most of the actual locations have been rounded to the approximate block level.

With the lack of transparency in terms of how information is being collected, reported, and mapped, it is rather challenging to assess the validity of information presented on GeoDASH at face value without cross-referencing other sources, to gain a more comprehensive understanding of the crime trends in different neighborhoods.

Some inherent biases that may appear when interpreting the data on the map are intersectional in nature— using GeoDASH without additional understanding of geographic, socioeconomic, and demographic characteristics of Vancouver may result in misinterpretation of crime trends.

For example, geographically speaking, certain areas may have higher concentrations of crime due to population density, land-use patterns, or proximity to transportation hubs (e.g. central business districts of Downtown Vancouver area vs suburban residential area in Langely)

From a socioeconomic perspective, areas with higher socioeconomic status may have greater resources for crime prevention measures leading to more reported crimes (e.g. West Vancouver), compared to more disadvantaged neighborhoods where crime may be more prevalent but underreported (e.g. Downtown Eastside).

Vancouver being a diverse city with pockets of immigrant cultural hubs and communities, differences in law enforcement practices targeting specific demographic groups can also result in skewed representations of crime patterns and disparities of enforcement outcomes.

To conclude, interpreting GeoDASH information as someone not from Vancouver might be challenging at face value, as there is a lack of nuanced understanding of the makeup of this city, which requires a lot of contexts to provide a more holistic and accurate interpretation of the presented data. Even for a Vancouverite, the historical background can be an insightful starting point to gain a better understanding of the city that I live in.

Learning Analytics Adventure: Factors that Impact Student Motivation and Completion of MOOCs

Abstract

Due to the massive number of potentially enrolled students, the open-access nature for learners to participate or not participate, and lack of physical space on online platforms, massive open online courses (MOOCs) embody a particular learning experience independent of time and space (İnan & Ebner, 2020). With the heterogeneity of learners with varying learning motivations, this impacts whether or not learners complete the course. By identifying the characteristics that matter to learners, the learning experience of MOOCs can be improved and prioritized accordingly (Nanda et. al, 2021).

Based on the Exit Survey questions of edX-based UBC MOOC on Climate Change, two open-ended post-course survey questions were analyzed. A frequency analysis was conducted and visualized as a word cloud with learning analytic tool AntConc, revealing that learners found the video content most helpful, whereas the assignments, lack of flexibility in time and deadlines, and unguided peer evaluation were aspects that negatively impacted their learning experience.

Statement Question and Literature Review

Based on Nanda et. al’s (2021) study on analysis of open-ended feedback from MOOC learners, due to the heterogeneity of learners that enroll in MOOCs, it is important to identify the characteristics that are most important to the different learners and to prioritize them accordingly based on qualitative post-course surveys. In their study, they conducted latent Dirichlet allocation (LDA) topic model on 150000 MOOC learners from 810 MOOCs in different subject areas with the following three open-ended questions:

Q1) What was your most favorite part of the course and why?

Q2) What was your least favorite part of the course and why?

Q3) How could the course be improved?

From their qualitative analysis, the researchers identified characteristics that impacted learner’s experience in completion of the MOOCs: the quality of course content, accurate description of prerequisites and required time commitment in course syllabus, quality of assessment and feedback, meaningful interaction with peers/instructors, engaging instructor and videos, accessibility of learning materials and usability of the platforms.

Based on these studies, I would like to conduct a similar review on the edx-based UBC MOOC course on climate change, provide some data visualization and generate some discussion for further development of collection of data, and improvement of the course for future iterations.

Methods and Tools

For the scope of this project, I hope to center exploratory qualitative analysis from the Exit Survey as a starting point to further investigate learner data surrounding student demographics,engagement in course content, engagement with their peers within the course to extrapolate beneficial information and gain some insight as to what are the characteristics and aspects of the course that contribute to learner’s motivation to complete or drop out of the course.

First I familiarized myself with the quantitative and qualitative data from the MOOC, and operationally defined some of the different themes of student behavior and data that I wanted to investigate, namely student motivation, student familiarity (to subject matter), student identity/demographics, student engagement (broken down to four categories) and qualitative feedback.

Table of Themes
Motivation
Climate Entry Survey	– Q2.1 What are the main reasons for taking the course? – Q3.1 How many weeks do you plan to engage in the course? – Q3.2 How many hours per week on average do you plan to spend on this course? – Q3.3 How frequently do you plan to use course components? – Q4.2 How many MOOCs have you participated at least partially? – Q4.3 Think about the MOOC you were most engaged in, what best describes the level of engagement?
Climate Exit Survey	– Q2.1 Were your goals for taking the course met? – Q2.2 Are you likely to… – Q3.1 How frequently did you use each of the course components? – Q3.3 How many hours per week did you spend in this course? – Q3.4 What interactions did you have with course peers? – Q4.1 Which of the following components of the course were you very satisfied with? – Q4.2 How did you find the following modules?
Familiarity (to Subject Matter)
Climate Entry Survey	– Q.1.1 ~ Q1.3 Climate Knowledge – Q4.1 Prior to this course, how familiar were you with the subject matter?
Climate Exit Survey	– Q.1.1 ~1.3 Climate Knowledge
Student Identity/Demographics
Climate Entry Survey	– Q5.1 Which country were you born in? – Q5.2 Which country do you currently reside in? – Q5.3 Main languages you speak – Q5.4 English proficiency
Student Engagement
Person_course_day_cleaned.tsv	General (time spent) – Avg_dt – Sdv_dt – Sum_dt Video Engagement: – Nevents – Nplayvideo – Ntranscript – Nvideos_viewed – Nvideos_watched_sec Forum Engagement – Nforum_reads – Nforum_posts – Nforum_threads – Nforum_endorsed – Nforum_threads – Nforum_comments Problems – Nproblems_attemped – Nproblems_answered
Qualitative Feedback
Climate Exit Survey – Q2.3 What did you like most about the course? – Q2.4 What did you like least about the course?

Qualitative Feedback Analysis

I utilized text analysis software AntConc for frequency analysis and visualization via the word cloud function, as it would be the best way to understand the top 50 most frequent words from the text at a glance.

There were 200 responses for the “What did you like most about the course” question, and 216 responses for the “What did you like least about the course” question. To prepare the text for frequency analysis. Due to the smaller scope and sample size, I manually cleaned up the comments by normalizing to consistent lowercase, removing unicode characters and punctuation, removing common stop words, and then proceeded to stemming and lemmatization.

Findings

In the question “What did you like most about the course?”, below are some notable words that stood out.

61 students reflected that the “videos” were helpful to understand the course content. Alongside the word cloud with higher frequency are words with positive sentiment such as “good”, “clear”, “understand” and “well”.

On the other hand, in the question “What did you like least about the course”, below are some notable words that stood out in the word cloud.

In the word cloud, the words “assignment”, “time” and “peer review” were some that I would like to mention. Having gone back and read the comments in detail, I found that they were somewhat related to each other. Many of the students reflected that they struggled with the assignments due to deadlines and personal time constraints, in addition to the peer review component that was required for completion of the assignment.

From the comments on assignments, 4 students shared that they came into the course with less background knowledge on the subject matter of climate change, feeling overwhelmed with the assignment and found “having to write essays on climate science before establishing a knowledge base was more than a little daunting.” There were also comments on how the assignments were rather demanding due to deadlines and personal time constraints, and would prefer “shorter or optional assignments” rather than the current structure.

Lastly, the assignment had a peer review component, in which 18 students voiced their dislike due to various reasons. Some students stated that “the rubrics for peer reviews were limiting and not quite matching the assignment” and had no guidance on how to provide constructive peer feedback, with varying degrees of effort put into peer evaluation such that many were in agreement that perhaps “it is necessary for staff review”.

Discussion and Future Development

One main limitation of this exploratory analysis of the qualitative analysis is only being able to conduct it after the course has ended and the surveys have been taken, and not in real-time as the course was in progress.

Aligning with Eriksson et. al’s (2017) identified three main factors influencing MOOC dropouts, which included mismatch between learner’s perception and actual course content and design; learner’s ability to manage time; and social aspects of learner community feeling.

Based on the findings of the frequency analysis as a starting point, I hope to discuss further potential investigation of data for further improvement and development of the MOOC course for future iterations based on some of the themes from the course survey comments.

Course Component, Learning Materials and Resources

Besides the positive sentiment towards utilization of videos in course delivery, I think it would be interesting to look further into the general utilization of course components, learning materials and resources. Based on the self-reported utilization of course components in the entry and exit survey, in comparison to the actual generated data of course component utilization, it would be valuable to investigate the difference to identify student expectations of what resources they think they will use versus the actual utilization of available resources, and whether there needs to be more development of certain materials for future iterations of the MOOC.

Peer Interactions and Peer Evaluations

In continuation of the findings for the negative aspects of the course, I think it would also be useful to look into the self-reported interaction with peers, and the actual generated data of peer interactions. With the utilization of Social Network Analysis, utilization of NodeXL for peer interactions can provide more insight on the socio-constructivist potential of MOOCs, Building rapport with the peers might also ease part of the negative comments on peer reviews, in addition to providing clearer rubric and guidelines for providing peer reviews, as “peer assessment techniques and exploiting peer support can revolutionize emergence of new pedagogical models in the MOOC approaches” (Yuan, Powell & Cetis, 2013).

Prior Experience with MOOCs and Time Management

In terms of students reflecting that there was lack of flexibility and deadlines, in addition to their personal time management and constraints, I believe Time Management should be a theme that I would further add to the Table of Themes. Some of the relevant questions that would help operational define this theme currently overlap with those in the Entry Survey for Motivation, and additional questions, such as whether or not they have full-time/part-time occupations, can be a proxy measure to how much time students may realistically have.

However, I believe some of these questions also overlap with another potential theme, namely Familiarity (with MOOCs) in terms of whether or not students have had previous pedagogical experience with MOOCs, assuming that those with previous experiences have a better idea on delegation of time and other metacognitive executive functioning. Having additional survey questions that target these themes might provide more points of data collection for generating a more well-rounded idea of the students motivations and capabilities.

Personal Reflection

One big challenge that I was faced with was learning to familiarize myself with the massive quantities of data that was generated from the MOOC course, and figure out ways to operationalize it in order to target what I wanted to investigate.

Another major challenge was the learning curve to learn how to use the different Learning Analytic Tools that were available. For Tableau, I had to go through their tutorial videos to understand how to input the data with each other to create the visualizations that I wanted. For AntConc, the tool itself was more straightforward due to the smaller scope of functions and settings, however, learning to format and prepare the comments for analysis was something new that I learned about. The dataset was manageable in terms of manually preparing for it, though should the scope increase, proper Python programming would be beneficial to complete the task. In retrospect, I believe NodeXL might have been a more robust text-analysis and social network analysis software to utilize for this project. Ideally, I would have liked to have more time to understand the data and the functions of the application to create more substantial visualizations that could provide more insight to the social interactions of students in the courses, especially when it comes to the socio-constructivist potential of crowdsourcing peer reviews and evaluation in MOOCs.

References and Literature

İnan, E., Ebner, M. (2020). Learning Analytics and MOOCs. In: Zaphiris, P., Ioannou, A. (eds) Learning and Collaboration Technologies. Designing, Developing and Deploying Learning Experiences. HCII 2020. Lecture Notes in Computer Science(), vol 12205. Springer, Cham. https://doi.org/10.1007/978-3-030-50513-4_18

Eriksson, T., Adawi, T., & Stöhr, C. (2017). “Time is the bottleneck”: A qualitative study exploring why learners drop out of MOOCs. Journal of Computing in Higher Education, 29(1), 133-146. https://doi.org/10.1007/s12528-016-9127-8

Khalil, Mohammad & Ebner, Martin. (2017). Driving Student Motivation in MOOCs through a Conceptual Activity-Motivation Framework. Zeitschrift für Hochschulentwicklung. 12. 101-122. 10.3217/zfhe-12-01/06.

Nanda, G., A. Douglas, K., R. Waller, D., E. Merzdorf, H., & Goldwasser, D. (2021). Analyzing large collections of open-ended feedback from MOOC learners using LDA topic modeling and qualitative analysis. IEEE Transactions on Learning Technologies, 14(2), 146-160. https://doi.org/10.1109/TLT.2021.3064798

Nawrot, I., & Doucet, A. (2014). Building engagement for MOOC students: Introducing support for time management on online learning platforms. Paper presented at the 1077-1082. https://doi.org/10.1145/2567948.2580054

Zhu, M., Sari, A.R. & Lee, M.M. Trends and Issues in MOOC Learning Analytics Empirical Research: A Systematic Literature Review (2011–2021). Educ Inf Technol 27, 10135–10160 (2022). https://doi.org/10.1007/s10639-022-11031-6

Evaluation of Learning Analytics Tool

Evaluation Tool and Motivation

For this assignment, I have decided to choose OnTask as the learning analytics tool of choice for analysis. I want to look into the ways that this tool goes above and beyond monitoring student behavior patterns for “early alert” by providing real-time feedback for students and suggestions for further improvement by recommending additional resources for reference. In addition, as it is not limited to one LMS system, it is a potentially useful add-on for a variety of contexts. I hope to reflect on the ways that this tool would be especially helpful in the context of the UBC OnTask Pilot Study for first-year physics course as an example.

I chose to utilize the Cooper Framework (Cooper, 2012) as the guide to evaluate this learning analytics tool, as it takes on a more “descriptive, rather definitive approach which allows to deal with real-world complexity of how analytics is” (p.3). I believe this framework refers to “characteristics of learning analytic tools which may overlap and are assumed to be extensible and adaptable” (p. 3), which I believe is important when it comes to benefiting from the affordances of the tools, while thinking about best practices for utilization in different educational contexts.

OnTask Tool Functions (OnTask, n.a)

OnTask Project “aims to improve the academic experience of students through the delivery of timely, personalized and actionable student feedback throughout their participation in a course”. Some core functions of OnTask include:

Assessment of data about student activities throughout the semester and allows instructor to design personalized feedback with suggestions about learning strategies for students to adjust their learning progressively
LMS- agnostic and receives data from various sources (i.e. online engagement, assessments, student information systems, electronic textbooks, discussion forums,etc.)
Directs students to specific chapters/examples in textbooks, suggest additional readings/resources, enroll in required workshops/tutorials/labs, redirects to university support services

Cooper’s Framework – UBC OnTask Pilot Study

Based on UBC’s OnTask Pilot Project (Moosvi, 2019), the tool was used in first-year physics course PHYS1117 by instructor Simon Bates and co-instructor Mateus Fandino to engage with students more and add personalized feedback for each student based on their performance throughout the weeks. Using Cooper’s proposed framework (Cooper, 2012), I have analyzed this case study as below:

Analysis subjects, objects, and clients	Analysis Subjects: first-year students in PHYS1117 Analysis Objects: first-year students in PHYS1117, specifically student performance within the course Analysis Clients: course instructors
Data Origin	● Data collected from LMS system Canvas (i.e. assignments, exams and assessments, student engagement data, etc.) ● In-class attendance and participation ● Laboratory sections
Orientation and Objectives	Orientation: Diagnostic in nature even though the pilot project is an exploratory one, as it aims to investigate whether personalized feedback can improve student performance. This study is also a reflective orientation, although the natural extension of the pilot project could be used to predict outcomes of future cohorts, in which a more diagnostic mode is expected. Objective: to enhance student performance and outcomes based on mix of quantitative and qualitative measures
Technical Approach	Assumption of technical approach being statistical and hypothesis testing in nature based on analysis of students’ previous course performance without personalized feedback from OnTask compared to students’ course performance with the personalized feedback from OnTask.
Embedded Theories	Socio-constructivist approaches for students to co-construct their learning process via receiving and implementing feedback given by instructors

Based on some of the feedback from students, there was a general positive perception from students for utilizing OnTask to provide feedback on what to focus on readings, reminders for upcoming assessments and deadlines, areas of confusion and commonly made mistakes, reflection of previous and future coursework, as well as helping students gauge their own progress in the course. There was also an increase of student engagement and interaction overall due to the weekly personalized newsletters that the instructors sent out.

Reflection

The pilot study results are in alignment with the findings of Pardo et. al (2019) on a first-year undergraduate engineering course. Although the size effect of personalized feedback was not large on the midterm scores, the result still provided a significant and positive impact on student satisfaction with feedback and academic performance in the midterm exam (p. 136). The researchers associated the feedback as an important factor to support student success, and highlighted the need to establish a connection between student data and how to utilize it to provide high quality feedback. Suggestions on creating comment templates to tailor to a course, or for algorithms to match between comments and observed data were provided.

Some further research on better techniques to identify individual differences in student participation in learning experiences and students study habits were highlighted, as these missing pieces of information could potentially provide additional information for interventions with potentially high effects.

In terms of applying it to my personal context of foreign language teaching and learning, due to the smaller scale of students per class and the increased amount of qualitative assessments, there might not be as big of a difference between utilizing OnTask versus the instructor providing feedback directly. However, I still believe OnTask could be a great tool to identify common challenges that students might be facing based on their qualitative assignments, and can still provide insight on student performance in courses regardless.

References

Cooper, A. (2012) CETIS Analytics Series Volume 1, No 7; A Framework of Characteristics for Analytics http://publications.cetis.ac.uk/2012/524

Lim, L., Gentili, S., Pardo, A., Dawson, S., & Gašević, D. (2018). Combining technology and human intelligence to provide feedback and learning support using OnTask . In Companion Proceedings of the 8th International Conference on Learning Analytics & Knowledge (LAK’18), Sydney, Australia. Academia.

Moosvi, Firas (2019, June 20). Learning Analytics Project OnTask: A Case Study https://learninganalytics.ubc.ca/ontask-a-case-study/

OnTask (n.a.). https://www.ontasklearning.org/

Pardo, A., Jovanovic, J., Dawson, S., Gašević, D., & Mirriahi, N. (2017). Using learning analytics to scale the provision of personalised feedback. British Journal of Educational Technology, doi:10.1111/bjet.12592

ELIZA: Exploring Early Artificial Intelligence

Brief

Approach the interaction as both a user and a critic.

Formulate 2-3 hypotheses to test and 3-5 questions to address before testing the limitations of the ELZIA software by engaging in conversation. Pay attention to the chatbot’s language processing and quality of response.

Annotate the transcript of the interaction with observations and feelings. In your reflection, discuss the ethical and practical limitations of the technology. Save the transcript of your interaction to submit with your work.

Hypotheses

Based on the old-fashioned interface of the chatbot, I immediately dialed back my expectations.

I presume that this is a very “user-driven” conversational chatbot, such that it will echo or rephrase what the user has said to help with reflection and further articulation of what the user has offered as a starting point in conversation.

Given that it uses “pattern matching” to give canned responses, I believe this chatbot will not be able to provide a wide range of conversational breadth and will be relying on pre-programmed scripts or more ‘generic therapeutic’ prompts.

Following the Rogerian psychotherapeutic methods of “returning the question back to the patient”, I believe that the conversations with the chatbot can get rather cyclical and stagnant, which may not be user-friendly and perhaps frustrating in terms of usability.

Questions

When it comes to human-computer interaction, I always think of the concept of the ‘uncanny valley’, originally introduced by Japanese robotics engineering professor Masahiro Mori (Kageki, 2012). It is in the uncanny valley where the emotional response of a human observer becomes increasingly negative towards a robot as it approximates the ‘almost but not quite there’ likeness of a human yet doesn’t quite reach full human likeness, especially unnatural appearances or interactions that deviate from normal human behaviors can elicit a discomfort and eeriness.

At the same time, there is the ELIZA effect, coined after this chatbot, in which humans have the tendency to project human traits of semantic comprehension, or even empathy into computer programs that have a textual interface (Weizenbaum, 1976)

Given these two phenomena, how do these two affect each other? Will the ELIZA effect bridge the valley such the uncanniness becomes overlooked? Will repeated interaction with a chatbot reveal its limitations such that it widens the valley even more?

How is it the case that Weizenbaum’s students exhibited strong emotional connections when they were texting with the ELIZA? Is it because they were not accustomed to this kind of interaction?

Now that texting and chatting online is so ubiquitous in our daily lives, is it just that we have become more sensitive and accustomed to texting etiquette and nuances such that we are able to distinguish texting patterns such that we are not overcome by the ELIZA effect?

Conversation with ELIZA

How to be a good friend?

	Conversation	Thoughts/Feelings
	How to be a good friend?
Eliza	Why do you ask?
	I want to be a good friend to my friends
Eliza	I sometimes also want to be a good friend to your friends.	Found it interesting that ELIZA made a self-referential comment, though it seems like it is having a hard time with (reflexive) pronouns
	Do you have friends?
Eliza	We were discussing you, not me.	This response feels a little jarring and perhaps overly blunt?
	What are some ideas to be a good friend?
Eliza	What is it that you really want to know?
	Do my friends like me?
Eliza	Would you say that you have psychological problems?	Felt a bit blind sighted by this remark. Unclear about the relevance to my previous question.
	Not many.
Eliza	Ok…”What am some ideas to be a good friend”. Tell me more.
	I think I am a good listener for my friends.
Eliza	Do you enjoy being a good listener for your friends?
	Sometimes. Sometimes I wish they could listen to me too.
Eliza	Would you say that you have psychological problems?	Feels a little bit discouraging to be asked the second time, unsure how to react to this, hence ending the conversation here

How to be happier?

	Conversation	Thoughts/Feelings
	How to be happier?
Eliza	Have you asked anyone else?
	Yes, my family and friends.
Eliza	Are you sure about that?	I assume this is ELIZA prompting me to dig deeper, but feels like it is questioning my information source and validity
	And my roommates.
Eliza	Can you elaborate on that?	Seems like an empty canned response, as it is unspecific and seems a little off-topic to my previous response
	On my roommates?
Eliza	That is an interesting response. Let me make a note of that for next time.	Unsure how to respond to this and therefore I stopped the conversation

I Have Been Feeling A Little Sad Recently

	Conversation	Thoughts/Feelings
	I have been feeling a little sad recently
Eliza	Is it normal to feel a little sad recently?
	Not sure, but I don’t feel good.
Eliza	Why don’t you feel good?
	I feel stressed about my relationship with my family
Eliza	Do you often feel stressed out about your relationship with your family?
	Yes, quite often actually
Eliza	Are you sure about that?	Feels a little stand-offish to be questioned about my certainty on how I feel, unsure if this is to prompt for further introspection
	Yes, I am sure.
Eliza	Did you come to me because you are sure?	Unclear why ELIZA would want to focus on my certainty my feelings rather than *how* I feel
	Not exactly, but I think it would be helpful to talk to someone about it.
Eliza	Do you really think so?	Could be a prompt to help me further examine whether my thoughts/feelings are true, even though it feels somewhat blunt and insensitive
	Yes, I don’t think it is the best keeping everything bottled up.
Eliza	Do you wish to be able to think it is the best keeping everything bottled up?	This is an interesting question, I believe this is where the user “can see through the code” of this canned response and how they extract certain keywords to put into pre-made sentences, due to the negative phrasing of my previous response, it seems ‘out of place’ to ask such a question
	No, I do not wish to think that way.
Eliza	You are being a bit negative and that won’t help us move forward.	Am a little bit surprised about how blunt ELIZA can be with its tone!

How to stop procrastinating?

	Conversation	Thoughts/Feelings
	How to stop procrastinating?
Eliza	Why do you ask?
	I would like to stop putting off doing certain things, especially when I have a deadline for it
Eliza	Does that question interest you?	Not sure why the rhetorical question?
	Yes
Eliza	Are you sure about that?	Feeling like second guessing myself
	Yes
Eliza	Let’s take a moment here. Think about what you just said and try to rephrase it a bit more specifically.	Not sure what the connections are to the previous two Yes/No question prompts
	I want to be more proactive and take action
Eliza	What would it mean to you if you got to be more proactive and take action
	I feel like I have more agency and self control with allocating my time and energy
Eliza	Which feelings? Tell me more…
	About how I feel when I am in control?	Not the best at providing clarification
Eliza	Do you enjoy feeling when you are in control?
	Sometimes, I enjoy knowing that I can make decisions and follow through with them
Eliza	I think you are blocking what you really want to say
	I like the feeling of having agency
Eliza	What does that suggest to you?
	That I can have an affect on my surroundings and change
Eliza	I am not sure I understand you fully.	I am assuming this is to prompt the user to expand and elaborate a little more on how they feel
	As in I can take action and my actions will move things forward to gain momentum.
Eliza	I see. Let’s try another topic and we will come back to that issue later	I have decided to see what happens when I continue to push through the conversation and change to a different topic
	Sure, I have a hard time making decisions.
Eliza	What does that suggest to you?
	I am indecisive.
Eliza	Do you believe it is normal to be indecisive?
	Yes, it is normal sometimes.
Eliza	I understand.	Unsure how to respond to this and therefore I stopped the conversation

Reflection (~500 words)

After chatting with ELIZA rather earnestly, there were instances when ELIZA’s prompts and responses felt like they landed within an acceptable range of ‘more or less’ human responses, yet still somehow to manage to fall short in other responses. It created a feeling of unease and perhaps mild discomfort, as if this chatbot is trying to understand, yet is unable to comprehend what I am trying to express. Sometimes ELIZA’s tone can seem overly blunt and seems to be missing the little nuances in human interactions that can elicit a stronger sense of empathy and understanding. For example, providing validation before asking to follow-up questions; paraphrasing and summarizing to show ‘active listening’ and engagement; actually circling back to topics that it has put on pause; asking how the different topics connect. Not to mention is has very limited ability to adapt to unexpected inputs or more complex discussions that are not provided in its programmed script. My experience of utilizing ELIZA was very much aligned with my initial hypotheses.

With the inconsistencies and errors that ELIZA showcased the longer I talked to it, I felt myself gradually losing more and more trust in the authenticity and reliability of its responses as I slowly felt like I could ‘see through the code’ of how this chatbot functions, gradually feeling less earnest and genuine with my responses. If this is the case, how is it that early users of ELIZA still attributed human-like feelings to the computer program despite it being (so blatantly, in my opinion) in the uncanny valley?

Perhaps if the chatbot stuck to more objective factual matters, the creepiness of the uncanny valley might not affect user’s attitudes compared to chatbots that are move involved in subjective emotional matters; in other words, I wonder if the uncanny valley effect feels more amplified towards chatbots like ELIZA, especially to those who might be in vulnerable mental states and seeking genuine emotional connections?

With the improvement and further development of technology, chatbots have improved in quality with more robust programming for speech and text recognition, expanding beyond textual interactions. The gap between the uncanny valley will slowly narrow, how will we then distinguish between the human and non-human?

References

Kageki, N. (2012, June 12). An Uncanny Mind: Masahiro Mori on the Uncanny Valley and Beyond. IEEE Spectrum. Retrieved from https://spectrum.ieee.org/an-uncanny-mind-masahiro-mori-on-the-uncanny-valley-and-beyond

Ronkowski, R. (n.d.). Eliza – Computer Therapist. Retrieved from https://web.njit.edu/~ronkowit/eliza.html

Weizenbaum, J. (1976). Computer power and human reason: From judgment to calculation. W. H. Freeman.

ETEC511 Final Project Retrospective

Project Design and Development

This project was something rather novel working in the realm of augmented reality, as I did not have a lot of previous experience with utilizing this kind of technology, especially in conjunction with language learning, which I am passionate about. It was fun to challenge myself in this way.

In terms of the design process, I think the literature review and supporting theories could be more thorough, as I would like to have the chance to dig deeper into existing research; put more thought in creating more robust scaffolded levels for language learners; and create supplemental potential user surveys to flesh out the usability testing.

As an example:
We changed our original idea of creating scaffolding levels based on the decreasing percentage of home language labels to increased delays in when the language labels will show up. This decision was made later in our design development, and though I think it is a better idea, I wished we had more background research to back it up.

If we had more time, I would like to develop the mock-up of our software a little more to create a more realistic demo of what could be achieved with the MyWebAR tools, to illustrate our concept and make it feel more grounded, concrete and feasible.

Collaboration and Teamwork

Bella, Jamie and Jennifer provided a lot of interesting observations from their hands-on experience working with elementary students in a classroom setting. From their teacher’s perspective and the observations they provided, we quickly identified an existing problem that new immigrant students were struggling with. This reminded me of my own experiences trying to integrate into new classroom environments with a complete language barrier when I was younger.

Though I do not work with students directly, I was able to provide some of the design-thinking to the project from my experiences in research as well as creating learning content for language instructors, and contributed to the project in a different way.

All four of us worked collectively on the project proposal, report and slides.
Jamie took on the main responsibility for creating the AR demo in MyWebAR. Though the tool claimed to be rather easy for beginners to create AR elements, it seemed like it was also rather limited in what it could actually do. Jamie did a great job of creating an alternative user interface that demonstrated what we needed it to do.

I think one of the main challenges was finding time to work collaboratively together, as Jennifer was in a drastically different time zone. Sometimes Bella, Jamie and I we would block off time and work together on the project together, which I personally found super helpful.

Moving forward, one project management skill I would like to develop is creating clearer project objectives with actionable items and deadlines to ensure that we all know what needs to be done within what timeframe. I believe this will help drastically with group projects in the future.

Disability Representation – Toph Beifong from Avatar: The Last Airbender

Introduction

The series takes place in a world with elements inspired by various Asian and Indigenous cultures. Some people in this world have the ability to “bend” (i.e. manipulate) one of the four natural elements of earth, fire, water and air. The Avatar is a being that has been reincarnated and the only one that can bend all four elements.

The story follows young Avatar, Aang, and his friends Katara, Sokka and Toph on a quest to prevent the Fire Nation from taking over the world.

The Blind Bandit – Toph Beifong

One of the characters with a visible disability is Toph, who was born blind.
She was brought up in a wealthy royal household with overprotective parents that isolated her from the world to prevent her from getting hurt due to her disability.

Toph learned earth-bending skills from badger moles, blind creatures that live underground, which she related to. She would sneak off to earth-bending tournaments under the moniker “The Blind Bandit”, often defeating opponents that were more able-bodies, athletic and stronger than her.

Disability Representation

Toph is a powerful earth-bender, not despite her disability, rather because of her blindness. Her blindness heightens her perception to sound and touch through vibrations travelling through the ground.

In the beginning, Toph tries hard to prove that she can “carry her own weight” as a projection and overcompensation of her overprotective parents’ excessive coddling. She gradually overcomes her fear of “being a burden to others” through learning to lean on her friends for support and guidance throughout the series. This “counter-frame of interdependence” of solidarity and friendship between the team grows as they continue on their adventures.

In the series, Toph is portrayed through her strengths and weaknesses, such as showing how she navigates environments that are inaccessible to her (i.e. in the air or water), or when she lets her friends down in ways that are related to her disability in battles. Doing so moves away from the “supercrip” model of representation and offers a more “dis/humanized” perspective of how Toph is represented (Goethals et al, 2022) as we see her growth and character development.

Lastly, I want to acknowledge my positionality that this is a personal analysis of a disability representations as someone who is no blind. However, I was able to find discussion threads on Reddit with personal experiences of those who are blind, which provides a “cross-expertise, iterative and dialogic” approach to participatory framing analysis (Goethals et al., 2022).

I was able to learn earth-bending not just as a martial art, but as an extension of my senses. For them, the natural earthbenders, it wasn't just about fighting. It was their way of interacting with the world" 
---- Toph Beifong from Book 5 Episode 12: The Firebending Masters

References

Avatar: The Last Airbender. Directed by Michael Dante DiMartino, Bryan Konietzko, Nickelodeon Animation Studios, 2005-2008

Goethals, T., Mortelmans, D., Van den Bulck, H., Van den Heurck, W., & Van Hove, G. (2022). I am not your metaphor: Frames and counter-frames in th representation of disability. Disability & Society, 37(5), 746–764. https://doi.org/10.1080/09687599.2020.1836478

Purks, E. (2020, July 6). Writing disability: Black Girl’s Digest. Retrieved January 31, 2023, from https://www.blackgirlsdigest.com/post/writing-disability

Tipping Point – Open Education Resource Textbooks Case Study

Creation of Open Education Resource Textbook with Interactive H5P elements for FREN1205 – French Conversation course in the Modern Languages Department at Langara College

Introduction

For the case of technological displacement, we were curious to explore the tendency and shift from physical textbooks to digital Open Education Resources (OERs) in higher education institutions. We were specifically interested in the tensions and opportunities that arose from the transition to online teaching and learning after the pandemic, especially with the normalization of online and hybrid e-learning.

We are grounding this inquiry of technological displacement in the case study of the creation of OER textbook with interactive H5P elements for a French conversation course at Langara College. In this assignment, we analyze the usability aspect of OERs from the instructor and student perspective, as well as explore the concerns of artificial intelligence, and issues surrounding digital labor in the process of creating OERs in higher education institutions.

Motivation and Background

The FREN1205 – French Conversation course at Langara College is offered in-person with the utilization of a digital OER textbook Le Français Interactif created by the instructor Mirabelle Tinio. To support our work, we had the opportunity to speak with the instructor to learn more about the case study. All case study context provided in this assignment came from this conversation. Below are some of the motivators for the creation of the OER textbook from both the students’ and instructor’s perspectives.

From the student perspective, the education landscape had been drastically transformed during the emergency transition to online teaching and learning during the beginning of the pandemic in 2020. The effects can be seen gradually resuming in-person teaching and learning once again in 2021, in which student surveys reflected that having additional supportive resources online available helped with their learning process and overall experience taking online courses. In addition, students reflected that physical textbooks were expensive and inaccessible, especially the ones that were ‘single-use’ for an individual course, and were less inclined to make such purchases.

From the instructor’s perspective, there were many factors that contributed to the transition of physical textbooks to a digital OER. The instructor that we interviewed had been teaching the French conversation course for the past at least 12 years. Though the original textbook they were using provided activities and exercises for everyday conversation scenarios, she found that the content was not up-to-date or culturally relevant enough for the students within the classroom. The instructor therefore found herself turning to other available language learning resources to patch together a curriculum plan that included vocabulary, grammar structure, and socio-cultural activities. The process was rather time consuming and she was never really satisfied with the existing resources.

With both students and instructor identifying that the current resources were not meeting their needs, it became clear that another resource should be introduced to solve the problem of learning resources for this course. Here, we can use the concept of technological utility to demonstrate, in part, why a tipping point occurred. Utility asks the question of if the technology fulfills the users’ needs or if it does what the users need it to do (Issa & Isaias, 2015, p. 4). Physical textbooks were not meeting the learners’ and instructor’s utility needs, therefore, a new technology needed to be introduced.

Simultaneously while working partially in the Educational Technology Department, there were many other instructors utilizing Pressbooks and other OER platforms to input resources into Brightspace, a learning management system. The existing integration of the learning management system and potential for further adaptation was an additional motivator for developing her own textbook as an OER for the class.

The Tipping Point

The opportunity and tipping point presented itself when BCcampus Open Education Foundation Grant for Institutions applications were open for project proposals for specifically utilizing H5P for Pressbooks in 2021. The grant was intended for British Columbia post-secondary institutions wishing to explore, initiate or relaunch open educational practices, resources, support and training on their campuses. Through this grant, the instructor was able to secure additional funding and support for creating the French Conversation OER textbook.

Benefits

Multi-modality, Interactivity and Flexibility

Learning languages is an activity that is inherently multimodal and incorporates a combination of multi-sensory and communicative modes (Dressman, 2019). The utilization of online OERs makes it possible to include multimedia and interactive H5P elements such that students can actively engage with the learning content, allows for more diversity in learning methods, as well as increasing the accessibility of course content.

Though the OER textbook included many different chapters and topics, each unit contained a similar format: the learning objectives, pre-test questionnaire, vocabulary, practice exercises, oral comprehension exercises, a post-test evaluation questionnaire, and self-reflection. This repeated format increases the OER’s usability because it is quickly learnable and memorable (Issa & Isaias, 2015, p. 33). The OER therefore creates a smoother user experience with less friction or frustration to navigate to the content than the physical textbooks, demonstrating again why this tipping point occurred (Issa & Isaias, 2015, p. 30).

The goal was to make the learning content accessible to both students and instructors with maximum flexibility and adaptability. Students could preview the units and prepare ahead of time before the classes; or review the units and practice on areas for further improvement, all at their own pace, with self-assessments available. Instructors can supplement the course delivery with additional resources, in-class activities or outing experiences, and utilize the textbook in a non-linear manner tailored to the needs and pace of the students in the classroom.

Living Texts

The content in the OER included resources that the instructor created and showcased content that previous students created as well, and can be seen as a co-created ‘living text’ (Philips, 2014) as a pedagogical tool, as well as a co-creation of knowledge within the classroom.

For example, in the activity “Interview a Francophone”, the instructor uploaded recorded interview videos of previous student’s work, as an exemplar of what the assignment would look like when current students approached the activity themselves, but also as an exercise for current students to practice their listening comprehension and understanding of French conversation in context. The instructor identified that this was to also make the students feel appreciated for their active contribution towards the course, and recognized students as part of the co-construction of literacy knowledge through this kind of interaction (Philips, 2014).

Creating an OER that operates as a living text supports increased usability because it allows for feedback to be implemented when offered by the learners (the users). A living text can push back against the challenge of “configuring the user”, where the designers imagine the “right way” for a user to engage with their technology instead of being open to how the users actually will engage with the technology (Woolgar, 1990). This OER as a living text can be adapted to user feedback and therefore there is not only one “right way” to use the resource. Instead, the OER can increase usability for a wider variety of users as instructors adapt it based on learner feedback. The instructor noted that keeping an OER like this up-to-date is very important. This is especially true if the OER is described by an instructor to learners as a living text that is responsive to their needs.

Equity, Diversity and Inclusion

As mentioned above, the multi-modality, interactivity and flexibility of the living texts contributes towards a classroom climate that reflects equity, diversity, and inclusion of the students that are currently taking the courses. This approach takes into consideration the positionality, lived-experiences, interests, and abilities of students within the classroom and their agency as an active participant in their own learning.

For example, taking the aforementioned activity of interview with a Francophone, with the crowd-sourced collaborative effort of the different interviewees, students are able to see the different kinds of ‘francophone-ness’ outside of the mainstream Eurocentric depiction of French speaking people, especially when it comes to the deep-rooted history of the French language as a tool of colonization.

By embracing inclusive pedagogical approaches and recognizing students’ diverse contributions, this approach to creating OER textbooks creates a supportive and accessible learning environment, fosters a sense of belonging, and affirms the value of students’ unique contributions to the learning process.

Challenges

Current Concerns: Teamwork Makes the Dream Work

One major challenge that the instructor encountered during the creation of this OER textbook was the lack of support from the institutional level, especially when new technological adaptations require more incentive and supporting resources to push for incorporation and utilization within the college, and furthermore, across institutions. Though the instructor did collaborate with other language instructors from the Modern Languages Department and advisors from the Educational Technology Department, there is a strong suggestion for creating a community of practice across institutions to support this work’s sustainability. The production of a brand new OER like this (as as its ongoing maintenance) involves significantly more time and energy than maintaining the status quo of using physical textbooks. There is a risk that the instructor’s digital labor of producing this kind of resource might be unknown by the institution if it is unseen.

On a practical and logistical consideration, this ensures the articulation of courses are leveled and aligned across institutions, especially when it concerns the transferability of courses and credits for pathway programs, such as Langara College. On a more idealized and aspirational endeavor, this promotes the collaboration and commitment to sharing knowledge and resources, encouraging accountability, peer reviews and continuous development of teaching and learning practices, enabling the community to build on each other’s work and fostering a culture of openness and collaboration in education.

Future Concerns: The Rise of Artificial Intelligence and Impact of Digital Labor

Though the BCcampus grant did provide funding for the instructor to develop the OER textbook, there needs to be more support when it comes to compensation of the unseen invisible work that is added on to the already existing duties of a teaching faculty member. With increased digitization of instruction within higher education, comes an expectation of an accelerated pace of work (Woodcock, 2018, p. 135). There can be an expectation, even implicitly, within institutions that work becomes “easier” as a result of digital resources like this OER textbook. This can result in work pressures and time pressures expanding for instructors who have created digitized aspects of their work.

Another risk for instructors is the value that is placed on published work to push an academic career forward (Woodcock, 2018, p. 136). The motivation to pursue the creation of open access work can be reduced if the institution the academic is working within has rewards for published work. While an OER like the one described in this case is a different kind of open access work than a journal piece, its creation and upkeep exist within the same labour hours for an instructor. The instructor must be significantly committed to the creation of the OER if there is limited institutional support, as described in this case, and also if there is institutional pressure to spend time doing other, more valued work, such as publishing at a more prestigious journal.

Finally, there is a tension inherent in the use of artificial intelligence in relation to OERs. As with this case study, we know that producing and maintaining OERs can be time, labor, and resource-intensive. With the rise of large language models like ChatGPT in the past year, there is a potential to employ AI tools like this to support the creation of OERs. This might seem to reduce the human labour needed to create an OER like Le Français Interactif. However, we also know that AI tools like ChatGPT do not appropriately cite sources and can even ‘make up’ information. Uncited sources are problematic because they effectively steal intellectual property from other academics and false information is problematic because it diminishes the reliability and utility of the OER.

Even more concerning is that AI language models are trained with data that can be biased and produce content that is embedded with this bias (Buolamwini, 2019). With an OER project like this outlined in our case study, it could be counter to the desire to create more culturally-relevant and inclusive resources to produce them in “partnership” with an AI tool. More relevant to this case study, regarding language translation, AI tools like DeepL can be helpful but are not yet at the point where they can translate as effectively as a human who speaks multiple languages. For this reason, instructors might be wary of using AI tools as “co-authors” for OERs to ensure the quality of the instructional or learning resource remains high.

Conclusion

This case study demonstrates how the creation of an OER textbook for the FREN1205 – French Conversation course at Langara College exemplifies a pivotal shift in educational resources toward digital platforms. This tipping point is a response to the evolving needs of both students and instructors in the post-pandemic era of education. Ideally, an OER textbook offers learners enhanced accessibility, flexibility, and more inclusivity within their educational experience. However, challenges such as institutional support for digital labour and concerns surrounding the rise of artificial intelligence underscore the importance of institutional buy-in and ethical considerations as we integrate OER textbooks into the student experience.

References

Buolamwini, J. (2019, February 7). Artificial Intelligence has a problem with gender and racial bias. Time. https://time.com/5520558/artificial-intelligence-racial-gender-bias/.

Dressman, M. (2019). Multimodality and language learning. In M. Dressman, & R. W. Sadler (Eds.), The handbook of informal language learning (pp. 39-55). John Wiley & Sons, Ltd. https://doi.org/10.1002/9781119472384.ch3

Issa, T., & Isaias, P. (2015) Usability and human computer interaction (HCI). In Sustainable Design (pp. 19-35). Springer.

Phillips, L. G., & Willis, L. (2014). Walking and talking with living texts: Breathing life against static standardisation. English Teaching : Practice and Critique, 13(1), 76.

Woodcock, J. (2018). Digital Labour in the University: Understanding the Transformations of Academic Work in the UK. tripleC: Communication, Capitalism & Critique. Open Access Journal for a Global Sustainable Information Society, 16(1) pp. 129-142.

Woolgar, S. (1990). Configuring the user: The case of usability trials. The Sociological Review, 38(1, Suppl.), S58-S99.

Intellectual Production #7: Digital Labour

Brief

Based on a close reading and careful analysis of Kate Crawford’s chapter on “Labour” in her book Atlas of AI. Identify and describe, in her terms, a kind of work you regard as “digital labour”. Illustrate your observations, with examples and ideas drawn from the readings.

Use any cartoon-creation/animation software of your choice to communicate your understanding of how, why and in what ways work today has revived older practices of industrial labour exploitation, especially on the role of AI in this reversal.

Preface

Drawing upon a conversation through text with my friend E, it made me reflect on her experience of creating online content in relation to ideas that I was reading about in this module about Digital Labour. That was the starting point to creating this collage comic about some of the connecting concepts and theories to what she was going through.

For this project, I decided to use a collage comic as the medium for a few reasons:

Collage Imagery

Drawing from the different readings in this module, it resembles a montage of concepts, akin to a collage where individual ideas are layered and interconnected with each other. I wanted to use juxtaposition of imagery and text to create a meme-like image similar to “content” that one would see on social media, as a critique yet participation in “content-creation” as a form of sharing of ideas.

Woodcut Printing and Pop Art

I chose to use woodcut-printing of imagery as this technique of printing can be mass produced due to the division of labour in the creation process such that artists can adapt to the medium easily without needing to learn the use of woodworking skills. Usually prints are described as “designed by” rather than “by” an artist, highlights the invisible labour from woodworkers and printers that carve the actual blocks, yet go uncredited.

I chose to use bright neon blocks of color reminiscent of the pop-art movement as the philosophy of pop-art not only celebrates the imagery and symbols of mass culture (i.e. advertisements, consumer products, etc.) they also critique the consumerist nature of society. This movement also challenged elitist notions of art by blurring boundaries between high and low culture, making art more accessible to a wider audience and thus democratizing the art. I think this parallel can be drawn to how social media democratizes many aspects of production and distribution of labour.

References

All images are sourced from the public domain archives of the British Library.

Crawford, K. (2021). Atlas of AI: Power, Politics, and the Planetary Costs of Artificial Intelligence. New Haven: Yale University Press. https://doi.org/10.12987/9780300252392

Duffy, B.E., (2017). (Not) Getting Paid to Do What You Love: Gender, Social Media, and Aspirational Work. Yale University Press. http://www.jstor.org/stable/j.ctt1q31skt

Gandini, A. (2021). Digital labour: an empty signifier? Media, Culture & Society, 43(2), 369-380. https://doi.org/10.1177/0163443720948018

ViziBLUE: Exploring Educational Data

Brief

Go to the ViziBLUE websiteLinks to an external site. and review the types of data collected about individuals on a typical university campus. Review the about page for ViziBLUELinks to an external site.; it provides background on how it came to exist.

Examine the various data sources as if you were a student on campus, with the data representing your own. Take note of these sources and consider the potential inferences that could be made about you by combining them.

At a brief glance on the ViziBLUE website, I was already unsettled by the broad range of information that ViziBLUE aims to collect from the student and staff/faculty body via the various services provided on campus. When I opened the individual data sources, I was surprised at the granularity of the data points that they can potentially collect and utilize, and it seems like there is no option of opting out of this data collection.

With the overlap of many similar data points in each data source, it becomes easy to triangulate and pinpoint demographics to identify the individual. Even when some of the data is stated to be “provided voluntarily” by the users themselves, through cross-examination of the data points, one can provide a general persona and make inferences of the individual, which can potentially negatively impact those in racialized, disabled, and queer minority communities. Where some data points are crucial for certain access to necessary services (e.g. diagnoses of disability for accessibility reasons), it might prove harmful in other contexts (e.g. medical records of disability may hinder immigration eligibility).

Though it does mention in AI Data that the university does not sell or license of personal information, “individually identifiable information may be shared outside U-M as required by law, or when we believe, sharing will help protect the safety, property, or rights of the university, members of the university community and university guests” is rather vague in terms of how and what is deemed to fit that criteria, which could be an important missing piece to the current website.

At Langara College, there is Institutional Research that collects and analyzes data for institution-wide policy forming, strategic planning and decision-making. It is not as granular and personal to that of ViziBLUE, and is more related to the functioning of the institution, it feels less intrusive compared to what University of Michigan is doing.

On one hand, it is important to make transparent to the users of the collected data to “help them view, understand and manage their personal information”, however, it also shows how much information is being collected and in situations of data breaches, it can have dire consequences.