This month’s LAVA meeting will even be more informal than most – but for those of you online before the holidays, I couldn’t not have at least one chatGPT themed LAVA session to wrap up 2023. My question is – what does chatGPT “know” about Learning Analytics? Let’s find out!
Homework: send me (or prepare) your prompts – what questions would you ask to find out what chatGPT “knows” about Learning Analytics? Are you an expert in a particular area of Learning Analytics – what is something you would consider “common knowledge” at this point – we will see if chatGPT gets it right.
In the session we will review the prompts, see what chatGPT comes up with, and evaluate the responses.
This week (Thursday, 2-3pm, Zoom) our LAVA session will be led by Warren Code, the Associate Director at Skylight.
Running for the six weeks from early July to the start of Jumpstart in mid-August, the Academic Essentials program ( https://you.ubc.ca/academic-essentials/ ) launched in Summer 2020 as a set of three optional Canvas courses intended to help incoming undergraduate students to prepare for their first year at UBC’s Vancouver and Okanagan campuses. I was one of the developers of the “Readiness for University Mathematics” course, providing a review of mathematical concepts and practice of associated skills as well as recommendations for studying and other engagement in a first Calculus course at UBC. In this session, I will be describing our approach so far in evaluating this review course in terms of outcomes in the credit-carrying math courses in the Fall term, where we are drawing on a self-assessment taken at the start of the course, student activity in Canvas and WeBWorK (the online homework system where most of the practice occurs), and some diagnostic measurements taken in the Calculus courses that have been running for several years now.
Our key questions are:
– Does the summer review course bolster students’ pre-calculus skills?
– How does participation in the summer impact students’ success in their Fall term Calculus course?
– How prepared are our incoming students in the last few cohorts?
I will discuss some preliminary findings for the first two questions from the 2020 cohort as we are preparing to add data from the 2021 cohort later this term. Our main team so far for this analysis is myself, Zohreh Moradi (Skylight’s Research Analyst) and Costanza Piccolo (Associate Professor of Teaching in Mathematics and one of the other co-developers of the “Readiness for University Mathematics” course).
Emma Novotny is the Design Manager from UBC Faculty of Arts. In this session, she will be leading a visualization feedback session for the group!
If you have a visualization that you think could be improved, that you would like the eye of a graphic designer on, or would otherwise like feedback from Emma please submit it to LAVA before the end of day on MondayFebruary 3rd . With the visualization, please include:
The context – what project is this for?
What is the goal of the visualization?
Who is the audience?
How will it be used?
Any other detail you think is important for feedback
If you don’t have your own visualization to show but know of one that you would like to see improved send that! Emma will choose 1-3 of the visualizations and spend time on the 11th talking about her thoughts on how to improve the visualization/ what she might change/other feedback.
Alison Myers is a Research Analyst at UBC Sauder School of Business. Working on a variety of data projects, she has been using Tableau for creating interactive dashboards and visualizations. In this session she will be presenting some tips and tricks and a closer to intermediate level demonstration of Tableau. From Alison:
“I will be presenting some tips and tricks and a closer to intermediate level demonstration of Tableau. In only an hour we won’t be able to go into too much detail, so if you are a beginner (or not a Tableau user at all) I think you will still get something out of this session, and some of the things I will talk about, I wish I had discovered earlier in my Tableau journey. The tips and tricks will be the “little things” that I often do when I am working on a Tableau project to keep myself organized (and sane). I will spend most of the time talking about things that I do when I’m building visualizations that either make things easier (for me), or take advantage of thinking about Tableau in a different way. This will likely include:
Calculated fields that you would likely find in all of my dashboards
Table calculations (although, I’ll be honest, I only have a basic understanding of this myself, so we’ll see how brushed up I can get before Tuesday)
I will briefly introduce some of the topics and include resources, and others I will go more into depth during the hour. As always, there is generally no one way to use Tableau, so this demonstration is from my experience only, and feedback is always welcome. I will try to share a workbook that you can all download to follow along (or that will at least have the same dataset)”
As Associate Director of Skylight, The Science Centre for Learning and Teaching , Warren Code, has has been involved with a range of teaching and learning projects across the Faculty of Science, with a particular focus on the Carl Wieman Science Education Initiative and other faculty/campus-wide initiatives. A substantial part of his work is advising and professional development for their department-based Science Education Specialists, evaluating impact on students and faculty, and connecting with people from UBC and other institutions interested in the accomplishments in teaching and learning within the Faculty of Science. He has also been connected to some analytics projects, including analysis of student response/log data in WeBWorK. In this session, Warren will give an update on the current state of Learning Analytics in the Faculty of Science:
“I’ll be presenting an update on learning analytics in the Faculty of Science: some notable recent projects, and a variety of ideas people at Science would like to explore with the right kinds approaches, partnerships, and support. I’ll be interested in hearing your experiences in any similar types of projects and any recommendations you may have as to how we might proceed. You will not need to prepare anything in advance.”
Currently, “New Analytics” can be enabled on a per-course basis as a beta feature on UBC’s installation of Canvas. Instructure intends to disable “Old Analytics” and cutover to “New Analytics” in their March 2020 release.
LA Project team has started a mission to review the features of New Analytics, in comparison to Old Analytics. In this session, Dennis Foung, Support Analyst from the project, will walk us through an overview of the identified differences.
This will be followed by a group discussion around some of the outstanding issues with New Analytics. In preparation for the transition, UBC is working on a change management plan. Any feedback from the community around issues that require prompt attention or escalation to Instructure would help institution in this process.
Exploring the data from CLAS/WeVu: A video annotation platform for learning (Session 1 of 2)
Leah Macfadyen is a tenure track Instructor in the Department of Language and Literacy Education, UBC Faculty of Education and Associate Director of and Instructor in the UBC Master of Educational Technology Program. Leah will be leading one of two LAVA sessions exploring CLAS/WeVu and the data generated from it.
“In this introductory session, I will demonstrate and discuss how I have used a collaborative video annotation tool with students in an online course. CLAS (the Collaborative Lecture Annotation System) – now available as WeVu to users outside UBC, was developed in the UBC Faculty of Arts, and is actively in use in various UBC faculties and elsewhere to support learner engagement with video (as well as audio and now image) resources in a variety of ways: whole-class annotation of lecture videos, one to one instructor feedback on learner ‘performance’ of different kinds, asynchronous group ‘video seminars’ and more. CLAS/WeVu offers learners additional tools to support self-regulated learning, offers educators tools to gather and give richer feedback, and – I would argue – can support more effective social presence in virtual learning environments.
CLAS/WeVu collects some data that might be usefully explored to reveal learner usage patterns. I will review what data is currently available, and share some sample data (and a data dictionary) with you. So far, little use has been been made of this data to give instructors or learners richer feedback.
I invite you to join in a small collective exploration of the data (either independently or working with others), experiment with analyses or visualizations that might be useful, and to return to LAVA on Tuesday November 19th to demo what you have achieved. Recommendations for future improvements in data collection and data structure will also be welcome.”
A few published studies investigating CLAS data:
Mirriahi, N., Jovanovic, J., Dawson, S., Gašević, D., & Pardo, A. (2018). Identifying engagement patterns with video annotation activities: A case study in professional development. Australasian Journal of Educational Technology, 34(1). https://doi.org/10.14742/ajet.3207
Mirriahi, N., Liaqat, D., Dawson, S. et al. (2016). Uncovering student learning profiles with a video annotation tool: reflective learning with and without instructional norms. Education Tech Research Dev 64: 1083. https://doi.org/10.1007/s11423-016-9449-2
Pardo, A, Mirriahi, N, Dawson, S, Zhao, Y, Zhao, A & Gasevic, D (2015). Identifying Learning Strategies Associated with Active use of Video Annotation Software. in Proceedings of the 5th International Conference on Learning Analytics & Knowledge (LAK 2015). ACM Press, New York, NY, USA, pp. 255-259. https://doi.org/10.1145/2723576.2723611
Mirriah, N. & Dawson, S. (2013). The pairing of lecture recording data with assessment scores: a method of discovering pedagogical impact. Proceedings of the Third International Conference on Learning Analytics and Knowledge.doi>10.1145/2460296.2460331
In this session, we will be watching “Practical Analytics in Curriculum Complexity – A Summer Analytics Academy Webinar”, which was part of the Webinar series offered by the Centre for Student Analytics (Utah State University) over the summer.
In this webinar, Dr. Mitchell Colver discusses the curriculum tool https://curricula.academicdashboards.org/ in the framework of sustainable analytics. One of the big questions discussed in the webinar is “How do you socialize analytics in a practical way?”.
There a lot of points that are valuable to consider and discuss in our contexts at UBC. We don’t expect to get through the entire video, and our hope is for it to generate some discussion amongst ourselves as we go.
Shenia has been working as a work learn student at UBC Faculty of Arts over the summer, designing and developing student-facing dashboards for edX MOOC courses. In this session, Shenia walked us through her project:
“As a summer Work Learn student, I developed a student-facing dashboard for MOOC (Massive Open Online Course) students on edX. The dashboard design is similar to widget developed by Dr. Dan Davis, available here https://dan7davis.github.io/papers/LAL_Davis.pdf, surrounding the central idea that MOOC learners can benefit from understanding the learning habits of successful MOOC students in a past iteration of the course they are taking. This project spanned from May 2019 to August 2019 under the supervision of Ms. Sanam Shirazi at UBC Arts ISIT.”
Marko is a Student Learning Analytics Developer at Sauder. In this session, he will be telling us about the Canvas project that he has been working on this last term. From Marko:
“As a newcomer to Learning Analytics, I’ve spent the last several months exploring techniques for data collection, cleaning, and visualization in an effort to pull meaningful insights from Canvas data. During this LAVA meeting, I will discuss some of the interesting discoveries and talking points I’ve come across throughout this process — with an emphasis on the Tableau Web Data Connector (WDC) technology. I’ll be going through the process of building and using a Tableau WDC at a high level, as well as discuss some potential technical challenges and shortcomings with this tool. I’ll root my discussion in an example WDC and visualization that I’ve built, and pose some higher-level LA and Data Vis questions to the group (I’d love to learn more!). Note, the talk will be somewhat technical in spirit but doesn’t require any background knowledge on programming concepts or Tableau.”