Educators strive to provide the best learning experience for our students. Data is ubiquitous and can be used by any educator regardless of subject, expertise, or teaching style. Predictive Analytics help educators positively impact the future of students by allowing us to suggest optimum learning pathways, courses, and learning strategies to enhance the development of each individual.
Our week-long crash course in Predictive Learning Analytics in Grade 7-12 education provides you with knowledge, practical applications, and opportunities to find your own relevant venture. Here is our site.
Your tasks:
- Scan through the entire website, then read what interests you. Feel free to reply to our blog post any time you have questions or interesting thoughts to share.
- Under “Applications and Implementation”,
- watch the TED Talk about PISA by Andreas Schelicher and discuss using the Padlet.
- learn about CEM data by reading/watching the resources provided and complete the MIDYIS Reflection near the bottom of the page.
- Try to find an interesting venture in Learning Analytics relevant to you. If you do find one, write a brief description and critique as a reply to this post. Please also share or pitch any original ventures you have.
- A brief experimental activity will be revealed on Friday June 23rd. Please visit the “Experimental Activity” page for instructions then.
Sincerely,
Roger, Vera & Mike
Great website, predictive learning crew. While we, as educators, all likely complete an informal data collection at the beginning of each semester/unit/day, it is interesting to see the process formalized, and discover how powerful data can be when planning, organizing, and preparing for upcoming lessons and activities. I personally use ‘show of hands’ indicators to see the proficiency from my class, I believe I will start structuring the practice to assist students who may be struggling in the near future. While we can’t see into the future, this allows us to make educated guesses and can only be beneficial to a student that might otherwise grow to feel left behind.
I’m glad you added the Ethical considerations as they may be overlooked in the pursuit of data, but consent is vitally important and we must avoid pigeon-holing students into categories by strictly using the data. Predictive learning looks be best use as a way of lifting and supporting students who MAY struggle and pushing and challenging those who look to be more advanced. (one little note, Ethical is spelt incorrectly on the header of that page. A quick easy editing fix)
My school is an AP school and while I don’t approve of how much we ‘teach to the test’, I do marvel at the copious amounts of data collected from all over the world. Whatever you may think about standardized testing, you can’t say that the data collected is useless.
Overall, I’m really impressed with your professional approach to the topic and the various interactive ways you designed the site. Well done, team!
Thanks for you comments and review of our site Sworley! I agree about the ethical concerns. The most successful approach I have had with our teachers has been to discuss the data we have as just one piece of the puzzle when it comes to understanding the whole student. Seeing it as a part of what we know about the student along with pastoral notes, interactions, classroom engagement helped our teachers see it as useful. I agree about the use of the word “may” as a potential indicator for students who may need more challenge but as we know students lives are complex!
I am at an IB school and we face some similar challenges. At the same time, if we removed the exams and standardization I wonder what might happen to the student grades (we certainly experienced inflation when government exams were removed in British Columbia).
How does your school use the AP data? Is it discussed at all?
Thanks for contributing!
Yes, we are actually measured against other AP schools in the province. The AP teachers use the high level of 4 and 5 test results as an appeal for the younger students to take the challenging courses. The guidance also use the grads and their college and university acceptances as proofs that AP courses are worth doing. Some of these are collected observational data but most are hard numbers. We have AP nights, where prospective students and parents come and learn about the programs. Much of the data is provided in that instance.
Great work! I really like the idea of “Use Data to Build Better Schools”, by adopting a data-driven approach, schools aim to enhance teaching practices, allocate resources effectively, and create a supportive learning environment that promotes the success of all students. When we are talking about some difficulties and challenges implementing this approach, two things just come up in my mind.
One of the key challenges is resistance to change, which may arise from stakeholders such as teachers, administrators, and parents. Some educators may harbor skepticism regarding the effectiveness of data-driven decision-making or feel overwhelmed by the additional workload associated with data analysis. Another challenge is the constraints of time and resources. Implementing data-driven practices necessitates dedicated staff, time, and resources. Schools may encounter limitations in terms of staffing, funding, or time availability, which can hinder the comprehensive implementation of data analysis and intervention strategies. What do you guys think of how to overcome these challenges?
Thanks for your comment! You’ve definitely brought up valid points. To address the additional time and resources required, the initial setup (e.g. of data collection software) could be intensive but since it would be a schoolwide approach, there will be many staff members chipping away at parts of it. On top of that, the same software and documents will be used every year, which means there are many opportunities for revision and improvements. A school could start with something as simple as an Excel sheet tracking attendance and grades, then compare with district-wide or global data. When a school has had some experience and success implementing data analysis, resistance should decrease as long as the data collection is not intrusive. For example, tracking attendance and grades is nothing new, but now more can be done with the same information by comparing, triangulating, and analyzing patterns.
Hello team – thank you for the website and helping me learn (and relearn) about learning analytics!
I was particularly interested in the section of your website about “attendance and punctuality records” and how these serve as important data sources. In my previous school, we utilized a platform called CPOMS (https://www.cpoms.co.uk/) which serves as a pastoral, safeguarding and wellbeing data point for students. How we used it in regards to attendance in particular is that when a student is absent for an assessment, we would input a message in the system. It then will be sent to stakeholders, comprising mandatory and optional people. For example, an absent student notice through CPOMS will be sent to the Senior School Principal, the guidance counselling team and the Advisory / Home room teacher but other administrators or teachers could be included with the tick of a box. This is not meant to be punitive; rather, it is meant to track patterns of absences that are linked to anxiety and other socio-emotional issues.
Pros to this system:
– the database keeps information centralized and organized
– teachers do not have to hunt down information about a student by asking around, both in person and through email
– emails full of information are limited (notifications, on the other hand are increased, as each inputted post equals an email)
Thanks!
Maurice
Thanks for sharing Maurice! My school also uses a similar centralized administrative platform and Google sheets to track student data. Like you said, one of the best features of the platforms is that teachers can add notes for students which could be praise or concerns. These notes can then be shared with the student, teachers, administrators or parents based on need. Students love it when they receive praise from one teacher and then other notified teachers congratulate them or add commendations when they see the student.
Maurice did your school see an increase in attendance because of these measures? Is that something that your school was tracking as well. I could see it getting tedious getting an email every time someone misses a day but helpful to identify a pattern of student absence!
You have highlighted one of the downsides. Yes, as an admin, you would receive a notification for EVERYTHING: uniform infraction, delayed assessment, incident reports etc… The other element is that you need to use an external device (personal phone) with an authentification tool to be able to log in. It can keep you connected for a week; it is an extra step in place for privacy issues which brings up other potential issues. For example, why do teachers have to use a personal smart phone for work? In other fields and businesses, this would be paid for by the company! It also assumes that every employee his their own device…
I honestly am unsure about the data connected to attendance but what I can add is that the school began implementation during Covid in-class instruction, so the data would have had complex factors:
Was the student sick?
Scared of getting sick?
Was a family member sick?
Were the parents worried about sending the child to school?
And the aforementioned question which deserves repeating: Is the student anxious about assessments and is missing class on these days on purpose?
What an exciting platform to use! I think that throughout/after covid we started seeing quite a few more of these apps focused on behavior tracking, some including positive reinforcement and other exciting ways of assessing student behavior. Though, I couldn’t imagine being a student when something like this exists, where my whereabouts are tracked and shared with so many people. In some respects, it feels a bit invasive, but that mostly comes from a place of unfamiliarity (and my history of being a chronic class skipper).
After doing research on some learning analytics companies, I found that Citivas Learning stood out for a few reasons. While many of the other companies leaned into the data mining and leveraged the results to show the efficiency of the teaching, Citivas focused more on inclusivity and sustainability in a learning environment. They lean toward equitability, something not as prominently shown in other businesses readings or websites. As a venture analyst, anytime you can define something you do as unique and stand out from the crowd, it’s a positive. This is an overcrowded field, with many competitors, so standing out is key to separating yourself.
Here is their official sales pitch:
Civitas Learning offers a suite of predictive analytics tools that help institutions identify at-risk students, optimize course scheduling, and improve student success outcomes. Their platform leverages machine learning algorithms to analyze various data points and generate actionable insights for educators and administrators.
As you can see, they lean hard on the at-risk students and student success. I’m only beginning to research them but this presenting group has inspired me to learn much more about these companies as I am in a teacher leadership position next school year and I might want to make it my duty to institute one of these services to work with our teachers and administration.
Citivas Learning Official Page:
https://www.civitaslearning.com/platform/?gclid=CjwKCAjwhdWkBhBZEiwA1ibLmOZx4kjNPo2cpV-uWSCJKvoSjCj4CiCE_msS4k12ZpQN2I4B1twWfhoC4yMQAvD_BwE
Thanks for sharing this learning analytics company Simon. I like how they are focussing on student success but also appealing to the universities and colleges from a financial perspective. Understanding that student success ultimately means financial stability for an institution is a helpful incentive for institutions (especially smaller ones or institutions looking to grow). It’s interesting that they seem to identify at-risk as those that are at-risk of dropping out and are targeting ways to help those students remain in college. I wonder if this approach leads to longer term success for the students or just keeps them present in college longer.
I quite enjoyed looking through your own site’s analytics on the “Experimental Activity” page. It is particularly interesting to note the “average engagement time” on the various site pages. As one might expect, the references page currently has the lowest engagement time as it is likely the least pertinent aspect of the site for many as the more content and activity focused pages garner more engagement time. I would be interested to employ Google Analytics on my own school’s website. On our website, each teacher is responsible for updating their own class/subject area pages. It would be interesting to see how many users and engagement time my pages gets each month and whether it reflects the effort that goes into updating the page. From a venture perspective, I can see how site analytics could serve as a particularly powerful tool in assessing the depth at which potential buyers or investors dive into the site. For example, if the analytics suggest that most users simply visit the homepage for a very limited engagement time, and do not engage with other aspects site, then it may highlight a lack of interest in the product, or at least how the product is being presented on the landing page.
I think this is a great point Nick. In education too often I feel like we are doing things (or being told to do things) when we are not clear what the outcomes are. I like the example you used of tracking your own sites analytics to see if parents/students are using the site in the way your school is hoping they will. If they data shows they are not then the school needs to think about how it is promoting that class/subject page and if the time investment in keeping them updated is worthwhile.
This stuck out to me as well. Recently, I have been paying closer and closer attention to emails I send out and the statistics that Microsoft Outlook provides. It is a little disheartening to see the results when I send larger emails to the staff at my school as it typically shows that less than 75% of staff open my emails and generally the amount of time spend reading the emails either means that the staff are all ready to compete for the speed reading world record or they are not really reading the email (probably the latter). Rather than being frustrated by this though, I think it warrants a look into how I am putting my emails together. Maybe they are too long, maybe I am sending too many, but the point is, data analytics can and arguably should be used to inform us about what is working and worth investing time in and what is not.
I think this reflection is really important Benjamin. I have had to restructure emails to only a few lines to try and increase the likelihood that they will be read. We moved to a single weekly method of major communication to try and limit the number of emails individuals were getting. Great example of using the data and thinking critically about what it means!
Great website with an immense amount of information about analytics.
The Ted Talk about PISA I found especially interesting. There are so many pieces of data they have collected to interpret the success of school systems, it was interesting to see the comparisons on a global scale. Data driven practice gives educators and administrations the opportunity to learn from past practices and improve methods to better support students.
My board has recently been strongly pushing the use of internal software to host all student assessment data, and standardizing this across grades and I am sure soon across schools. There are a lot of different tools within this software to interpret the data and analyze how a student is performing across subject areas on specific skills. The downfall of this is teachers have been given minimal time to learn about the capabilities of this software, very little time to collaborate and discuss with teaching partners about what data to record and analyze. For movements like this one to be effective I think there needs to be considerable time set aside for teachers to deliberate what data should be recorded, for what purpose, and looking at the bigger picture of what we hoping to gain from the data we choose to track. If there isn’t thoughtful deliberation about how the data can be used, then it turns into more work for teachers without the added benefit of feedback about their practice and student success.
Hi Kelby, thank you for sharing your thoughts on learning analytics. I can relate to the challenges you mentioned regarding the implementation of internal software for student assessment. It’s disheartening when teachers are given minimal time to understand the capabilities of such tools and insufficient opportunities to discuss what data should be recorded and analyzed.
I believe data-driven initiatives need to be applied in a thoughtful and inclusive approach. Teachers should have dedicated time to deliberate on which data points are meaningful and relevant to their instructional practices. Moreover, it’s important to consider the broader goals and outcomes we hope to achieve through data analysis. Without a clear vision and purpose. Otherwise, collecting data can become an additional burden for teachers without providing meaningful feedback on their practice or student success. I surely hope your institution could adopt a more thoughtful and inclusive approach.
In order to overcome these challenges, it’s crucial for educational institutions to prioritize ongoing professional development opportunities that focus on data literacy and analysis. Teachers should be empowered with the knowledge and skills to interpret and utilize data effectively in their classrooms. Additionally, establishing collaborative platforms or communities of practice where educators can exchange ideas, share best practices, and learn from each other’s experiences would be immensely valuable.
If you have any further thoughts or ideas on this topic, I would love to hear them. Let’s continue the conversation!
Great website with a lot of information! I enjoyed looking into the venture opportunity on Panorama Education. It seems like a really useful tool that gives data on the whole student rather than just an academic outlook that many LMS systems provide. Great to see it includes the ability to monitor SEL and provides “playbook” suggestions for intervention plans that can be tailored to the student.
I would see this especially helpful at the beginning of the year when, as a teacher, I am unfamiliar with new students and how to best support them. I find often there is a gap and room for communication between teachers/departments about student needs early on – especially looking at a more holistic view rather than past report card grades. Thanks for sharing, really great job on this one.
Hi Paul, I completely agree with your suggestion of taking a more proactive approach to close the communication gap between educator and learner. It’s essential to move beyond relying solely on past report card grades and adopt a more holistic view of students.
While report card grades provide valuable information, they often focus primarily on-paper performance and may likely not capture the full range of a student’s abilities, challenges, and social-emotional well-being. At the beginning of the year, when teachers are getting to know their students, having access to a platform like Panorama Education can be particularly valuable. It can provide insights into students’ strengths, areas for growth, and specific challenges they may face.
In my workplace, we use Salesforce to boost student retention rate, and we incorporate a student care alert system to bridge the gap between staff and faculty. For instance, a faculty member identifies a student with anxiety symptoms, they could then raise a care alert to a ticketing system, and this ticket will be forwarded to corresponding staff to follow up.
If you have any further thoughts or ideas on improving communication and holistic student support, I would love to hear them.
I use Salesforce in a corporate environment and didn’t even think about its applicability to a school or learning environment!!!! When I worked for Toronto public health we used Salesforce, and each person would have their own profile with different interventions, but basically, any healthcare worker in Ontario could add a clinical note to this person’s profile, which meant that it was easy to stay up to date on their health situation and see what other staff had identified as areas of concern. I totally see how this could be used in a classroom to keep track of student progress and ensure that the documentation that previous teachers are taking could also help inform new teachers. The largest concern of mine for this kind of system would be ensuring that new teachers do not hold biases towards the students based on the notes from previous teachers.
Experimental Activity – I enjoyed looking at the data and it was very cool being reminded that this is a global class! Who else is in Indonesia like me?!
Here are my questions:
– Is “Google Analytics” fairly easy to set up? Would it be like an extension through G Chrome and is it only accessible for the authors of a webpage?
– What happens when someone keeps a webpage open for hours or days like I often do? Would that skew the data?
– What is your team planning on doing next with this data? Will it help you analyze which activities were “successful” and which ones were “less successful”?
Thank you,
Maurice
– Is “Google Analytics” fairly easy to set up? Would it be like an extension through G Chrome and is it only accessible for the authors of a webpage?
Yes it was easy to set up even with no prior experience. I followed the instructions to register, embedded the code in our Google Site, and it began collecting data.
– What happens when someone keeps a webpage open for hours or days like I often do? Would that skew the data?
I’m guessing there are ways for Google to determine if you’re active by monitoring page navigation but that’s an interesting point.
– What is your team planning on doing next with this data? Will it help you analyze which activities were “successful” and which ones were “less successful”?
We thought this would an interesting meta activity in that we’re analyzing data while presenting data analysis. It will also be a feedback/verification tool for our OER and content.
Hmmm – I tried to install to my Assignment 1, but I kept on facing this issue where it says: “Data collection isn’t active for your website. If you installed tags more than 48 hours ago, make sure they are set up correctly.”
Did you put the HTML code (Google Tag) installed manually into the title of each page? I followed a Youtube tutorial but the theirs was a bit different as there were two codes to input per page. Is this what you did?
Thank you Roger!
Google Analytics Feedback:
I used to use Google Analytics a lot at my old school to track patterns and location data. When combing through the results on the experimental activity, I was surprised to see the ‘Average Engagement Time’ so low. Some of this could be explained because when watching videos, I tend to open them in YouTube, which sends me away from the original site. That time logged watching the videos are not counted towards the engagement time on the page. I think the same might be true with the Padlet and Survey. By having so many dynamic, yet off site activities, they contribute to not telling us the whole story of the website and the actual amount of engagement each participant had with your work.
It was a great idea to add this and it fit really well into your theme. Well done!
Thanks Simon ! I was a bit concerned about the time on pages as well but you can see how this data would be really helpful if you were using the site for business. I hadn’t considered the point about moving to an external host (youtube or padlet) as a reason the time on page was less!
Hi Predictive Learning Analytics Team,
I really enjoyed your website! The depth of information and how it was presented was great in relearning about predictive analytics. In particular the video from Educause stood out because it highlighted implementation as one of the biggest challenges for educators and institutions once they have collected the data.
I also enjoyed looking at the experimental activity performed as part of the OER. I am relatively unfamiliar with Google Analytics, so I was wondering what “Event Count” is tracking in the data. Is it the number of interactions (e.g. clicking an element) by users on a specific page? I think over a larger sample size, this kind of dashboard can be incredibly useful in planning action items, specifically as formulating survey questions to add a qualitative dataset to the existing quantitative data. Thanks again for the information.
“An event allows you to measure a specific interaction or occurrence on your website or app. For example, you can use an event to measure when someone loads a page, clicks a link, or completes a purchase, or to measure system behavior, such as when an app crashes or an impression is served.”
https://support.google.com/analytics/answer/9322688?hl=en#zippy=%2Crealtime-report%2Cdebugview-report
Since we did not make any adjustments to the events, Google has automatically categorized interactions with the website into the following:
page_view
user_engagement
scroll
session_start
first_visit
I can see applications for analyzing the content and media visitors engage with on the website. We can gauge interest in a link, video, or document on the website based on the event count.
“An event allows you to measure a specific interaction or occurrence on your website or app. For example, you can use an event to measure when someone loads a page, clicks a link, or completes a purchase, or to measure system behavior, such as when an app crashes or an impression is served.”
https://support.google.com/analytics/answer/9322688?hl=en#zippy=%2Crealtime-report%2Cdebugview-report
Since we did not make any adjustments to the events, Google has automatically categorized interactions with the website into the following:
page_view
user_engagement
scroll
session_start
first_visit
I can see applications for analyzing the content and media visitors engage with on the website. We can gauge interest in a link, video, or document on the website based on the event count.
One type of learning analytics I use frequently is the data from subscriptions the college purchases to give to students. Grammarly is a good example, we can see which students are using it and which aren’t and identify strategies to help those that need strategies for “successful use”.
Another aspect I think is interesting is trying to use qualitative data for learning analytics for a full wholistic picture.
Hi Lowell, thank you for taking time to respond to our teamwork.
I agree with you that incorporating qualitative data in learning analytics could provide a more holistic understanding of student performance and engagement. Not only quantitative data provides valuable metrics, but also qualitative data adds depth by capturing the context, motivations, and experiences that shape students’ learning journeys. They say, it takes two to tango. I could only say the same when it comes to quantitative approach and qualitative approach. A combination of both could lead to more informed and personalized teaching strategies.
In line with your idea of quantitative approach, I would like to suggest exploring the potential of predictive learning analytics tools and applications. For example, tools like Brightspace Analytics, Civitas Learning, or Echo360 can analyze data patterns and identify students who may be at risk of falling behind or facing challenges. I use Brightspace Analytics for work, and I could easily tell student engagement from the synchronous data.
Additionally, it might be interesting to consider the integration of sentiment analysis tools, such as IBM Watson or RapidMiner, to analyze students’ written work. These tools can extract sentiment and emotional indicators, allowing educators to gain insights into students’ course experiences, well-being, and potential areas of improvement.
I hope you find our teamwork helpful.
Hi Predictive Learning Analytics Team,
I really liked this OER and found your experimental activity to be really insightful, it reminded me of when I used to teach through the D2L platform – which has a learning analytics feature embedded in the platform. As a backend user, I could see how frequently the students were visiting the site, how long they were looking at particular documents, at what time and how many times they checked something, if they downloaded the material …etc. Having been a student prior to becoming a TA, it was quite humbling to think that my professors had so much of my web usage available to them. Frankly, I was fairly shocked to see that the information was so easily accessed and displayed. On the one hand, I think that it’s helpful for for me to be able to identify who might be falling behind and not checking on materials, on the other, I also felt like I was invading my student’s privacy. I myself felt a bit violated to know that my professors could have such a level of surveillance on my personal usage and analytics. Up until then, I was under the impression that my data was being gathered to create broad statistics, to identify general learning trends, perhaps within the classroom or in the University. I felt far more comfortable thinking that I was a fraction of a larger data set and that my information was analyzed collectively with other students. I am 99.9% confident that my students, like myself prior, did not know that this information was being collected and shared with their teachers. I can appreciate that when using platforms we should be assuming that our data is being collected and analyzed, to some degree at least, but I think as educators we need to be more transparent and share the HOW and WHY that the data is being collected – especially in school situations. As we continue to move towards data-driven practices, sharing the outputs/outcomes of learning analytics can create informed and self-aware students. Maybe this practice would even give students the critical thinking skills to assess their data privacy outside of school, which is a skill the majority of us lack, and future generations so desperately need.
I think learning analytics has great power to identify patterns that can support or dissuade our practices, and be used in really helpful and informative ways. But, I do not think we are ready to introduce extensive learning analytics to the classroom, because I truly do not think we are prepared enough to face the implications of collecting personal data (especially for minors). Broadly speaking, we have little to no protection against tech corps using (and abusing?) our data. We do not have the policies and the systems in place to ensure that folks are empowered and protected, in fact, we are very very very far away from that. I also know that sounds very dystopia and pessimistic, I also know that these risks are to some degree unavoidable with many apps, technologies, etc.. and also that these risks are already assumed so often in classrooms when using external services and platforms, but its certainly something to keep in mind!
Thanks for an informative and interesting aspect to learning, ie as a measurement tool. What is not so evident are
1. The competence of the teacher or teaching mode ie. what is being fed into the system of learning must also be measured accurately and competently
2. The fact that learning at elementary and high school levels occur in different phases for every participant / student , nor does it stop at that point.
3. Analytics can only project probability but does not appear to consider other aspects of learning.
4. One of the most valid points in successful learning is the amount of vested interest held by the learner in the topic at hand. This is further dependent on how teachers are best
able to introduce material and the methods used to stimulate learning. alongside natural teaching skills.
5. One of the key factors in learning at elementary school levels is concentration ability and brain fog after being in classrooms all day ie. how healthy is the learning environment.
6 Machine learning is not the master of learning or education, rather the concept serves as a complimentary device used to enrich and facilitate learning.
7. Human beings, being individuals from moment of birth are equally able to discern and choose what is of interest to each person or not. Machines / or other AI concepts to date cannot accurately mimic this unique human quality.