Learning Analytics (LA): In a Nutshell
Learning analytics evaluates data so that stakeholders can determine how to improve learning outcomes measured by grades, retention, or completion. LA collects and analyzes student data in order to look for correlations between student activities and learning outcomes. However, this can be a challenging process because of the various expert perspectives regarding defining learning analytics.
A Few Definitions of Learning Analytics
Currently, there are many definitions of learning analytics because there is no expert agreement due to differences in perspectives regarding processes of implementation and levels of success. Below are a few definitions to consider:
– LAK ‘11 states that LA is “the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs.”
– EDUCAUSE’s Next Generation learning initiative states that LA is “the use of data and models to predict student progress and performance, and the ability to act on that information.”
– Elias (2011) describes LA as “an emerging field in which sophisticated analytic tools are used to improve learning and education.”
– Siemens (2010) views LA as “the use of intelligent data, learner-produced data, and analysis models to discover information and social connections, and to predict and advise on learning.”
– Johnson et al. (2011) defines LA as “the interpretation of a wide range of data produced by and gathered on behalf of students in order to assess academic progress, predict future performance, and spot potential issues.”
Learning Analytics: In Conversation
This video is an introduction to learning analytics pertaining to educational settings. The conversation takes the spirit of the definitions into consideration.
Model of Learning Analytics: The Spirit of the Definitions at Play
Click Image to Enlarge
(http://www.greller.eu/wordpress/?p=1467)
This model is designed to assist the stakeholders in the planning for learning analytics in order to measure learning effectiveness. This model is also designed to help stakeholders remain focused on the learning issues that are most important to them. This model is not the only approach or framework that works to measure learning effectiveness, but it does reflect the spirit of the definitions of learning analytics, which is useful to the stakeholders.
Questions to Consider (Weighing the Effectiveness of Learning Analytics)
– How do activities in learning environments correlate to learner success?
– How can learning analytics be used to identify and to promote effective learning?
– What insights into learning analytics data are most useful, and who should be using them?
– How do learners make decisions about how to spend their limited time and attention?
– What types of frameworks promote learning?
Objectives
– Acquire knowledge of learning analytics
– Recognize the complexities involved in learning analytics
– Evaluate the effectiveness of learning analytics
– Understand learning analytics in education
We would like to thank you for taking the time to participate. Enjoy!
Huge upside in learning or a huge downside in social control?
As the world embraces data mining we are reminded of its peril by recent disclosures of data mining by NSA in the USA as exposed by Edward Snowden.
Are we on a path of self-destruction as we standardize our citizenship though data based programmed education? It smacks of eugenics with the aim of enhancing humanity, but while doing so making us vulnerable to judgement errors on a massive scale, i.e. the holocaust which was justified by referencing Darwinian science data.
Should we embrace diversity instead as a model where local conditions ( and yes, local data ) influence outcomes, not remote data based decisions based on ‘big data’. Is it time to unplug ourselves from the internet to deprive ‘big data’ gatherers of our data???
Hi David,
As a primary grade teacher, I do not work directly with big data or with learning analytics, but I suspect that learning analytics is used in our public education system. For example, I suspect that learning analytics is used from standardized assessments like the Foundation Skills Assessments (FSAs) in Grades 4 and 7. I think that learning analytics could have value if it is implemented with the intention of improving student achievement and student success. In all, I suspect that the Web and the Internet are here to stay, and learning analytics and big data will continue to prevail for years to come, regardless of whether we agree with it or not, or whether we like it or not.
In my capacity I have very little to do with LA, though I believe our marketing side use these to determine retention stats. In my area, the only time I would use this is in the numbers to do with an online programme, known as grammarly. It gives me the data on usage.
Hi Dana,
Learning analytics, as assisting data driven adaptations, has an excellent prospects of improving delivery, and I’m all for it. I see ‘learning analytics’ as mostly based on local data not “big data” or “non of your business” data.
‘Big data’ needs to be anonymous ( by legislation in my mind ) but has a place in developing content delivery in a generic sense.
Hi David,
Yes, I think that learning analytics has its place and value in education as long as it is intended to improve student learning, achievement, and success. Also, I understand that ‘big data’ can be used to address problems in education systems. There will be implications to this, but big data could be valuable if it is used to improve student learning. In the end, all stakeholders in education are looking for better ways to improve student learning. Learning analytics and big data are just the latest tactics.
An item which I’ve noticed is that a focus on numbers will often drive a greater focus on numbers. It’s intention can be to improve student learning, but it seems that often the numbers lose their meanings and take on their own roles as goals.
I’ve seen it with the larger tests, such as the EQAO (the provincial standardized testing for Ontario), but we experience it with smaller “data collection” tests as well. We use a product called “CASI” put out by Nelson, which is designed to assess students’ reading comprehension. We tracked the info on large charts, looking for weak questions/students. Initially, I found that as we taught students the “proper” way to answer the questions, we killed the innovation which came from higher-level students. Exceptional answers rarely follow a formula. While we were reaching higher numbers of students attaining standard, we were also strangling original thinking.
And we weren’t teaching reading. We were teaching how to answer a specific type of question.
In a way, I think that elements of gamification come into play here as well. The numbers become the badges. You get bragging rights between classes/schools/boards. Although proper teaching can be a route to increased scores, there are often “cheats” which focus more on the letter than the spirit of the underlying learning.
Ideally, it feels like it should be a powerful tool. It feels that in practice, sometimes the tool overwhelms the objective.
Hi John,
Innovative, exceptional, or original thinking does not have to be necessarily compromised because of formulas. For example, we teach that a paragraph is in three parts: an opening sentence, supporting details, and a concluding sentence. But, within this formula, students can be as creative, exceptional, and innovative as they want with their thinking, words, and sentences as long as they are clear. We teach paragraph writing as a formula. Otherwise, students will write as a single stream of consciousness.
Also, I think that there are proper ways to answer questions. Otherwise, students do not use complete sentences, or students do not answer the question directly. As a primary grade teacher, I am always teaching students sequencing, ordering, classifying, etc. These skills are needed to learn the ABCs and the 123s, etc. Primary grade teachers are always using formulaic methods. Otherwise, students do not learn that the alphabet and numbers are in sequence.
I think that learning analytics and data could be useful in guiding and encouraging effective and efficient teaching. Numbers can reveal a lot. It’s just that we might not like what they reveal. Also, learning analytics and big data are relatively new tactics in education. Both deserve a fair amount time to get better.
For me I would expand the idea that “often the numbers lose meaning” and that ” the tool overwhelms the objective”. I’ve found that one of the main challenges associated with any type of analytics in education is that most practitioners will focus on the most easily quantifiable measures foremost, and sometimes only on those. As has been stated already, numbers feel comfortable, and they feel objective. So we focus on those. This is a challenge I see a lot as well in the world of business being a people function professional. The world of business is dominated by the ‘numbers’ people, and so is the world of ‘big data’ regardless of where the big data analysis is happening. I believe that narrow focus on just the most easily quantifiable aspect is what drives assessment and analysis strategies to go astray, and as you say, strangle original thinking and learning in the case of when it’s applied to education (but the same is true when it’s applied to almost anything).
I think the way forward is to move towards more holistic assessment and analysis frameworks when working with big data, incorporating elements that are harder to quantify. A holistic analysis framework that incorporates quantitative metrics, as well as qualitative metrics can help ensure that the analytical tools being used are both driving the achievement of learning standards that are needed (as bmehregani said, we need learners to be able to write a paragraph properly!), without only rewarding those that learn ‘how to answer the question’ instead.
I think that this is a balanced approached of incorporating LA into education while using big data in the background. Both your posts paint a promising picture of the future of education where teachers could help improve student achievement and success, which is what all stakeholders in education want. The current traditional model of teaching does not give teachers much one-on-one support to students. Even with a roving educational assistant in the classroom, there is only so much support that can be done in a 40 to 60 minute period.
I agree, from a government’s perspective or the perspective of a large organization or institution big data can be very useful in identifying and analyzing structural inefficiencies. School Boards would be an example of social institutions that could draw useful conclusions from such data when organizing its resources and/or allocating funding.
Hi David,
I also think that big data could be used to draw conclusions when looking to make education more efficient, more relevant, and more effective in order to improve student learning. After all, there are a lot of learning/teaching inefficiencies and ineffectiveness at the school level. It is just a matter of time before governments consider using big data as a tactic to streamline education and then impose its conclusions unto the school districts. In the end, only a few educators will recognize that big data will have entered into education.
Hi John,
Your observation on how numbers can overwhelm the stated objectives speaks to the issue of how we need to be careful in being too eager to buy into what outside agencies have prescribed as a solution based on their big data observations.
Big data as used by publishing houses can be far removed from what our individual students may need, and the motivation of publishing houses is, I would think, often to package for a sale without necessarily meeting or testing for their own stated objectives.
I am still advocating local student data analytics as most useful in individualizing student progress, with the teacher drawing conclusions that might then appropriately be discussed in a School Based Team setting.
Governments and School district with their superior access to big data have over the years not had a particularly stellar track record when making decisions on policy or making decisions for teachers in the classroom.
I’m still learning about the field, so making a judgment call on the potential of learning analytics may be a bit out of my league. The best I can do is liken it to the recent statistical analysis trend in sports management. Not to say that learning and playing sports are one in the same, but the recent shift to using analytics in place of traditional perception-based decision making bears some resemblance. As a sports fan, the “statistical revolution” has been moderately successful. But statistics have not completely replaced traditional scouting and coaching methods, as they have been met with only marginal success. Purists see statistical analysis as overrated, while statistical analysists preach as if statistics are some sort of Rosetta Stone of truth in sports. The truth, of course, is somewhere in the middle.
I feel like learning analytics has similar transformative potential. Traditionalists will look at it as invasive, and a threat to traditional learning values. Supporters will see it as a giant step towards the refinement and customization of learning, a way to situationally affect potential learning outcomes. Again, the truth will likely lie somewhere in the middle. Just as advanced statistics is an additional tool for the modern coach/manager, so too will Learning Analytics will be another tool at the disposal of the modern teacher/developer/administrator. If used strategically it is likely to produce positive outcomes.
Hi Brendan,
Nicely stated! Regarding learning analytics, “Somewhere in the middle” is a sensible, reasonable, and balanced perspective unlike polar viewpoints, which usually end up in futile debate, bearing no or minimal results.
Learning analytics is a learning curve for all stakeholders in education. Time will tell what works and what does not work, like everything else in life.
My initial reaction to Learning Analytics was that it sounds very much like a Chinese way of learning. Study a topic, test your knowledge, move to the next level. Chinese students would excel at this form of learning. I don’t want to come across as stereotypical or racial. I taught in China and have experienced first-hand Chinese methods and styles of learning. The emphasis was not on the process on learning, but they would use strategies to memorize content to be able to move to the next level. Even in teaching them a language, a whole language/immersion approach was confusing for the students. They wanted grammar rules, formulas to build sentences. LA would be a viable venture if a platform was made to teach English to the Chinese, the millions of Chinese students would excel and learn the language quickly. The creator would be rich. Although I am not sure if you could say that at the end of the units of study that a student would be fluent in reading, writing and speaking English. We have some collegues in this cohort that are currently teaching in China. What do you think?
The BC k-12 school system, reflected in the new curriculum, is shifting to making the process of learning more important. Units of study have been decreased, teachers are given more autonomy in creating authentic learning rather than focusing on a list of learning outcomes. We see this same shift in the assessment model. Assessment for learning instead of only assessment of learning is the model the new curriculum also has adopted.
As a venture, I don’t think that it is going to” jump to the top of the class” in K-12 school, but I listened to a debate with university profs, they were very positive about the capabilities of learning analytics. It is a tool to increase knowledge. The professors were debating the idea of learning, what learning really is. Would a data driven program be able to assess learning. On the onther side of the coin, do any of our tests assess learning? Most multiple choice tests measure short term information recall. It was a very interesting debate. The subtitle for the debate is “Learning analytics (aka learner relationship management) has been highly touted as powerful application to promote student course completion and graduation. But are course completion and graduation the same as “learning”?” Here is the link http://www.educause.edu/eli/events/eli-annual-meeting/2013/2013/debate-club-wheres-learning-learning-analytics
Do we in the classroom do a better job of assessing than an intuitive data driven machine?
I think that the data from the Foundation Skills Assessments in Grades 4 and 7 are used in learning analytical ways. Sadly, the assessments are used mostly for ranking, but the data could be used to improve student learning, achievement, and success. LA is not the bad guy. It is the users of the data. Rather than use the data to advance teaching practice and learning processes, the data is used to advance ranking among schools, which is not in the best interest of the students. I think that LA could be used as a tool to teach and to learn more effectively and more efficiently, and make student learning more relevant to students. For teachers to do a better job of assessing student learning than a machine means that teachers have to craft the right questions, be well experienced in assessment for learning, and knowledgeable in assessment implementation and analysis. I am not sure if many or most teachers are there yet even though many or most might think otherwise.
The most powerful example that I’ve experienced so far when it comes to the potential of how learning analytics could transform education come from the excellent book by Clayton Christensen (the guy that coined ‘disruptive innovation), Disrupting Class (http://www.amazon.com/Disrupting-Class-Disruptive-Innovation-Change/dp/0071592067).
In the book Christensen paints the following scene : learners are working away at their integrated desk terminal in a classroom that has a screen and any materials they might need. The software on the terminal is gamified, and adjusts the learning activities and contents to the student’s ability level. The system provides the teacher with a real-time dashboard of how every learner is doing. The system can determine from the learner’s responses (using big data techniques) what the students is most likely struggling with. The system has a variety of learning activities, and explanations for each possible challenge a student might struggle with. The system goes methodically through different learning activities that take different approaches, and that are built for learners with different learning styles. If the system goes through a number of them, and the student is still stuck, it alerts the teacher. The teacher then can go and work directly with the student while already having seen all of the student’s attempts on the teacher dashboard and can provide that personalized one on one support. All of the other learners continue to learn.
This incredibly powerful tool would be driven by big data, and it highlights what learning analytics could one day do: free the teacher to work with learners one on one, to coach the student, to help the student build meaning, instead of the current model that traps teachers on a stage where they need to deliver content and learning activities to the students.
Of course a hybrid would be ideal since some learning activities would require intra-personal components, but one can see how powerful analytics can be from the scene Christensen paints in his book.
Wow, thank-you for sharing this! I think that every student should have his or her own computer with Internet connection so that Web-based software and applications are available and accessible at any time. In fact, there are many stories about classrooms having iPads that are connected to the Web through Wi-Fi. Many of these classrooms are at the primary grade level.
In all, it is having the teacher as a guide on the side rather than a sage on stage. That is, the teacher acts as a facilitator and a supporter, providing individualized attention when needed.
Welcome to learning analytics (and to big data) in education!
Thank you for describing how LA can be used in the classroom. I have a much better understanding of it now. I think the difficulty in LA is not the collection of data or the analysis, but the utilization of the data. It may be able to tell you if a student is successful or not for certain learning objectives but does it offer a method to help the student or the teacher once a problem area is identified?
My only experience with LA is during surgical simulation. I have used a virtual simulator to perform surgery which has LA built in. It measures my path length, accuracy, efficiency, but it is up to me to interpret this analysis and figure out what I need to do to improve. I would find it more helpful if it would guide me through tasks or give me pointers for improvement. Perhaps this is only a limitation for this particular simulator and LA systems used in other educational circumstances are more advanced….
I completely agree with David that big data may not necessarily be the way to go. Many of the problems with the current education system have arisen from the ‘one size fits all’ mentality. Many of the best learning experiences that I have discovered or experienced have resulted from unique local opportunities or locally relevant adaptations to learning outcomes and environments.
This does not mean that I am dismissing all potential values of learning analytics but I think we have to be very careful and selective about who is doing the analyzing, as well as what the purpose is and how it is used. In general it seems that the further the analysis is from the individual student, the greater the chance of it being irrelevant. I can see how big data may be necessary to produce algorithms for tutoring programs, etc. but these programs must be very adaptable to individual users in order to be effective.
The public school system itself is one size fits all, but from within that system, there are many unique experiences and unique opportunities. Within the public school system, every school in every school district offers unique experiences and opportunities. In all, there has to be an overarching standard or system that acts as a springboard to local differences. Otherwise, everybody everywhere would be doing something different, which would not help advance the collective. This is where LA and big data come in. Both tools could offer any education system an opportunity to advance learning and teaching. It is not the tool, but the users of the tool who would make it irrelevant to students.
As a learning support teacher I can see a lot of value in what Alex is talking about. Being able to set students up with learning activities that automatically adapt/modify to meet their needs in real time is an amazing advantage for teaching. If educators are immediately able to determine when students are struggling and require 1-1 support, students are much more likely to be successful. LA could also provide students with learning activities that are much more closely connected to their interests and learning strengths.
I also agree with the caution around big data. Big data does not necessarily explain what is happening for each individual learner. Making sweeping changes as the result of this kind of data will not necessarily result in learning improvements for all students. Personally, I would like access to LA data for my students in real time so that I can make decisions about support/direction individual students need as soon as possible. I see LA more as a tool to help guide my decision making process for my practice.
My concern with LA would be that educators begin to rely on the data alone and not other social/emotional or academic factors that may be influencing learning. It can be very challenging to determine how much these sorts of factors are affecting learning. It is unlikely that LA will be able to glean that type of personal information from students. In these situations there is no substitute for the teacher-student relationship. Used in partnership with these sorts of relationship building strategies, LA can provide otherwise useful data to help educators make informed decisions on behalf of their students.
Nicely stated! As a former LST and current classroom teacher, I would consider LA only to support student learning, not replace it with teacher-student relationships. I would never rely solely LA, but I would consider the data as formative. If LA happens to be a productive tool/strategy to improve teaching and learning, I would pay attention to the data. A balanced perspective gives room for advancing education systems as well as improving student learning and improving teaching practices.
Like others, I think the biggest problem with learning analytics is not collecting the data, it’s making sense of it, and then it’s what you do with it. I have an Etsy shop, which gives me a lot of data on how many people view my shop, where they come from, how long they spend on the page, etc. And, I’ve just signed up for the more intense Google Analytics service, so there’s no question I have information about traffic to my site and what people are doing on it. Although I can see trends in the data, it hasn’t translated into any meaningful changes to my approach…yet. There’s a super steep learning curve if you want to actually do something with the data. But of course, this is a sales context and I’m dealing with an unknown pool of potential buyers. In an educational context, your students are a known entity and any data about their learning can be put to more immediate and targeted use.