| As the use of online learning tools and educational technologies increases, the amount of available data that is produced through these interactions has increased at an exponential rate. Many of the largest companies in the world, such as Google, Amazon and IBM have already incorporated the use of data-mining in order to increase their revenues, identify trends and gain a competitive advantage in their respective industries; however, the use of analytics and data-mining in education has thus far been minimal. |  |
What Are Learning Analytics?
- In the field of education, the term learning analytics refers to:
- “The measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs” (SoLAR).
To learn more about learning analytics, watch the following video: |
Activity #1: What Can Learning Analytics Do for You?In the previous video, it is stated that analytics must start with a question or hypothesis. Related to your own experience in education, what:
through the possible use of learning analytics? Please post your question/problem in the comments section below: |
In thinking about this based on previous comments about engagement and visual learning. I think it would be interesting to compare students engagement and level of interaction based on the User Interface design.
As design thinking increases in technology it would be interesting to see how that changes the valuable interactions students have inside of an e-learning environment and what designs produce the greatest level of engagement and deep thinking.
The question that I would like to answer is “what is the benefit to student learning with a BYOD policy in schools?” Does this policy assist students with inquiry work more or less than the traditional school funded technology policies? I have always been a proponent of BYOD policies, but does this create a learning gap between those with devices and those without? By using analytics, I wonder if this question could be answered?
In education, the problem I would like to solve includes understanding how effective is technological learning in the classroom and to what extent is learning through tablets, computers, and other devices beneficial in comparison with traditional learning models. I have always been a proponent of traditional learning models, and believe that IT integration is important but must be properly used in order to receive the benefits. Most teachers are not aware of how to use technology, and training is still lacking even with new revisions, such as what the OCT is proposing to extend the teaching program to two years to help teachers understand how to teach with technology. I want to see the data, the results, the actual analytics for this. The question I would like to answer is: How effective will the IT training be for teachers, and will the teachers actually implement this in the classroom?
I am working with two colleague to deliver a SOOC (Small, Supported, Open, Online Course). Our goal is retention of learners. We had a 42% retention rate as opposed to approximately 8% for MOOCs. I would use LA to support the design of the course, to see where people lost interest in an activity or when they leave the course. It would also be interesting to provide participants with different pathways through the course and determine which is the most successful. If we apply LA to entrepreneurial activities, they would help better determine the success of an MVP and when to pivot.
Hi Kendra
That sounds like an interesting use of analytics; have you already included any LA algorithms in your design? I would be interested in hearing any experiences you have incorporating the use of LA into your SSOC.
I just started a new job where we are introducing a LMS for required workplace training courses, and our staff are located across various office, some quite remote. Accessing learning materials, spending time to complete them and using the resources within the LMS are items that I predict will be a problem (simply because that has always been a problem for this company). What I think analytics could offer is some insight into the number of people actually using the Learning Management System, how much time they spend on each course, their success levels (based on the time they took to do their course) and any other resources that they access (so what are people finding useful, and what is being under utilized).
I teach online math courses, and I would like to use learning analytics to find out if online practice quizzes improve students understanding of math concepts and their overall performance in the course. These quizzes will be open-book, in which students complete at home, and they serve as a practice and a review. While supervised quizzes encourage students to review various concepts, I also believe online practice quizzes allow students to pinpoint their troubles in the course right on the spot, so they can go over related concepts. If there are solid data to support the hypothesis, then perhaps schools can invest in online quiz tools to help students with their math learning.
On the MOOCs, the reason for dropping out is still under a wide investigation. I would like to know more about Lurkers’, the silent participants’, activities on the courses, and would like to know what the hindrances are to moving on to the next stages. Also, I am interested in the comparison between the participants’ original culture/ languages and their online activities.
At my school I am working intensely on literacy and numeracy skills. We are also using variety of formative and summative assessment techniques for/ of learning. I want to find out how I can use the analytical information to implement successful online assessment program ?
I would be interested to see how I could use analytics to drive all stages of my Math K9-12 teaching practice in an online environment. Investigating its use in diagnostic, formative, and summative assessment could help me identify the needs, strengths and weaknesses of my students as well as provide feedback on validity of questions that I pose in assignments, tests, etc. In addition, I would be interested to see how analytics can be successfully embedded in already available applications (e.g. Gizmos, games, etc.) to provide a better picture of where the students are and where they need to go.
Hi Ana
You raise some interesting questions; however, when starting off with learning analytics it may be better to focus on one type of assessment to narrow things down.
Your final point is also interesting; we didn’t include it in the final version of the challenges page, but one more issue facing learning analytics is the scope of data capture. More specifically, the problem that not all student data is included in traditional sources of educational big data (eg. LMSs), so what other sources of educational data can educators/administrators use to make predictive models more accurate?
I am interested in knowing how to efficiently pull assessment data from application-based quizzes and games. Educators need to be able to easily quantify student performance in an mLearning environment. Edugames, mobile quizzes, etc. should include data gathering/sorting features, or third-party applications that manage data need to integrate with mLearning applications. Quantifying results give mLearning applications greater credibility because the data makes clear the impact they make on the learning process. Assessing the value of the data also duly informs educators and consumers if a particular mLearning application is worth the invested time and money.
I see that a lot of people share interests between Learning Analytics and Online Games/Game Based Learning. It’s not surprising as these online areas would be easier than other environments to obtain data. Perhaps since using games and other apps are new to education we’re interested in the data or success of them.
When thinking about my own experience in education, I would love to discover how Learning Analytics could work with my schools behavior tracking forms for our office referral data. We have been working on our School Improvement Plan but the tracking and collection of data has been extremely time consuming. Another way that I would love to use Learning Analytics in my Guidance practice would be to track courses selection when students register online for grade ten courses. I would be able to provide data for high-school guidance counselors that would help with rough estimates for creating sections for courses.
I never thought of using Learning Analytics for course selection. You could easily map trend analysis for patterns of specific students. While you’re likely aware of which courses are the most popular you could have data to support what type of students enrol in specific courses. I know a lot of work goes into course selection so this could easily help.
I would be interested in knowing the long term analytics of Game Based Learning. It seems to attact alot short term interest from students but does it affect long term change in learning? How sustainable is it over time or is it just a fad that comes and goes with new games?
I like how your interest and question ties into another OER topic we’ve had in this course. It’s nice to see how a variety of these ventures could be tied together in selecting and choosing a program or technology for use in our classrooms.
Also wondering how analytics can be used to help in determining assessment strategies for literacy or augmenting student literacy or achievement upon province wide literacy testing.
Hello Group 10,
Having worked on anti-bullying campaigns and social justice groups, wondering how analytics can help solve or diminish the numbers of bullying cases within a school environment? As well, another possibility of using analytics to determine safety issues for a student body and how to address them.
Hi Leah,
It’s taken me some time to respond to your question as I have to admit I was stumped when I first read. I love the aspect of using analytics to diminish bullying but could not think of any ideas on how to do this off the top of my head. After doing some research I see that analytics has been used on social networking sites in order to help stop cyber bullying before it gets out of hand.
Here is an article about steps Facebook has taken in this subject: http://blogs.sas.com/content/text-mining/2010/10/22/text-analytics-how-to-capture-the-online-bully/
Of course Facebook is a public site and no one should treat someone horribly on a social networking site, but I do see some issues of privacy here as well.
Does anyone else have any ideas on how to use analytics in this case?
The problem I would like investigate is what is the optimum instructional time that maximizes student engagement in secondary schools.
Really interesting question, I’d be really interested to see the results of an investigation like this. Depending on the hypothetical results you would receive what kind of changes would you implement to your instruction?
I would like to explore how Learning Analytics can provide customized information to develop a dropout alert system. High school dropout rates are a challenging problem in some school districts. It seems that this is especially so with distance education.
So the problem I would like to solve is how to apply the LA datasets to learn the behavioural patterns of students who dropped out in the past and current students who show similar patterns. The teacher can then be alerted in time if students are in danger of falling behind or dropping out and intervene in an appropriate manner.
Hi Kristen,
I’ve also encountered this in my school. Enrollment is diminishing at a rapid rate, be good to determine where and how to augment enrollment somehow, especially in high schools.
This would be an interesting trend analysis. I’m sure each student would have a variety of reasons for dropping out, but perhaps through analytics you could detect a similar pattern/signs in order to take any necessary steps for prevention.
What kind of learning analytics, and how it is used here at UBC, primarily for our MET, and other online programs?
It would be nice to have some essential information on LA at UBC, and to get the suggestions for in house learning analytics.
What does UBC do to timely and proactively support student and faculty success based on those LA data?
Thanks in advance to whoever can provide information on LA@UBC.
I would like to explore success rates for 10-12 online courses and/or how we can use data to make changes to our processes prior to the end of a school year so we aren’t making changes a year too late for some students.
Great question. By the time we’re able to identify a solution to successful learning there’s very little time to it develop over the short school year or semester. One of the possibilities of learning analytics – and we have to be careful that we don’t cross any privacy or ethical concerns – is that if a pervious teacher identifies an application or personalized learning technique that allows a student to be successful in their learning they can share that idea with a teacher who receives this student the following year. In this way we could spend less time experimenting and more time implementing successful learning strategies for students.
HI Chris – I like this question, I have a similar one! My district is pushing online education b/c our schools are so small we can’t offer everything, all the time. I know that some students excel online, but I often wonder about science and math courses. Are students who take Chem 12 online, really as prepared for a first year university chem course, as those who take Chem 12 in class. I would doubt it, but I’d like some hard number via analytics to prove my suspicions.
Data analytics could be used to collect purposeful data on almost anything. A few of the things that I might collect and analyze data on in my work environment currently might be:
-student use of a particular website or resource; how often are they visiting? how are they accessing it? where are they going after they access it?
-student technology use; what types of devices are students using? what apps are they using? how often are they using? for what purpose?
-data on learning performance is flexible learning spaces
That is just to name a few.
I like the range of questions that you have here. There is a balance between learning resources and the medium in which students interact with.
I think that my question I would like to answer is how useful or how much is online formative assessment actually used. If we put practice quizzes and other checks for students and leave it up to them to use, will they use it and does it really help them?
Absolutely. You could also check to see how students fair on the more difficult questions vs the easier questions. Are they continually getting the same type of questions wrong or right?
Additionally which students are the ones who are participating in the practice quizzes? Are they likely to be higher achievement students or vice versa?
This is a great question!
I would like to know the correlation between the number of downloads for our handout materials and the level of competency in our application.
Hi Riea,
I think, for a simplistic look at your question even a free analytical solution like Google analytics could work if your application is hosted on line. Not only can it track the number of downloads, it can also track the flow of users after they download the handout material. Tests could be posted to check the proficiency level of students and GA could track how many completed them successfully. It does not give you a direct correlation, but you can pretty much make sense of the data.
For a better and more scientific understanding, an analytic product which could track not just the number of downloads but also indicate ‘who’ downloaded the material and ‘who’ completed the proficiency test successfully would be needed.
Hi Sayali,
How does analytics work wrt this particular area. Is it a system set up by administrators to look at the data and numbers, like an algorithmic system of trends?
There are mixed perceptions of Learning Analytics in professional circles.
I believe that Learning Analytics has its good and validated purpose in analytical support of teaching and learning (education), but the role of LA should not be exaggerated.
A s a result of latest research and LA, I would like to see the newest findings, on:
1. Usability (advantages/disadvantages, weak/strong areas, etc. of fully online courses vs. F2F and blended learning delivery.
2. What areas in institutional eLearning need to improve for best eLearning outcomes, etc.
Please take a look at the AU excellent qualitative evaluation on the evolution of the LA tool “LOCO-Analyst”: https://files.semtech.athabascau.ca/public/TRs/TR-SemTech-11012011.pdf
Hi Milorad,
I like how you emphasized “support” in your comment. Learning Analytics is a tool, however we need to be careful on focusing on it too much.
Also thank you for sharing that document. It was nice to see a study that looked at the progression of a tool. I also enjoyed that they focused on multiple factors in engagement and achievement rather than just a few.
Hi Milorad,
I agree that Learning Analytics is like any other technology and should be treated as just that. Just as (according to me) e-learning cannot fully replace the role and value of an experienced teacher actively supervising the process, Learning Analytics can only assist in making decisions or highlights trends which are easier to miss given a huge data set.
I find LA more necessary in cases of distance learning or where the student population is huge.
I would like to see analytics used to answer questions about the success of students identified with learning disabilities.
Hi Brenden,
learning analytics for students with learning disabilities definitely seems a good idea. For an open-ended question like this, the starting point could be something which loosely termed as black box analytics. It involves running machine learning algorithms against a large data set of such students to see what pops up. Any trends or patterns can that can be seen could then be investigated using the predictive and adaptive learning analytics.
I would be interested to know if students write more in response to discussion questions when online as compared to writing in a journal. Other relevant data related to levels of engagement and the impact on literacy skills would also be of interest.
I share your interest in this topic. I had this discussion with my students a couple months ago about provincial exams and if they would prefer to write it using an electronic exam or with the old fashioned pen and paper. To my surprise they were divided on this subject.
Some liked the idea of being able to get their thoughts out quicker and could play around with where they wanted their text.
While others enjoyed taking their time and the idea that doing their exam by hand does not have the pressure to have a final “good copy”, but using pen it’s viewed as a good rough draft.
However, preference does not always reflect success. It would be interesting to see if engagement level for writing increased as a result of online communities or discussion. I think there are a number of factors you could look at in terms of length and quality of response. You could also identify which subjects/topics they are likely to engage more with.
Interesting video! I know that a lot of attention has drawn to google – analytics, and analyzing surfing patterns of people on the web, but I hadn’t given much thought to how analytics could be used in my school setting. One question I have is – how can analytics be used to make informed decisions on the purchasing of technology in a school setting?
I never thought about how it can be used for purchasing technology. Thank you for raising that point. I think I got so focused on how learning analytics can help with trend analysis and improved student learning that I forgot about the programs that help make that happen.
Often times we might be quick to purchase a specific technological function due to its cost and popularity but we might be unsure of its potential learning outcomes. Having a couple test items with learning analytics might help to identify the most effective and useful programs/tools to improve student learning.
I would love to know if using online games in specific subjects increases student engagement levels and/or achievement levels.
A great question. This would be easy to collect data – you would be able to see how often students sign into their gaming programs, how long they play the game per each login, how quickly they move between levels, and of course you could compare achievement results from before they used the game to after.
I would also be interested in this area. More specifically, in the relationship between commercial video games and student achievement.
Great question! There is a lot of research on this topic. Here is another article that talks about the implications of game based learning.
Empirical evidence within this study and others have shown that students have;
1. Increased engagement and motivation
2.Preparation for future careers
3.Improve self-efficacy
4. Improves cognitive skills and literacy
5. Improved academic achievement
http://linked.eun.org/c/document_library/get_file?p_l_id=22220&folderId=24664&name=DLFE-912.pdf
Interesting question. Just to add, that there has been done lots of research on that topic, and it is definitely proven that gaming in general is beneficial for strengthening a range of cognitive skills such as spatial navigation, reasoning, memory and perception, etc. In my humble opinion, the LA may be useful to determine which online game is more useful in developing certain skills, relative to increasing student engagement levels and/or achievement levels.
Please take a look at the following European research article here:
http://www.psychologytoday.com/blog/media-spotlight/201402/are-there-benefits-in-playing-video-games
I would like to know the engagement level for: Teacher to Student, Teacher to Content, Student to Content, and Content to Content
I am hoping to improve engagement levels by trying different approaches and keeping an eye on the above engagement indicator
Interesting question. What kind of environment are you interesting in learning about? An online environment? Or face-to-face?
It is easy to reflect on engagement within a class just by observing those who are paying attention or raising their hands to answer questions. However, these might not always be true factors of engagement. Having visual/big data might help to solve these discrepancies.
Great and really interesting question! As non-teacher and creative, I have a few questions that relate to a design venture for which I’ve only got small-scale data, analytic would be wonderful to provide ‘bigger’ or larger scale sets of data: 1 ) Do visual arts ‘creatives’ prefer e-learning content and architecture that is visually organized or has a larger number of accompanying visuals within the content? 2 ) Do a greater majority of visual arts students prefer highly visual environments to more text-based ones? 3 ) Does the aspect of visual communication in any way impact or correlate with the willingness for post-secondary students to spend longer times immersed in e-learning environments?
The problem I would like to explore would be retention rates in online K-12 courses
You could receive quite a bit of data in regards to retention rates in online courses. It is easy to obtain information on students who enrol in another course in the following years, how often they sign in, how long they work on each assignment.
Another aspect you would be able to explore is the success of students who login in more than others or vice versa. There is always the ability to have a correlation between retention and success.
Please make sure to go review a program that could be useful to you in obtaining this data/information.