Performance-Based Funding
|
Transparency & Accountability
|
Improving Student Outcomes
|
| Will future educational funding be determined through analytic means? If so, what are the implications? |
Who will have access to data, and who will be responsible for the data? |
Will analytics actually lead to improved student outcomes? How will these outcomes be defined and measured? |
Misuse of Analytics
|
Privacy & Ethical Issues
|
Allocation of Resources
|
| What effects could learning analytics have if the data or findings are misused or misrepresented? |
What are some of the privacy and ethical issues in regards to learning analytics? How can students’ privacy be protected? |
If educational resources are allocated based on analytics, what effects could incorrect data have on educational systems? |
I think that the primary concern is in the contraction and interpretation of the data. I understand that analytics really is providing basis for a statistical mean understanding of what is happening with regards to making decisions about the next direction to take. Relying so heavily on statistical approaches and analytics systems could create an educational landscape where we have a new ideology of how to define the ‘perfect student’. That is to perpetuate one type of educational belief and or opinion(s) onto society as whole and potentially create an A standard that is one dimensional because of wealth of data driven educational ideologies. This isn’t to say that learning analytics shouldn’t be used, but there should by multiple views and perspectives that are chosen to drive data forward. The problem inside the technology space is that data in various fields is often controlled by one or two major players. If this happens to education, we could face fundamental problems with data analytic theory and epistemological views on education.
The most disconcerting part is performance-based funding, since you cannot determine how much a school should be allotted based on achievers or underachievers. Everyone is truly different, and everyone will learn differently. If one group of students in one school is faltering, and another group is succeeding, who is to blame, the teacher, the school, the children, the parents? There are several factors in allocating resources, and I agree with Courtney on this one, in that you cannot choose a school, since awarding both would have equal value. I think equal distribution is the only fair route, and despite the marks, you need to provide everyone with the same opportunities
For the sake of clarification we have updated the term “Misrepresentation” to “Misuse” of analytics as suggested by Bobbi K. as misuse conveys a broader category of challenges regarding analytics. Bobbi also provided a reference to an Educause study “Analytics in Higher Education: Benefits, Barriers, Progress and Recommendations”
http://net.educause.edu/ir/library/pdf/ERS1207/ers1207.pdf (Page 15)
I am worried about all of the above issues, but I tend to agree with Kendra. I want to know who has my data and who actually owns it? I hate the idea of a company somewhere owning the information that I entered about my students. Who is stopping them from using it for something that I feel is unethical? Will I always have access to that data. If I don’t really own it, I can’t expect to have unrestricted access to it.
James.
For me privacy is the big issue who owns my data? Do I? Can I get it back please?
Did the first three men in the video answer the question?
I chose Privacy & Ethical Issues but I think it’s closely tied with Transparency and Accountability. With privacy issues in healthcare being in the forefront right now, I think privacy/ethical issues in education will become an interesting topic to follow. What I find interesting is the question of who owns the data and for how long, if it is not the student does the data have to be kept?
Absolutely I think that is a big concern for everyone. It would be important to read the contract with specific learning analytic programs and their stance on ownership of data and if it can be reproduced/reused without permission from the teacher/student/parent.
I think privacy and ethics are are the greatest issues. How information is collected and how it is used needs to be transparent especially when it can include the information of students. Many students are living in very unique circumstances where seemingly positive and useful information may actually be detrimental. The simple reporting of attendance statistics has caused a lot of stress for students I have known living in foster care.
While all categories are important and need to be reflected in the successful launch of an analytic tool, I believe that improving student outcomes is most concerning. No one will invest into an analytic tool that does not guarantee to improve student outcomes. I agree that security is a very important factor in customer satisfaction, but the tool’s influence on learning outcomes is what makes the product appealing. I need to know exactly how an analytic tool is going to benefit students, or else there is no premise to invest. Security should be standard and analytic tools have no place in the education technology market if they have the potential to do more harm than good. However, it is difficult to assess the usefulness of an analytic tool in a particular learning environment without testing it, so I require detailed information about what it offers before I check if its providing what it should.
I find the issue of data protection and privacy as an extremely important and most challenging issues regarding the use of analytic in education. This can cause learners and teachers to question both the immediate security of their data and who may eventually access information about their knowledge or competencies. These issues are gaining interest and call for greater transparency within the sector.
I think that all the above issues are important when looking at the future of learning analytics, but at its the core lies improving student outcomes. This field ought to be really well developed, accurate. and valid if we are to start looking at Learning Analytics (LA) as something of value. That being said, LA needs to be well integrated in different contexts and customizable to user’s needs which could be a real challenge. Also currently available programs that offer LA feature need to be seamlessly integrated within the “teacher’s” curriculum and within existing systems in order to provide more accurate rather than segmented picture of measuring student outcomes. This is of importance if we are talking about differentiated and varied instruction. LA framework needs to be accessible at any given point in time and over time in order to provide more meaningful feedback to both teachers and students.
That’s a good point Ana! It makes one question how Learning Analytics would define and measure student outcomes in the context of differentiated instruction.
Transparency &Accountability associated with privacy issues are the most concerns to me. Recently, in Japan, there was a big personal information data breach from the private educational institutions. In the case, at least 7 million elementary and middle school students’ data was leaked from an engineer, and sold to the business competitors.
The more learning analytics gain their power in the educational institutions, the more the risk of misuses increases. Having and managing analytic systems involve more personnel who are not directly related to education or pedagogy. The incidents taught us the critical problems in the ethics of those wider personnel around educational data.
We had a similar situation in Surrey this past June but it did not involve a third party. In the midst of our strike an administrator e-mailed 250 parents of grade 12 students an attachment of their names, student numbers and grades.
The e-mail was intended to send information about the strike but a wrong attachment was sent out. I believe this just shows how easy it can be to breach privacy without intending to.
I wonder what kind of LA is currently used at UBC, and how well are those 6 challenges addressed.
Does anybody have any information on that?
Thanks in advance.
My concerns are also related to the privacy and ethical issues in regards to learning analytics data.
For best protection, there are many questions and issues, that need to be strictly defined, such as:
1. Which built-in LA policies, tools and instruments will guaranty student’s privacy at the institution, and what about the access rights and responsibilities, data ownership and use policies during the study, and after the student graduate?
2. Under what circumstances, the LA data might be shared with government agencies, police, etc.?
3. Who has the right and obligation to guard data, and how much ownership and the decision power does the student have on the data collected?
4. May the student request that the LA data be surrendered or destroyed, after student finishes at the institution?
5. May/should the student fully or partially opt-out from the LA data collection?
All those questions/issues have to be well addressed to ensure full privacy protection and avoid ethical issues.
Regards, Milorad
Thanks for the citation, Sayali. I didn’t take ETEC 500, and am going to look up the article. Ecological ethics is an interesting concept. We touched on the ecological aspects of ed tech in ETEC 511. I like tying the different strands of what we covered in the MET courses together.
Like most of us, I’m concerned about student privacy and the ethical issues regarding learning analytics, for all of the same reasons. I’m also concerned about not only the misrepresentation of the data, but the misuse of the data. Teachers don’t like when you treat their students as mere data. They don’t like when you say that data is evidence to your success or failure as a teacher. Just look at the reactions from schools each time the Fraser Institute posts their rankings. There is the danger that some could misuse the information gained from learning analytics to further a political agenda or to label teachers, students, or communities
I really like your point here. Often times we can easily get caught up in the idea of hard data that we forget that the data is a result of people. I’ve really enjoyed a lot of comments so far this week that highlight the importance of learning analytics as “support” to our decisions.
I see that 41.67% including me have voted Ethical and Privacy Issues as our biggest concerns. It seems to me that inherent in the use of learning analytics is the assumption that knowledge of a student’s behavior is advantageous for the student, instructor, and educational institution. I don’t want to challenge the educational purpose of learning analytics or how LA can benefit the learner and the retention and success rate of the institution. However, I remember being surprised and uncomfortable when I developed a Moodle library course for undergraduates in 2011 and learned that there were built-in features that could track their every activity (which I did not use).
I’m not sure if there are frameworks that address these ethical issues in “data-driven” higher education institutions. Earlier today I decided to read further on this topic and came across an article by Slade, S., & and Prinsloo, P. (2013). Learning Analytics: Ethical Issues and Dilemmas. American Behavioral Scientist, 57(10), 1510-1529.
The authors propose that learning analytics should engage students as collaborators and not as mere recipients of interventions and services. There should be more transparency. Not only should students provide informed consent regarding the collection, use, and storage of data, but they should also voluntarily collaborate in providing data and access to data to allow learning analytics to serve their learning and development, and not just the efficiency of institutional profiling and interventions. In sum, the authors propose a “student-centric,” as opposed to an “intervention-centric,” approach to learning analytics (Slade & Prinsloo, 2013). I think K-12 schools and higher ed need to have some frameworks in place to guide the use of learning analytics.
A very good point Kristen !
We had a similar article on ethical issues for consideration in the ETEC 500 course on Educational Research. Flinders, David J. (1992). In search of ethical guidance: constructing a basis for dialogue. Qualitative Studies in Education, 5(2), 101-115.
The authors classify ethics into 4 types – Utilitarian (appraise moral reasoning on the basis of its positive and negative consequences.), Deontological (research procedures conform to accepted codes of behavior like informed consent etc.), Relational (research be informed primarily by respect and consideration for those we seek to understand) and Ecological (place an emphasis on the interdependent nature of relationships, striving to situate moral decisions within their broadest possible context. You involve the subjects in research and get concerned in their wellbeing, you become an active participant in their betterment)
As in research, so in learning analytics, I think the more we move towards Ecological Ethics, the better it would be for all the stakeholders.
I think privacy and ethical issues will be the biggest challenge for Learning Analytics. Questions around ownership of the data, protection of individuals and as has been mentioned above this becomes especially important when dealing with elementary and secondary school students. This is an incredibly difficult question as the data that can be derived form learning analytics can be invaluable to informing our decisions around student learning and directions institutions should move in but we need to tread very carefully as we move forward in this brave new world.
Chris
I agree Chris,
I see this concern popping out every now and then since the start of the group presentations – in case of Apps, Cloud learning and now learning analytics. I think in an increasingly connected world, any concerns about privacy need to be taken very seriously.
Looking at Activity #4 I have to say that Privacy and Ethical issues are my biggest concerns regarding challenges for Learning Analytics. When I took the pool on the side of the screen and saw the results I wasn’t surprised that these issues had the highest results by our class. The first thing that comes to my mind is maintaining student confidentiality with trust, accountability and also transparency. How can the use of this application guarantee all of these aspects? Who owns the student data? What happens if the student transfers, what is done with the data? I love analytics but I realize that I have many more questions as I learn more about the issue.
Great points. One of my biggest concerns with this aspect is the point you raised about ownership. Who owns the data collected? What implications are put in place for protection of student data? If we are going to use learning analytics in our classroom there would likely need to be permission slips signed and parents would need to be informed. Would the school put forth a policy?
These are so closely intertwined that it is difficult to seperate as some others have already mentioned. I think for my personal situation in an elelmentary position, privacy and ethical issues are foremost on our minds. We are dealing with students who are underage and don’t understand the implications and ramifications of breach of privacy. However the value of correct analytics could revolutionize their education for the better, so a balance must be achieved. There will need to be a strict code of ethics governing data collection and how it is used. It is easy to monitor an individual, like a teacher, but how do we monitor large corporations who will be collecting and using data. This is a very large issue and one that needs to be addressed at a much higher level than the schools. I think we are going to be seeing alot of changes in the laws and regulations that govern this area of technology in the near future.
I have to say I’ve really enjoyed everyones comments and discussion on these challenges. While we were prepping our OER I originally didn’t see how closely related these topics were. However, through our discussions I’ve realized how the impact of one challenge can lead to another. However I think this is beneficial that we realize all these implications in order to make us more cognitive when selecting specific venture programs.
I had a difficult time choosing one particular challenge. The one that concerns me the most are two that are intertwined, privacy and ethical concerns in relation to transparency and accountability. In general, this is often overlooked for many programs or giving out personal data since it is so common to share information online and in collaborative spaces. In education, data will help with assessments and improving student outcomes but so much information on student specifics is a concern. This is especially true from the standpoint of students with learning challenges as for many of them, this information is of a medical nature. How much of it will be used to help with student achievement and to what degree or level is student information shared? The intermixing of transparency is a concern as well because parents or other stakeholders would have/may want access to this private information as well. Who will always have access to it, be able to share it and send it in terms of the analytics use. I also agree with all of the other comments above that not all analytics are equal so determining how effective they are for different things must be considered. It would not be beneficial if analytics were misinterpreted in some way i.e. EQAO scores and an entire board followed a set outcome only to find out it was misrepresented or interpreted incorrectly.
Hi Leah,
You’ve highlighted just how valuable it is to consider all of the challenges listed above. I believe we’re all apprehensive of all the challenges, but depending on each situation, environment or learning level. We might place value on one challenge more than the other. I believe it is important though that we heavily research what that challenge is before placing less importance on the other ones.
Hi Courtney,
Yes I agree but privacy and ethical concerns were at the forefront of my concerns especially with such sensitive data that can be collected, especially that of students. It is the one that I used to vote upon. Thank you for your response!
Cheers
Angela V – sorry put this in the wrong spot initially 🙂
I chose funding as my primary concern, but I think that it is closely related to the allocation of resources. Funding in education has always been an issue where practitioners and funders seem to differ in their opinion. I think that analytics could actually have a positive effect on funding, however, the risk is that misrepresentation of data, lack of time to make decisions or other hiccups in the data analysis could be harmful for funding decisions. I don’t want to give the impression that analytics should not be used to make funding decision, just that the risk is very high!
Absolutely. So many of these challenges are closely related to one another. Misrepresentation of data could lead to an improper allocation of funds and resources. I think you raise a good point about the danger of relying solely on data. Learning analytics can provide valuable data to make informed decisions perhaps there should be multiple factors that contribute to these decisions.
I chose misrepresentation of analytics for several reasons. All programs are not created equally and the data obtained from one program may not be comparable to data obtained from a similar program. Data analytics can be a useful tool for supporting students and educators but should not be the sole source of information. There are many variables that need to be taken into consideration and not all programs will account for these.
Great point. Learning analytics can help support to make an informed decision, but we need to be careful that its not the only factor. Your point about all programs not being created equally can also highlight that data can be read differently by different people. One individual could interpret data differently from another person.
I think misrepresentation is very important (I don’t know for sure if it is the MOST important) because wrong data is worse than no data at all. Misrepresenting data can lead to a variety of issues and this is also tied to ethical issues I think. Misrepresenting data can be used by organizations to look better, make others look worse, or to make more money and with analytics, it is literally all about the data.
Thank you for raising a concern over ethic issues. Misrepresenting data – or a manipulation of data easily raises some concerns of ethics.
Improving Student Outcome is the main purpose of the analytics and this is why it is the most important.
What are the sort of problems or challenges that you see arising from focusing on just this aspect? What happens if student outcome is not improved?
I chose misrepresentation of analytics (which it might be good to add misuse of analytic data to the description). There’s such a big gap for unanticipated user behavior that can have a large influence on what one thinks they are recording, and what is actually happening and being measured. For example, analytics that measure length of page visit or times visited to measure how long a student spends on a particular page or resource, can be deceiving. I’ve run into the problem once where an instructors’ data panel had shown minimal time spent on a resource, when in fact students had been copy/pasting resources or printing them to an external PDF (which takes only a few seconds) so they could then close out the LMS environment and read the assigned materials in environments of their own choosing (import PDF onto iPad for annotation, or annotation and highlighting in Adobe reader etc.). Many custom measures don’t and can’t take into account certain variables where procedures for use differ between user groups.
I agree. In any environment there are always going to be internal and external factors that threaten the reliability and validity of the information obtained. Whoever is the decision maker in regards to this information needs to be aware of these factors in order to make a proper informed decision. It might be useful in analytics to back up quantitative data with qualitative to have a balance of information present.
I chose performance based funding as I think the potential for creating have and have not schools is possible. That said perhaps the opposite is true, if we can better identify schools where learners are struggling then resources can be directed in the most efficient way.
It’s difficult to tell how someone would allocate resources in this regard. If you realized that a school was performing better and achieving higher assessments results as a result of a set of class iPads – would it be better to reward that school with another set to further improve their learning? Or would it be better to send the class set to an under achieving school in hopes that the iPads have the same result?
I agree this is definitely a challenge/danger of learning analytics and of those who are in charge of using this data to make decisions.