The four readings presented for this week’s activity were rich in information and quite different than much of the educational literature I have encountered in the MET program (I could say that about most readings in this course). Learner documentation is a point of contention among my own faculty, as the depth of needed detail and perspectives on its use vary greatly. Policy guides our hand for some data: we must retain major evaluative components for up to a year. Others feel that documentation justifying every graded objective should be stored in perpetuity.
The article by Dr. Carol Wien (2013) was a step out of my comfort zone. Aside from being a parent of a 1-year-old and a 3-year-old, I have never taught children. A standout message is Dr Wien’s commentary that documentation offers insight into children’s thinking, worldview, and feelings (Wien, 2013), and that, when we transcribe their experiences for others to see, it can broaden our perspectives and responsiveness (Wien, 2013). I found this article taking a very, what I might call candidly, “Piaget on steroids” approach to empowering children. Documenting children’s learning experience supports educator inquiry, can promote collaborative reflections, and treating children as researchers is profoundly learner-centric (Wien, 2012).
The de Castell et al. article was validating, and a fun foray into evaluating effective gamification always makes for an interesting read. My ultimate takeaway is something I feel all teachers wrestle with: standardized tests underestimate learning and knowledge, and (new thought) that documentation can reveal understanding not measurable with traditional methodologies like multiple-choice (de Castell et al. 2016). The multimodal expression and documentation revealed deep learning and enabled assessors to evaluate the quality of thinking, an extremely difficult aspect to assess with true-or-false quiz questions (de Castell et al., 2016).
Romero, Ventura, and Garcia (2008) brought me straight back to Leah Macfadyen’s course on learning analytics and her substantially cited articles on the subject. Learning analytics on LMSs can be controversial, as while information such as log data, clicks, and dwell time can hint at a student’s engagement, you have no way of knowing what they are doing at the time. This documentation is helpful from a metaknowledge perspective, as educators can compile patterns of behaviour and examine trends in student success using large datasets (Romero et al., 2008). It can also give the educator a chance to see how students are using the course, how they navigate it, and how they typically move through the content (Romero et al., 2008).
In the Bjiarveit et al (2019) article, I was pleasantly surprised at how wholesome and holistic the approach to learner documentation was. The process of integrating children’s voices and making their interests, strengths, and stories visible through methods such as artwork, photographs, written observations, and transcriptions of discussions frames it as a powerful tool (Bjiarveit et al., 2019). In the context of students who may not have a voice, it provides one through other multimodal means, such as symbolism and visuals. Like the article by Dr. Carol Wein (2013), it emphasized the role documentation can play in facilitating holistic learning and in enabling children’s potential when individualized education is encouraged through such methods (Bjiarveit et al., 2019).
My choice of Poison
I find myself particularly drawn to performative assessments in my teaching context. Educating emergency room nurses involves preparing learners with the foundational knowledge to make quick, safe, proactive, and reactive decisions under stress, and assessing whether they can do so. We often find that students who perform well on these assessments demonstrate excellent recall and problem-solving in the other assessment areas, except for the essays. As my faculty often says to each other while grading papers, “Writing a good paper does not make you a great ER nurse.” I particularly enjoy performative assessments because they provide a sense of cathartic relief. I can see the student using all areas of Bloom’s taxonomy in real time and in fluid circumstances. As an educator, it gives me a moment to lean back and go “they have got it!” with sometimes extreme satisfaction.
We use a clinical evaluation tool (CET) to grade evaluative assessment moments within the clinical setting. Noteworthy learning moments are usually discussed between the educator and the learner shortly after through a private, informal debrief. These debriefs allow the learner to reflect and share about their experience; they are encouraged to share their feelings and takeaways, and the instructor can use them as a launching pad to grow their knowledge through appropriate scaffolding. These experiences and shared debriefs are what “fill” the CET tool. In a fluid, well-established learner/instructor relationship, the tool holds no surprises, and what each places in the boxes I usually find identical. Discord usually arises from miscommunication or a lack of insight on the part of the learner. See the images below for the “status” checklist and freehand example area, where learners rationalize their choices with reflective examples. The instructor and learner go over the tool throughout the clinical placement, with formal reviews happening at a midterm point and a final point.
The domains are extensive. Through many revisions and reviews by dozens of faculty and instructors actively using it, the consensus is that it generally encompasses the learning outcomes we aim to assess. The document is copied, students receive a signed version for their own records at the end of the clinical experience, and a copy is made and stored at the institution for 10 years. It is exceedingly rare for a copy to be pulled up; the most common reason is when a faculty member uses it to provide specific examples for reference letters.
Documenting and assessing in this manner certainly changes the process learners go through when gaining knowledge and pursuing understanding in this context. The documentation positions knowledge as contextual, demonstrated through clinical competence, and multifaceted (thinking of the domains). The assessment drives a pedagogy of continuous, criterion-based evaluation, built on reflection by both the learner and the instructor. Knowledge in this manner is something to be demonstrated and performed.
Ethics
I view this tool as having two primary ethical concerns: the first is privacy. The potential intimate details of learners’ reflections in an institutional database raise questions about data security and recourse in the event of a breach. Faculty outside those immediately involved with the student could quite easily access the records, and this is certainly unknown to the students. The second is the abuse potential: if there is a heavy bias toward either the learner or the instructor, and the learner feels they have little recourse to disagree with the instructor’s assessment. My faculty always ensures that more than one instructor is involved with struggling students, to allow for a more open dialogue. This is certainly far from foolproof, as while it can often provide a second check and perspective, if there is a positive pre-existing relationship between the instructor and the support personnel, it is easy to see how the bias may just be copied.
In the broader context of social justice, the CET encompasses domains related to equity-oriented care. However, the tool itself presents ethical challenges. The tool is a performative checklist observing demonstrated knowledge in action in a technical environment. There is little to no room for acknowledging differing ways of knowing, culture-based semiotic differences, and disability.
References
de Castell, S., Jenson, J. Thumlert, K., & Muehrer, R. (2016). Assessing assessment: An exploratory study of game-based, multimodal learning in Epidemic. Digital Culture & Education (8)
Romero, C; Ventura, S. & Garcia, C. (2008) Data mining in course management systems: Moodle case study and tutorial. Computers & Education, 51, Issue 1, pp.368-384 https://doi.org/10.1016/j.compedu.2007.05.016
Carol Ann Wien (2013) Making Learning Visible Through Pedagogical Documentation http://conference-handouts.s3.amazonaws.com/2019-nctm-san-diego/pdfs/378-
Bjiarveit et al (2019). The Living Wall: Implementing and Interpreting Pedagogical Documentation in Specialized Settings https://journals.uvic.ca/index.php/jcs/article/view/19058