In order to prepare for a meeting of the Scholarship of Teaching and Learning Community of Practice, I recently started reading a few articles on “authentic assessment.” I have considered this idea before (see short blog post here), but I thought I’d write a bit more about just what authentic assessment is and how it might be implemented in philosophy.
Authentic assessment–what
A brief overview of authentic assessment can be found in Svinicki (2004). According to Svinicki, authentic assessment “is based on student activities that replicate real world performances as closely as possible” (23). She also lists several criteria for assessments to be authentic, from Wiggins (1998):
1. The assessment is realistic; it reflects the way the information or skills would be used in the “real world.”
2. The assessment requires judgment and innovation; it is based on solving unstructured problems that could easily have more than one right answer and, as such, requires the learner to make informed choices.
3. The assessment asks the student to “do” the subject, that is, to go through the procedures that are typical to the discipline under study.
4. The assessment is done in situations as similar to the contexts in which the related skills are performed as possible.
5. The assessment requires the student to demonstrate a wide range of skills that are related to the complex problem, including some that involve judgment.
6. The assessment allows for feedback, practice, and second chances to solve the problem being addressed. (23-24)
She points to an example of how one might assign a paper as an authentic assessment. Rather than just writing an essay about law generally (perhaps legal theory?), one might ask students to write an essay arguing for why a particular law should be changed. Or even better, write a letter to legislators with that argument (25).
Turns out there are numerous lists of what criteria should be used for authentic assessment, though (not surprising?). I have only looked at a few articles, and only those that are available for easy reading online (i.e., not books, or articles in books, or articles in journals to which our library does not have a digital subscription–I know this is lazy, but I’m not doing a major lit review here!). Here’s what I’ve found.
In Ashford-Rowe et al. (2014), eight questions are given that are said to get to the essential aspects of authentic assessment. These were first developed from a literature review on authentic assessment, then subjected to evaluation and discussion by several experts in educational design and assessment, and then used to redesign a module for a course upon which they gathered student and instructor feedback to determine whether the redesign solved some of the problems faced in the earlier design.
(1) To what extent does the assessment activity challenge the student?
(2) Is a performance, or product, required as a final assessment outcome?
(3) Does the assessment activity require that transfer of learning has occurred, by means of demonstration of skill?
(4) Does the assessment activity require that metacognition is demonstrated?
(5) Does the assessment require a product or performance that could be recognised as authentic by a client or stakeholder? (accuracy)
(6) Is fidelity required in the assessment environment? And the assessment tools (actual or simulated)?
(7) Does the assessment activity require discussion and feedback?
(8) Does the assessment activity require that students collaborate? (219-220)
Regarding number 3, transfer of learning, the authors state: “The authentic assessment activity should support the notion that knowledge and skills learnt in one area can be applied within other, often unrelated, areas” (208). I think the idea here is that the knowledge and skills being assessed should be ones that can transfer to environments beyond the academic setting, which is the whole idea with authentic assessment I think.
Number 4, metacognition, has to do with self-assessment, monitoring one’s own progress, the quality of one’s work, reflecting on the what one is doing and how it is useful beyond the classroom, etc.
Number 6, regarding fidelity, has to do with the degree to which the environment in which the assessment takes place, and the tools used, are similar to what will be used and how, outside of the academic setting.
The point of number 8, collaboration, is that, as the authors state, “The ability to collaborate is indispensable in most work environments” (210). So having assessments that involve collaboration would be important to their authenticity for many work environments. [Though not all, perhaps. And not all authentic assessment needs to be tied to the workplace, right? Couldn’t it be that students are developing skills and attitudes that they can use in other aspects of their lives outside of an educational context?]
Gulikers et al. (2004) define authentic assessment as “an assessment requiring students to use the same competencies, or combinations of knowledge, skills, and attitudes, that they need to apply in the criterion situation in professional life” (69). They took a somewhat different approach to determining the nature of authentic assessments than that reflected in the two lists above. They, too, started with a literature review, but from that focused on five dimensions of authentic assessments, each of which can vary in their authenticity:
(a) the assessment task
(b) the physical context
(c) the social context
(d) the assessment result or form
(e) the assessment criteria (70)
Whereas the above two lists look at the kinds of qualities an assessment should have to count as “authentic,” this list looks at several dimensions of assessments and then considers what sorts of qualities in each dimension would make an assessment more or less authentic.
So, for example, an authentic task would be, given their definition of authentic assessment as connected to professional practice, one that students would face in their professional lives. Specifically, they define an authentic task as one that “resembles the criterion task with respect to the integration of knowledge, skills, and attitudes, its complexity, and its ownership” (71), where ownership has to do with who develops the problem and solution, the employee or the employer (I think that’s their point).
The physical context has to do with what sorts of physical objects people will be working on, and also the tools they will generally be using. It makes assessments less authentic if we deprive students of tools in academic settings that they will be allowed to use in professional settings, or give them tools in academic settings that they generally won’t have access to in professional settings. Time constraints for completing the task are also relevant here, for if professionals have days to complete a task, asking students to do it in hours is less authentic.
The social context has to do with how one would be working with others (or not) in the professional setting. Specifically, they specify that if the task in the professional setting would involve collaboration, then the assessment should do so, but not otherwise.
The assessment result or form has to do with the product created through the task. It should be something that students could be asked to do in their professional lives, something that “permits making valid inferences about the underlying competencies,” which may require more than one task, with a variety of “indicators of learning” (75).
Finally, the criteria for the assessment should be similar to those used in a professional setting and connected to professional competencies.
Authentic assessment and philosophy
Though Gulikers et al. (2004) tie authentic assessment pretty closely to professional life, and thus what they say might seem to be most relevant to disciplines where professional practice is directly part of courses (such as medicine, business, architecture, clinical psychology, and more), the overview in Svinicki (2004) suggests that authentic assessments could take place in a wide variety of disciplines. What could it look like in philosophy?
I think this is a somewhat tricky question, because unlike some other fields, where what one studies is quite directly related to a particular kind of activity one might engage in after receiving a degree, philosophy is a field in which we practice skills and develop attitudes that can be used in a wide variety of activities, both within and beyond one’s professional life. What are those skills and attitudes? Well, that’s a whole different issue that could take months to determine (and we’re working on some of that by developing program outcomes for our major in philosophy here at UBC), but for now let’s just stick with the easy, but overly vague answers like: the ability to reason clearly; to analyze problems into their component parts and see interrelationships between these; to consider implications of particular beliefs or actions; to make a strong case for one approach to a problem over another; to identify assumptions lying behind various beliefs, approaches, practices; to locate the fundamental disagreements between two or more “sides” to a debate and thereby possibly find a way forward; to communicate clearly, orally and in writing; to take a charitable attitude towards opponents and focus on their arguments rather than the persons involved; and more.
So what could it mean to do a task in philosophy in a similar way, with similar tools, for example, as what one might encounter in a work environment? Because the skills and attitudes developed in philosophy might be used in many different work environments, which one do we pick? Or, even more broadly, since many of these skills and attitudes can be practiced in everyday life, why restrict ourselves to what one might do in a work environment?
Perhaps, though, this means we have a lot more leeway, which could be a good thing. Maybe authentic assessments in philosophy could be anything that connects to what one might do with philosophical thinking, speaking and writing skills outside of the educational setting. And if several courses included them during a students’ educational career, they could perhaps see how philosophy can be valuable in many aspects of their lives, having done different sorts of authentic assessments applying those skills to different kinds of activities.
When I came up with a couple of possible authentic assessments in philosophy courses last summer, I believe I was thinking along these lines–something that the students would do that would mirror an activity they might engage in outside of class. One, which I implemented this year in my moral theory course, asked students to apply the moral theories we’re studying to a moral dilemma or issue of some kind. This isn’t exactly like an authentic assessment, though, because I’m not sure that I would expect anyone in their everyday lives to read Kant and Mill and then try to apply them to moral dilemmas they face. Maybe some people do, but I’m not really sure that’s the main value of normative moral theories (I’m still working on what I think that value is, exactly).
Another one of the suggested assignments from that earlier blog post was that students would reflect on how they use philosophical thinking or speaking or writing in their lives outside of the course. That one isn’t asking them to do so, though, so it’s not like mirroring a task they might use outside the class; it’s just asking them to reflect on how they already do so.
So I think I need to consider further just what an authentic assessment in philosophy might look like (the one from Svinicki (2004), above, about writing a letter to legislators to change a law is a good candidate), and how I might include one in a course I teach in the future. Possible ideas off the top of my head:
- Take a discussion of a moral issue (for example) in the media and clearly lay out the positions on the various “sides” and what arguments underlie those. Evaluate those arguments. (We do this sort of thing all the time in philosophy, but not always by starting with media reports, which would be the sort of thing one might do in one’s everyday life.) Or, identify assumptions in those positions.
- Write a letter to the editor or an op-ed piece about some particular moral or other issue, laying out clear arguments for your case.
- Participate in or even facilitate a meeting of a Socrates Cafe, a philosophical discussion held in a public place for anyone who is interested to join.
- Make a case to the university, or your employer, or someone else for something that you’d like to see changed. Give a clear, logical argument for why it should be changed, and how. Can collaborate with others on this project.
Okay, this is hard.
And it occurs to me that some of what we already do might be like an authentic activity, even if not an authentic assessment. For example, when we ask students to engage in philosophical discussion in small groups during class, this is the sort of thing they might also do in their lives outside of class (don’t know how many do, but we are giving them practice for improving such activities in the future).
Hmmm…gotta think more on this…
Any ideas are welcome, in the comments below!
Works Cited
Ashford-Rowe, K., Herrington, J. & Brown, C. (2014). Establishing the critical elements that determine authentic assessment. Assessment & Evaluation in Higher Education, 39(2), 205-222. DOI: 10.1080/02602938.2013.819566
Gulikers, J.T.M., Bastiaens, T.J., Kirschner, P.A. (2004). A five-dimensional framework for authentic assessment. Educational Technology Research and Development, 52(3), 67-86. Available on JSTOR, here: http://www.jstor.org/stable/30220391?
Svinicki, M. D. (2004). Authentic assessment: Testing in reality. New Directions in Teaching and Learning, 100, 23-29. Available behind a paywall, here: http://onlinelibrary.wiley.com/doi/10.1002/tl.167/abstract
Wiggins, G. (1998). Educative Assessment: Designing Assessments to Inform and Improve Student Performance. San Francisco: Jossey-Bass.
The question of what authentic assessment would look like in philosophy is a very good one. I really liked how you didn’t try to hastily assimilate philosophy with other practical disciplines such as law or business consulting, but instead tried to frame this question in terms of philosophy as an academic discipline.
What philosophy, unlike the other disciplines, can offer, I think, is a theory of how knowledge of one’s situation would change the outcome. Philosophy can offer this, because philosophy is a discipline which tries to suggest and refine theories on what is the case, or of reality.
For example, let’s say the exercise is in sociology. The student is asked to compare two communities. In the first community, the members have individual and collective needs to address, but no knowledge of how the community is governed, nor that of the rationale behind this system. In the second community, the members have needs, and they also share a good theory of how and why their community is governed in a particular way. How would the self-knowledge at the communal level change the overall forms of government in these two communities? I think that this is the type of exercise that philosophy students would benefit from and also have the opportunity to directly apply the ideas which are developed in the literature.
Another example is how self-knowledge changes the quality or character of individual decisions. For example, take two individuals, one who has studies the nature and justification behind the institution of marriage, the other who has never studied such things and simply wants to marry because one marries when one has the chance to do so. How would the marriage relation work out differently in these two cases?
In these types of questions, the problems are very real, yet the approaches one could take are so diverse that I almost feel that it is (perhaps) only a philosophy student who could truly address them. The problems and assignments can be replicated for areas in science, art, business, and so on, and in each case students can grapple with the particulars of each area. Just a thought.
These are excellent points, Kenji…thanks so much! This makes me think that really, in philosophy, the sorts of things we ask students to do when reading about, criticizing, revising theories of reality is itself the sort of thing they might do out there in “real life,” so it’s already authentic. And we could encourage students to see that by reflecting on the sorts of examples you’ve given here, how decisions and actions would be different between people who have thought about the theories and arguments, reasons lying behind one sort of action or policy vs another, and those who haven’t. So we can help students to see that what we’re asking them to do in philosophy courses isn’t something separate from “real life” that we then have to somehow “apply” to real life, but rather it is the exact sort of activity that they can (and should) do in their lives.
This helps me with the discussion of our program learning outcomes for the Philosophy BA here at UBC. We are supposed to come up with specific things outside of the classroom that students can do as a result of having to take philosophy classes, something beyond being able to write a paper on Descartes, which is not something one would do after university unless going on to grad school. But the very thought processes one uses when reading, discussing, writing about Descartes (and others) is precisely what we hope students will do after they graduate. It’s just not necessarily that they’d produce an essay.
Thank you for the thought-provoking and helpful comments!
Even as an outside, the phrase “program leaning outcomes” makes me feel nervous. Are these “outcomes” assessed, perhaps even by a higher bureaucracy? If the philosophy department is losing autonomy, then that is lamentable. In any case, I hope that those who demand such outcomes do not assess them based on merely individual economic success.
Of all the assignments and activities at UBC, writing a paper on Descartes (or Kant) was actually the most valuable for me. It is easy to incorporate fieldwork and the like to philosophy courses, but the real challenge would be to make such activities as interesting as writing an essay on a classical philosopher. I would also add that such activities related to “real life” should come at the senior level courses, since it would be difficult to “apply” philosophical thinking unless one first went through the training process of reading good text, writing good papers, and assessing arguments.
Lastly, let me share one entry written by Smart Bear back in 2010: http://blog.asmartbear.com/authentic-dead.html
And, thanks for keeping the public up to date with the state of philosophy education!