Tag Archives: authentic assessments

Non-disposable assignments in Intro to Philosophy

Remixed from two CC0 images on Pixabay: trash can and No symbol

Disposable assignments

In the past couple of years I’ve really been grabbed by the issue of “disposable assignments,” as discussed by David Wiley here:

These are assignments that students complain about doing and faculty complain about grading. They’re assignments that add no value to the world – after a student spends three hours creating it, a teacher spends 30 minutes grading it, and then the student throws it away. Not only do these assignments add no value to the world, they actually suck value out of the world.

A non-disposable assignment, then, is one that adds value to the world beyond simply being something the students have to do to get a grade. A similar idea is expressed by Derek Bruff in a post on the idea of “students as producers”–treating students as producers of knowledge, rather than only as consumers: Bruff talks about students creating work for “authentic audiences,” beyond just the teacher.

Wiley gives an example of a non-disposable assignment: students taking instructional materials in the course (which are openly licensed) and revising/remixing them to create tutorials for future students in the course. Other examples can be found in this growing list of examples of open pedagogy. One that I often hear about is asking students to edit or create Wikipedia articles. Or students could post their work more locally, but still have it be publicly visible, such as what Judy Chan does with her students’ research projects at UBC (click on “team projects” in the various years). Simon Bates has his physics students create learning objects to help their peers (see this story for more).

Students as producers in philosophy courses

I have already started to ask students to do some activities that could add value to the world, whether to their fellow students and/or beyond.

In a second-year moral theory course I asked them to sign up for 1-2 days on which to do “reading notes” on the class wiki page: they had to outline one of the main arguments in the text assigned for that day and write down questions for their small group to discuss. You can see those here (organized by group number).
In a first-year, introduction to philosophy course I have asked students to:
- blog about what they think philosophy is, both at the beginning and end of the course–this, I thought, could provide some interesting information to others about what our students think “philosophy” is. I don’t have those blog posts visible anymore because I didn’t ask students if I could keep them posted after the course was finished (d’oh!!).
- write a blog post describing how they see philosophical activity going on in the world around them, beyond the class–I thought this could be useful to show the range of what can count as philosophical activity. I do still have those posts up (but not for long, because again I forgot to ask for permission to keep the posts up after the course is finished…I will do that this term!): https://blogs.ubc.ca/phil102 (click on “philosophy in the world”)

But now that I’m working on my Intro to Philosophy course for Fall 2015 (see planning doc here), I’m trying to think through some other options for assignments with authentic audiences and that add value to the world. Here are some ideas (not that I’m going to implement all of these; I’m just brainstorming).

Editing Wikipedia articles on philosophy
- This is a big task; it requires that students learn how to do so (not just technologically, but in terms of the rules and practices of Wikipedia), plus determining which articles need editing, etc.
- I would prefer to start with students creating Wikipedia-style articles on philosophers or texts on the UBC Wiki first. Then other students (in future classes) could edit those, and then maybe eventually we could move to doing something on Wikipedia itself (the content would be good, and maybe students would be motivated to move some of it over to Wikipedia at that point).

Creating tutorials or other “learning objects” for their fellow students and for the public
- As noted above, Simon Bates does this in his Physics 101 course, and I can pretty easily see how one might ask students to do so for basic physics concepts. But why not do so for some basic philosophy concepts too?
  - e.g., find something you find difficult in the course, and once you feel you have a handle on it, create something to help other students
- could be done in groups (probably best, with a large class like Intro to Phil (150 students))
- could be text based, but better if also incorporates some other kinds of visual or auditory elements (e.g., a video, or incorporating images, or slides or something)

Creating study questions or suggestions of “what to focus on” for the readings
- students often get lost in reading primary philosophical texts, and I haven’t yet managed to write up study questions or suggestions for what to focus on for each reading. This would definitely be useful to other students.
- But wouldn’t it be cruel to ask students to do this for later students when I haven’t done it for them myself? and do I have time to do this before the Fall term this year? Unfortunately not.

Creating lists of “common problems” or advice for writing, after doing peer review of each others’ work and self-reflecting on their own
- I do provide quite a lot of writing advice to students, but I wonder if advice coming from students’ direct experience in my courses might be helpful to later students?

Creating possible exam questions
- I ask students to do this informally, in groups, as part of the review for the final exam. But why not formalize this somehow so their suggestions are posted publicly? The course page on the UBC Wiki seems like a good place, at least to start. Then students could see them from year to year.
- A number of instructors at UBC use PeerWise as a tool for students to ask and answer questions. It seems like an interesting thing, but:
  - It’s not public; but it could be used to generate questions and then the best ones could be made public somewhere
  - It’s limited to multiple choice questions, which I hardly ever use (and never on exams)

Those are my ideas for now. Have any others? Or comments on any of this? Please comment, below!

Authentic assessment and philosophy

In order to prepare for a meeting of the Scholarship of Teaching and Learning Community of Practice, I recently started reading a few articles on “authentic assessment.” I have considered this idea before (see short blog post here), but I thought I’d write a bit more about just what authentic assessment is and how it might be implemented in philosophy.

Authentic assessment–what

A brief overview of authentic assessment can be found in Svinicki (2004). According to Svinicki, authentic assessment “is based on student activities that replicate real world performances as closely as possible” (23). She also lists several criteria for assessments to be authentic, from Wiggins (1998):

1. The assessment is realistic; it reflects the way the information or skills would be used in the “real world.”

2. The assessment requires judgment and innovation; it is based on solving unstructured problems that could easily have more than one right answer and, as such, requires the learner to make informed choices.

3. The assessment asks the student to “do” the subject, that is, to go through the procedures that are typical to the discipline under study.

4. The assessment is done in situations as similar to the contexts in which the related skills are performed as possible.

5. The assessment requires the student to demonstrate a wide range of skills that are related to the complex problem, including some that involve judgment.

6. The assessment allows for feedback, practice, and second chances to solve the problem being addressed. (23-24)

She points to an example of how one might assign a paper as an authentic assessment. Rather than just writing an essay about law generally (perhaps legal theory?), one might ask students to write an essay arguing for why a particular law should be changed. Or even better, write a letter to legislators with that argument (25).

Turns out there are numerous lists of what criteria should be used for authentic assessment, though (not surprising?). I have only looked at a few articles, and only those that are available for easy reading online (i.e., not books, or articles in books, or articles in journals to which our library does not have a digital subscription–I know this is lazy, but I’m not doing a major lit review here!). Here’s what I’ve found.

In Ashford-Rowe et al. (2014), eight questions are given that are said to get to the essential aspects of authentic assessment. These were first developed from a literature review on authentic assessment, then subjected to evaluation and discussion by several experts in educational design and assessment, and then used to redesign a module for a course upon which they gathered student and instructor feedback to determine whether the redesign solved some of the problems faced in the earlier design.

(1) To what extent does the assessment activity challenge the student?

(2) Is a performance, or product, required as a final assessment outcome?

(3) Does the assessment activity require that transfer of learning has occurred, by means of demonstration of skill?

(4) Does the assessment activity require that metacognition is demonstrated?

(5) Does the assessment require a product or performance that could be recognised as authentic by a client or stakeholder? (accuracy)

(6) Is fidelity required in the assessment environment? And the assessment tools (actual or simulated)?

(7) Does the assessment activity require discussion and feedback?

(8) Does the assessment activity require that students collaborate? (219-220)

Regarding number 3, transfer of learning, the authors state: “The authentic assessment activity should support the notion that knowledge and skills learnt in one area can be applied within other, often unrelated, areas” (208). I think the idea here is that the knowledge and skills being assessed should be ones that can transfer to environments beyond the academic setting, which is the whole idea with authentic assessment I think.

Number 4, metacognition, has to do with self-assessment, monitoring one’s own progress, the quality of one’s work, reflecting on the what one is doing and how it is useful beyond the classroom, etc.

Number 6, regarding fidelity, has to do with the degree to which the environment in which the assessment takes place, and the tools used, are similar to what will be used and how, outside of the academic setting.

The point of number 8, collaboration, is that, as the authors state, “The ability to collaborate is indispensable in most work environments” (210). So having assessments that involve collaboration would be important to their authenticity for many work environments. [Though not all, perhaps. And not all authentic assessment needs to be tied to the workplace, right? Couldn’t it be that students are developing skills and attitudes that they can use in other aspects of their lives outside of an educational context?]

Gulikers et al. (2004) define authentic assessment as “an assessment requiring students to use the same competencies, or combinations of knowledge, skills, and attitudes, that they need to apply in the criterion situation in professional life” (69). They took a somewhat different approach to determining the nature of authentic assessments than that reflected in the two lists above. They, too, started with a literature review, but from that focused on five dimensions of authentic assessments, each of which can vary in their authenticity:

(a) the assessment task

(b) the physical context

(c) the social context

(d) the assessment result or form

(e) the assessment criteria (70)

Whereas the above two lists look at the kinds of qualities an assessment should have to count as “authentic,” this list looks at several dimensions of assessments and then considers what sorts of qualities in each dimension would make an assessment more or less authentic.

So, for example, an authentic task would be, given their definition of authentic assessment as connected to professional practice, one that students would face in their professional lives. Specifically, they define an authentic task as one that “resembles the criterion task with respect to the integration of knowledge, skills, and attitudes, its complexity, and its ownership” (71), where ownership has to do with who develops the problem and solution, the employee or the employer (I think that’s their point).

The physical context has to do with what sorts of physical objects people will be working on, and also the tools they will generally be using. It makes assessments less authentic if we deprive students of tools in academic settings that they will be allowed to use in professional settings, or give them tools in academic settings that they generally won’t have access to in professional settings. Time constraints for completing the task are also relevant here, for if professionals have days to complete a task, asking students to do it in hours is less authentic.

The social context has to do with how one would be working with others (or not) in the professional setting. Specifically, they specify that if the task in the professional setting would involve collaboration, then the assessment should do so, but not otherwise.

The assessment result or form has to do with the product created through the task. It should be something that students could be asked to do in their professional lives, something that “permits making valid inferences about the underlying competencies,” which may require more than one task, with a variety of “indicators of learning” (75).

Finally, the criteria for the assessment should be similar to those used in a professional setting and connected to professional competencies.

Authentic assessment and philosophy

Though Gulikers et al. (2004) tie authentic assessment pretty closely to professional life, and thus what they say might seem to be most relevant to disciplines where professional practice is directly part of courses (such as medicine, business, architecture, clinical psychology, and more), the overview in Svinicki (2004) suggests that authentic assessments could take place in a wide variety of disciplines. What could it look like in philosophy?

I think this is a somewhat tricky question, because unlike some other fields, where what one studies is quite directly related to a particular kind of activity one might engage in after receiving a degree, philosophy is a field in which we practice skills and develop attitudes that can be used in a wide variety of activities, both within and beyond one’s professional life. What are those skills and attitudes? Well, that’s a whole different issue that could take months to determine (and we’re working on some of that by developing program outcomes for our major in philosophy here at UBC), but for now let’s just stick with the easy, but overly vague answers like: the ability to reason clearly; to analyze problems into their component parts and see interrelationships between these; to consider implications of particular beliefs or actions; to make a strong case for one approach to a problem over another; to identify assumptions lying behind various beliefs, approaches, practices; to locate the fundamental disagreements between two or more “sides” to a debate and thereby possibly find a way forward; to communicate clearly, orally and in writing; to take a charitable attitude towards opponents and focus on their arguments rather than the persons involved; and more.

So what could it mean to do a task in philosophy in a similar way, with similar tools, for example, as what one might encounter in a work environment? Because the skills and attitudes developed in philosophy might be used in many different work environments, which one do we pick? Or, even more broadly, since many of these skills and attitudes can be practiced in everyday life, why restrict ourselves to what one might do in a work environment?

Perhaps, though, this means we have a lot more leeway, which could be a good thing. Maybe authentic assessments in philosophy could be anything that connects to what one might do with philosophical thinking, speaking and writing skills outside of the educational setting. And if several courses included them during a students’ educational career, they could perhaps see how philosophy can be valuable in many aspects of their lives, having done different sorts of authentic assessments applying those skills to different kinds of activities.

When I came up with a couple of possible authentic assessments in philosophy courses last summer, I believe I was thinking along these lines–something that the students would do that would mirror an activity they might engage in outside of class. One, which I implemented this year in my moral theory course, asked students to apply the moral theories we’re studying to a moral dilemma or issue of some kind. This isn’t exactly like an authentic assessment, though, because I’m not sure that I would expect anyone in their everyday lives to read Kant and Mill and then try to apply them to moral dilemmas they face. Maybe some people do, but I’m not really sure that’s the main value of normative moral theories (I’m still working on what I think that value is, exactly).

Another one of the suggested assignments from that earlier blog post was that students would reflect on how they use philosophical thinking or speaking or writing in their lives outside of the course. That one isn’t asking them to do so, though, so it’s not like mirroring a task they might use outside the class; it’s just asking them to reflect on how they already do so.

So I think I need to consider further just what an authentic assessment in philosophy might look like (the one from Svinicki (2004), above, about writing a letter to legislators to change a law is a good candidate), and how I might include one in a course I teach in the future. Possible ideas off the top of my head:

Take a discussion of a moral issue (for example) in the media and clearly lay out the positions on the various “sides” and what arguments underlie those. Evaluate those arguments. (We do this sort of thing all the time in philosophy, but not always by starting with media reports, which would be the sort of thing one might do in one’s everyday life.) Or, identify assumptions in those positions.
Write a letter to the editor or an op-ed piece about some particular moral or other issue, laying out clear arguments for your case.
Participate in or even facilitate a meeting of a Socrates Cafe, a philosophical discussion held in a public place for anyone who is interested to join.
Make a case to the university, or your employer, or someone else for something that you’d like to see changed. Give a clear, logical argument for why it should be changed, and how. Can collaborate with others on this project.

Okay, this is hard.

And it occurs to me that some of what we already do might be like an authentic activity, even if not an authentic assessment. For example, when we ask students to engage in philosophical discussion in small groups during class, this is the sort of thing they might also do in their lives outside of class (don’t know how many do, but we are giving them practice for improving such activities in the future).

Hmmm…gotta think more on this…

Any ideas are welcome, in the comments below!

Works Cited

Ashford-Rowe, K., Herrington, J. & Brown, C. (2014). Establishing the critical elements that determine authentic assessment. Assessment & Evaluation in Higher Education, 39(2), 205-222. DOI: 10.1080/02602938.2013.819566

Gulikers, J.T.M., Bastiaens, T.J., Kirschner, P.A. (2004). A five-dimensional framework for authentic assessment. Educational Technology Research and Development, 52(3), 67-86. Available on JSTOR, here: http://www.jstor.org/stable/30220391?

Svinicki, M. D. (2004). Authentic assessment: Testing in reality. New Directions in Teaching and Learning, 100, 23-29. Available behind a paywall, here: http://onlinelibrary.wiley.com/doi/10.1002/tl.167/abstract

Wiggins, G. (1998). Educative Assessment: Designing Assessments to Inform and Improve Student Performance. San Francisco: Jossey-Bass.

Authentic assessments in two PHIL classes

For the blended learning course I’m taking on teaching a blended learning course, we were asked to design an “authentic assessment” for one of our courses. An authentic assessment, from what I understand, is one in which students are either simulating or doing the very sorts of activities you hope they will be able to do outside of class, after they take the course. In addition, according to some of the text of the course I’m taking,

According to Eisner (1993), authentic assessment projects should reveal how students go about solving the problems (process) and should have more than one correct solution. They should:

Promote ‘how’ knowledge as opposed to the ‘what’ knowledge measured in ‘traditional’ assignments;

Provide a way for students to develop an understanding of complex course material that will serve them beyond the classroom;

Encourage higher-order cognitive skills;

Involve students more extensively in the development of the assessment and the grading criteria.

PHIL 102: Introduction to Philosophy

Here is an idea for an authentic assessment activity for my Introduction to Philosophy course.

Rationale

In PHIL 102, Introduction to Philosophy, the main theme of the course is investigating what philosophy is, what philosophers do, and the value of these things, both by reading about what philosophers themselves have said about these questions, and by considering what the philosophers whose texts we are reading are doing with their lives and their writing.

One of the things I’d like students to be able to do by the end of the course is to recognize ways in which they themselves engage in philosophical activity, in their everyday lives.

Activity

Students will write a reflective blog post towards the end of the term in which they discuss two things they do in their lives that could show philosophical thinking or addressing of philosophical questions. They will also add a short summary of their post for a class wiki page on this question.

Learning objective addressed: “Explain at least two ways in which you yourself use philosophical thinking or address philosophical questions in your everyday life.”

Instructions

Now that the course is nearly over, you should have a pretty good idea of what philosophy is and what philosophers do. It’s time to consider the ways in which you yourself engage in philosophy. This assignment consists of two parts:

1. Write a blog post on the class blog in which you do the following:

Discuss at least two ways in which you yourself use philosophical thinking or consider philosophical questions in your own life, your own day-to-day activities, your major life decisions, etc.
Explain why these could be considered “philosophical,” referring to at least one of the philosophers or texts or ideas we’ve discussed in class.
This blog post should be at least 300 words long, but no longer than 800 words

2. After you’ve completed your blog post, contribute your two ways to the class wiki page for this assignment [give URL for this here].

Write a one or two-sentence summary of each of the ways you engage in philosophical thinking or activity and put them as bullet points on the wiki page.
Christina will then organize these under general categories after they are posted, to make them easier to read through, and we’ll discuss the results in class

Marking criteria

This assignment will be marked using a three-level system:

1. Plus:

Your blog post discusses at least two ways in which you engage in philosophical thinking or address philosophical questions in your life
Your blog post adequately explains how these things are philosophical, referring to at least one of the philosophers/texts/ideas we’ve discussed in class.
Your blog post is between 300 and 800 words long.
You wrote a one- or two-sentence summary of each of the two things you discussed in your post, on the class wiki page.
Both the post and the wiki entry were completed by the due date and time.

2. Minus:

Your blog post discusses only one way in which you engage in philosophical thinking or address philosophical questions in your life, or
Your blog post does not adequately explain how this/these activities are philosophical, and/or doesn’t refer to at least one of the philosophers/texts/ideas we’ve discussed in class, or
Your blog post is less than 300 words or more than 800 words, or
Your blog post was fine, but you didn’t submit your one- or two-sentence summary of each point discussed in the post on the wiki page, or
Your blog post and/or wiki entry were submitted after the due date and time, but no later than six days afterwards.

3. Zero:

Your post and/or wiki page entry was not completed, or
Your blog post and/or wiki entry were completed seven or more days after the due date.

Thoughts/questions

I wanted this assignment to not only be useful for the students writing the posts themselves, to get them to think about how philosophy plays a role in their own lives, but also to others. That’s why I thought of having them post to a wiki page–there are often over 100 students in this course, and reading that many different blog posts will be too much for anyone else visiting the course (my courses are on open sites, on UBC Blogs, so anyone can visit them; students always have the option of posting under a pseudonym, or with a password so only the rest of the class can read, or private to me if they choose).

But just having a list of one- or two-sentence summaries on a wiki page is too messy too. So I thought I’d try to categorize them myself after they’re posted, and say something like: 15 people said x, 8 people said y, etc.

Of course, this is more work for me. Any ideas on how to make it so that we have a kind of summary document that might be useful for students in the class as well as others, without me having to go through and categorize all the entries? It’s okay if I have to do so (it’s just busy work, and easy), but if there are other ways I’d love to hear them!

PHIL 230: INTRODUCTION TO MORAL THEORY

Here is an idea for an authentic assessment for this course. Students will be writing in a “moral issue” journal throughout the course, starting with what they think about a particular moral issue, then comparing this with what they think each of the philosophers we study would say about it, and then concluding with their thoughts on the value of trying to come up with moral theories such as the ones we’ve studied. For this assignment, I’d like students to be able to take what they’re reflecting on in their moral issue journals and refine part of it into a formal essay.

This way, they’ll be using what they have learned in the course in thinking about moral issues they may face around them in their everyday lives.

Moral issue paper

For this paper, you’ll be using what you’ve reflected on in your moral issue journal and writing a formal paper. The idea here, as with the moral issue journal, is to apply the moral theories we’ve been studying to a moral issue that you might face in your life, or one that involves a larger group of people such as a community or nation. In this way, you’ll be making connections between what we’re studying in class and your life beyond.

Instructions

Using the moral issue you’ve been focusing on in your moral issue journal, write an argumentative paper that argues for how a consequentialist and a Kantian would approach the issue. Include also your own view on whether one approach is better than the other for this particular issue, and why (or why not; it may just be that they are very different and there’s no clear reason to choose one over the other).

Parts of the essay

Note from the Guidelines for essays handout that your essay should have an introduction with your thesis statement, a conclusion that wraps up the essay in some way, and body paragraphs that provide adequate arguments for the conclusion.

Your thesis should include (note that a thesis can be more than one sentence):

A summary statement of what a consequentialist and a Kantian would say about the issue
A summary of your view on whether or not one approach is better

Be sure to explain the moral issue you’re addressing, early on in the essay.

Length

The essay should be between 5 and 8 pages, typed, double-spaced, with margins between 0.75 and 1 inches, and font size between 11 and 12 points. [Or 2000-3000 words?]

Quotes, paraphrases, and citing sources

Quotes vs paraphrases: It’s usually best to have a mixture of both. You should use quotes where it’s important to give the author’s exact words, where the words themselves help you to make a point. This is often the case when a passage can be interpreted in more than one way, and you want to justify your interpretation with the words of the author. You can also use quotes where you need an extended passage to make your point (be sure to indent quotes over 4 lines long, 5 spaces on the left).

Citing sources in the paragraphs: Whether you give a quote or paraphrase a specific point from the text, you should give a page number or section/paragraph number to show where the information can be found in the text. You choose your favourite citation style, or you can just give the author’s last name plus the page or section number, in parentheses: (Kant 55). (This is the MLA style.) If you are citing more than one text by an author, give a shortened version of the title of the text in the parentheses as well: (Kant, Religion 99).

Citing sources at the end of the essay: Be sure to give a works cited page that includes all the texts you cited in parentheses in the essay. Again, you can use any citation style you wish, but be sure to include all the information that that citation style requires. For example, you can see how to create a Works Cited list in MLA style here [give URL].

Avoid plagiarism: It is the policy of the Instructor to prosecute plagiarism to the fullest extent allowed by UBC. Any use of another’s words, including just a sentence or part of a sentence, without citation, constitutes plagiarism. Use of another’s ideas without citation does as well. To avoid plagiarism, always give a citation whenever you have taken ideas or direct words from another source. Please see this page on the course website for information on how to avoid plagiarism, especially when you’re paraphrasing ideas or quoting from another source—quite a lot of plagiarism is not on purpose, just because students don’t understand the rules! https://blogs.ubc.ca/phil102/resources/

Depth of explanation and narrowness vs. breadth and superficiality: It’s usually best to focus your paper on a small number of claims and argue for them in some depth rather than trying to range widely over a very large number of claims that you then only have space to justify very quickly. Pick the strongest points for each, consequentialism and Kantianism, and focus on those.

Audience you should write for: Write this essay as if you were writing for someone who is in the class, has not read the texts, and has not attended the class meetings (say, a friend or family member). Explain your view, and the arguments of the philosophers you discuss, in as much depth as would be needed to make them clear to such an audience.

Marking: See the marking rubric posted here on the course website [give URL].

Late penalty: 5 points off per weekday late, unless otherwise agreed to by the Instructor (may require documentation). I do not generally give extensions due to students’ workloads, only for things that are unexpected and unavoidable such as medical issues; so plan ahead if you have multiple assignments due around the time that this essay is due!