Category Archives: Assignments

Grading rubrics in philosophy

This is a quick post designed to collect links to grading rubrics in philosophy, for the sake of putting them together in one place for graduate student TAs in our department to refer to if they want to see some examples.

Here is a recent version of a grading rubric for essays that I use in my courses, including Introduction to Philosophy and an interdisciplinary course called Arts One. I’m including a PDF version and also an MS Word version in case anyone wants to use and edit it (Word is often easier to edit). It is licensed CC BY, which means you can use it and change it if state that it’s adapted from mine as the original source.

Hendricks Philosophy Paper Rubric (PDF)

Hendricks Philosophy Paper Rubric (MS Word)

 

Daily Nous had a post in May 2017 with what they called “An impressively detailed philosophy paper grading rubric,” by Micah T. Lewin.

 

 

Mara Harrell of Carnegie Mellon has created this rubric (MS Word) for marking philosophy essays, which is even more detailed than the one above.

 

This paper marking rubric by Melissa Jacquart includes point values for each cell, which is also an option. Giving points for each part of the rubric can make marking quicker, though it also be somewhat problematic because it’s hard to include every aspect of what makes a good paper in a rubric, and sometimes it’s how things work together that leads to a better essay even if some parts are not as strong as one might like.

 

The Teach Philosophy 101 website has a list of rubrics (including some of the above) that has some not only for grading essays, but also for other kinds of assignments.

 

I’d be happy to hear about other rubrics not on this list!

 

 

Collaborating with students on objectives & assessments

I just did a quick read of the following article:

Abdelmalak, M. (2016). Faculty-Student Partnerships in Assessment. IJTLHE : International Journal of Teaching and Learning in Higher Education, 28(2), 193–203.

See the TOC for this issue, with link to the open-access PDF of the article, here.

The article reports on a study of a course in Education in which 6 graduate students collaborated with the professor on developing the course objectives, the assessments to meet those, and the criteria for assessing the work. The students brainstormed ideas, and then they agreed on objectives, goals, criteria based on what they shared in those ideas, and based on negotiations afterwards. Clearly this process would work best in a small class.

The author found that for the grad students involved,

  • collaborating on these things gave them a sense of control over their learning (unsurprising), which increased their motivation to learn.

However,

  • even though they had agreed to provide peer feedback on a writing assignment, most felt uncomfortable providing deep feedback to their peers due to a sense of lack of knowledge and a reticence to take up a perceived position of power over other students
  • some found the whole process difficult because they were used to an instructor deciding all of these things for them.

 

I have been thinking a lot lately about getting students more involved in creating assignments, though mostly what I teach are first-year courses and I think their lack of knowledge about the subject at that point means it would be best to not have them try to decide all the assignments. Plus, I have over 100 students in some of my first-year courses, and that makes such things difficult.

But I think something like this could work in a 4th year course (my 4th year course is max 25 students). The students still might not be able to come up with objectives that have to do with specific content they have yet to learn, but they might be able to come up with good ones about other aspects of the course; and I like the idea of them deciding on assignments after and grounded in the objectives they hope to achieve. Why write a paper, for example? Just because that’s what we always do in Philosophy, or for some other reason? What are we trying to achieve by writing papers? Are there other ways to achieve those goals?

In my experience, most fourth-year students in Philosophy courses don’t have too much issue with providing peer feedback that is critical and useful, so I don’t think I’d run into that problem. But it might be a bit difficult for them to go through this whole exercise because they’re not used to doing it. I think it would be really useful for them to work through why courses are designed as they are, and re-design them as needed to fit goals that are shared by the class.

I haven’t taught a 4th year course since 2014, and I’m not scheduled to do so next year either, but maybe the next time I do I’ll try something like this. Perhaps not for all the assignments, but for one or two to start with.

Has anyone tried anything like this before? If so, how did it go?


Update later on Aug. 12:

Robin DeRosa responded on Twitter that she had done this sort of thing with a first-year composition class–see the thread of that conversation here.

The syllabus, with student-created objectives and policies for that course, is here.

I had thought this wouldn’t work with first-years, but I can see how it works for a composition course in which students come in with some general knowledge about writing–you can get a sense of that from the objectives they created.

For first-year philosophy students, I think they might have a harder time determining just what they want to get out of a course when many of them don’t even really know what philosophy is yet, or why it is worth taking a course on!

Robin had a good suggestion:

So one could have them collaborate on one or two things that they can bring to the table.

And I love this point:

 

Also, Juliet O’Brien gave some great ideas via Twitter, which I’ll just post here as they are pretty self-explanatory I think!

 

 

You can see more about Juliet’s courses from this page: https://metametamedieval.com/courses/

And here is a link to a PDF that explains some of what she’s talking about above.

 

Presentation on SoTL research re: peer feedback

In mid-November I gave a presentation at the SoTL Symposium in Banff, Alberta, Canada, sponsored by Mount Royal University.

It’s a little difficult to describe this complex research, so I’ll let my (long) abstract for the presentation tell at least part of the story.


750-word abstract

Title: Tracking a dose-response curve for peer feedback on writing

There is a good deal of research showing that peer feedback can contribute to improvements in student writing (Cho & MacArthur, 2010; Crossman & Kite, 2012). Though intuitively one might think that students would benefit most from receiving peer comments on their written work, several studies have shown that student writing benefits both from comments given as well as comments received–indeed, sometimes the former more than the latter (Li, Liu & Steckelberg, 2010; Cho & MacArthur, 2011).

There are, however, some gaps in the literature on the impact of peer feedback on improving student writing. First, most studies published on this topic consider the effect of peer feedback on revisions to a single essay, rather than on whether students use peer comments on one essay when writing another essay. Cho and MacArthur (2011) is an exception: the authors found that students who wrote reviews of writing samples by students in a past course produced better writing on a different topic than those who either only read those samples or who read something else. In addition, there is little research on what one might call a “dose-response” curve for the impact of peer feedback on student writing—how are the “doses” of peer feedback related to the “response” of improvement in writing? It could be that peer feedback is more effective in improving writing after a certain number of feedback sessions, and/or that there are diminishing returns after quite a few sessions.

To address these gaps in the literature, we designed a research study focusing on peer feedback in a first-year, writing intensive course at a large university in North America. In this course students write an essay every two weeks, and they meet every week for a full year in groups of four plus their professor to give comments on each others’ essays (the same group stays together for half or the full year, depending on the instructor). With between 20 and 22 such meetings per year, students get a heavy dose of peer feedback sessions, and this is a good opportunity to measure the dose-response curve mentioned above. We can also test the difference in the dose-response curve for the peer feedback groups that change halfway through the year versus those who remain the same over the year. Further, we can evaluate the degree to which students use comments given by others, as well as comments they give to others, on later essays.

While at times researchers try to gauge improvement in student work on the basis of peer feedback by looking at coarse evaluations of quality before and after peer feedback (e.g., Sullivan & Pratt, 1996; Braine, 2001), because many things besides peer feedback could go into improving the quality of student work, more specific links between what is said in peer feedback and changes in student work are preferable. Thus, we will compare each student’s later essays with comments given to them (and those they gave to others) on previous ones, to see if the comments are reflected in the later essays, using a process similar to that described in Hewett (2000).

During the 2013-2014 academic year we ran a pilot study with just one of those sections (sixteen students, out of whom thirteen agreed to participate), to refine our data collection and analysis methods. For the pilot program we collected ten essays from each of the students who agreed to participate, comments they received from their peers on those essays, as well as comments they gave to their peers. For each essay, students received comments from three other students plus the instructor. We will use the instructor comments to, first, see whether student comments begin to approach instructor comments over time, and to isolate those things that only students commented on (not the instructor) to see if students use those in their essays (or if they mainly focus on those things that the instructor said also).

In this session, the Principal Investigator will report on the results of this pilot study and what we have learned about dealing with such a large data set, whether we can see any patterns from this pilot group of thirteen students, and how we will design a larger study on the basis of these results.


 

It turned out that we were still in the process of coding all the data when I gave the presentation, so we don’t yet have full results. We have coded all the comments on all the essays (10 essays from 13 participants), but are still coding the essays themselves (had finished 10 essays each from 6 participants, so a total of 60 essays).

I’m not sure the slides themselves tell the whole story very clearly, but I’m happy to answer questions if anyone has any. I’m saving up writing a narrative about the results until we have the full results in (hopefully in a couple of months!).

We’re also putting in a grant proposal to run the study with a larger sample (didn’t get a grant last year we were trying to get…will try again this year).

Here are the slides!

Non-disposable assignments in Intro to Philosophy

NoDisposableAssignments

Remixed from two CC0 images on Pixabay: trash can and No symbol

Disposable assignments

In the past couple of years I’ve really been grabbed by the issue of “disposable assignments,” as discussed by David Wiley here:

These are assignments that students complain about doing and faculty complain about grading. They’re assignments that add no value to the world – after a student spends three hours creating it, a teacher spends 30 minutes grading it, and then the student throws it away. Not only do these assignments add no value to the world, they actually suck value out of the world.

A non-disposable assignment, then, is one that adds value to the world beyond simply being something the students have to do to get a grade. A similar idea is expressed by Derek Bruff in a post on the idea of “students as producers”–treating students as producers of knowledge, rather than only as consumers: Bruff talks about students creating work for “authentic audiences,” beyond just the teacher.

Wiley gives an example of a non-disposable assignment: students taking instructional materials in the course (which are openly licensed) and revising/remixing them to create tutorials for future students in the course. Other examples can be found in this growing list of examples of open pedagogy. One that I often hear about is asking students to edit or create Wikipedia articles. Or students could post their work more locally, but still have it be publicly visible, such as what Judy Chan does with her students’ research projects at UBC (click on “team projects” in the various years). Simon Bates has his physics students create learning objects to help their peers (see this story for more).

Students as producers in philosophy courses

I have already started to ask students to do some activities that could add value to the world, whether to their fellow students and/or beyond.

  • In a second-year moral theory course I asked them to sign up for 1-2 days on which to do “reading notes” on the class wiki page: they had to outline one of the main arguments in the text assigned for that day and write down questions for their small group to discuss. You can see those here (organized by group number).
  • In a first-year, introduction to philosophy course I have asked students to:
    • blog about what they think philosophy is, both at the beginning and end of the course–this, I thought, could provide some interesting information to others about what our students think “philosophy” is. I don’t have those blog posts visible anymore because I didn’t ask students if I could keep them posted after the course was finished (d’oh!!).
    • write a blog post describing how they see philosophical activity going on in the world around them, beyond the class–I thought this could be useful to show the range of what can count as philosophical activity. I do still have those posts up (but not for long, because again I forgot to ask for permission to keep the posts up after the course is finished…I will do that this term!): http://blogs.ubc.ca/phil102 (click on “philosophy in the world”)

 

But now that I’m working on my Intro to Philosophy course for Fall 2015 (see planning doc here), I’m trying to think through some other options for assignments with authentic audiences and that add value to the world. Here are some ideas (not that I’m going to implement all of these; I’m just brainstorming).

  • Editing Wikipedia articles on philosophy
    • This is a big task; it requires that students learn how to do so (not just technologically, but in terms of the rules and practices of Wikipedia), plus determining which articles need editing, etc.
    • I would prefer to start with students creating Wikipedia-style articles on philosophers or texts on the UBC Wiki first. Then other students (in future classes) could edit those, and then maybe eventually we could move to doing something on Wikipedia itself (the content would be good, and maybe students would be motivated to move some of it over to Wikipedia at that point).
  • Creating tutorials or other “learning objects” for their fellow students and for the public
    • As noted above, Simon Bates does this in his Physics 101 course, and I can pretty easily see how one might ask students to do so for basic physics concepts. But why not do so for some basic philosophy concepts too?
      • e.g., find something you find difficult in the course, and once you feel you have a handle on it, create something to help other students
    • could be done in groups (probably best, with a large class like Intro to Phil (150 students))
    • could be text based, but better if also incorporates some other kinds of visual or auditory elements (e.g., a video, or incorporating images, or slides or something)
  • Creating study questions or suggestions of “what to focus on” for the readings
    • students often get lost in reading primary philosophical texts, and I haven’t yet managed to write up study questions or suggestions for what to focus on for each reading. This would definitely be useful to other students.
    • But wouldn’t it be cruel to ask students to do this for later students when I haven’t done it for them myself? and do I have time to do this before the Fall term this year? Unfortunately not.
  • Creating lists of “common problems” or advice for writing, after doing peer review of each others’ work and self-reflecting on their own
    • I do provide quite a lot of writing advice to students, but I wonder if advice coming from students’ direct experience in my courses might be helpful to later students?
  • Creating possible exam questions
    • I ask students to do this informally, in groups, as part of the review for the final exam. But why not formalize this somehow so their suggestions are posted publicly? The course page on the UBC Wiki seems like a good place, at least to start. Then students could see them from year to year.
    • A number of instructors at UBC use PeerWise as a tool for students to ask and answer questions. It seems like an interesting thing, but:
      • It’s not public; but it could be used to generate questions and then the best ones could be made public somewhere
      • It’s limited to multiple choice questions, which I hardly ever use (and never on exams)

 

Those are my ideas for now. Have any others? Or comments on any of this? Please comment, below!

Rubrics and peer feedback

I’ve been participating in an open, online course called Human MOOC: Humanizing Online Instruction. It’s officially over now, but I’m just completing a couple of final things from it.

One of the sections was on peer review/peer feedback by students of each others’ work. There was a link to a very helpful resource on peer feedback from the teaching and learning centre at Washington University in St. Louis. This page, linked to the previous one, is also very useful: “How to Plan and Guide In-class Peer Review Sessions.” A couple of things struck me about these resources that I wanted to comment on briefly.

What rubric/criteria should students use to do peer review?

On the first resource linked above, the following is stated:

Some instructors ask their students to evaluate their peers’ writing using the same criteria the instructor uses when grading papers (e.g., quality of thesis, adequacy of support, coherence, etc.). Undergraduate students often have an inadequate understanding of these criteria, and as a result, they either ignore or inappropriately apply such criteria during peer-review sessions (Nilson 2003).

The second resource states similarly:

The role of the peer-reviewer should be that of a reader, not an evaluator or grader. Do not replicate the grading criteria when designing these worksheets. Your students will not necessarily be qualified to apply these criteria effectively, and they may feel uncomfortable if they are given the responsibility to pronounce an overall judgment on their peers’ work.

This makes sense, though at the same time it’s troubling because if the students can’t understand the rubrics we use to mark their work, then how can they understand why they got the mark they did, or what they need to do to improve? It seems to me the answer here is not to ask students to use a different rubric when doing peer review than what we use to mark, but changing the rubric we use to mark so that it makes more sense to students (if there are comprehension problems). Now, I haven’t read the work by Nilson cited above, but it would be interesting to look more carefully into what undergraduate students tend to understand or not understand, or why, and then change one’s rubric accordingly.

One way one might do this, perhaps, is to ask them to use one’s marking rubric to evaluate sample essays and then invite feedback on the rubric as/after they are doing this. Then one can maybe catch some of the things students don’t understand before one uses the rubric for marking the essays?

Mock peer review session

The second resource suggests that one holds a mock session to begin with, which seems an excellent idea. It connects with the importance of training students in peer review before asking them to engage in it on work for the course (as discussed in Sluijsmans et al., 2002).

The idea would be to give them a “fake” essay of a kind similar to what they need to write, give them the peer review worksheet, and ask them to come up with comments on the paper. This can be done individually or in groups. Then, in class, have students give their comments to the whole group and the instructor writes them down on something that can be shown on the screen (or, alternatively, one could have them write the comments on a shared document online so they could be projected easily and the instructor doesn’t have to re-write them!). Then the class can have a discussion on the essay, the comments, and the marking worksheet/rubric, to clear up any confusion or help students improve their comments–e.g., moving from “good introduction” to saying what about the introduction is good, in particular.

This is an excellent idea, and I’m going to incorporate it in my upcoming philosophy class this summer. In Arts One we meet every week to do peer review of essays, in groups of four students plus the prof, so we can help students learn how to do peer review well on an almost one-to-one basis. And, since they do it every week for a year, they get quite good at it after awhile, even a very short time, actually!

 

Self-assessment

I could have sworn that the resources linked above from Washington University also talked about the value of students doing self-assessment of their own work, but now I can’t find that on those pages. But I was thinking that after they do peer feedback on each others’ work, it would be useful for them to go back to their own work and give feedback on it. It seems to me that after reading and commenting on others’ work, seeing what works/what doesn’t work, one could come to one’s own with fresh eyes, having learned from others’ work and also having distanced oneself from one’s own a bit.

I think I’ll try asking students to submit the peer review worksheet on their own essays after doing the peer feedback on others’, when they turn in their drafts post-peer-feedback.

 

Works cited

Nilson, Linda. (2003). “Improving Student Peer Feedback.” College Teaching, 51 (1), p. 34-38.
Sluijsmans, D. M. A., Brand-Gruwel, S., van Merriënboer, J. J. G., & Bastiaens, T. J. (2002). The training of peer assessment skills to promote the development of reflection skills in teacher education. Studies in Educational Evaluation, 29(1), 23–42. http://doi.org/10.1016/S0191-491X(03)90003-4

 

 

Media!

So, this is kind of exciting for me. I’ve been mentioned in some articles and blog posts, and even interviewed!

First, as a result of my presentation at the Open Education Conference 2014 in Washington, DC, I was interviewed by Jenni Hayman of the Open Policy Network about UBC’s Policy 81. You can see a video recording of this interview, which was done via Skype, on the OPN blog.

Most recently, there was a writeup of my research on peer feedback on writing, on the BCcampus website.

And then there was an article about the three Faculty Fellows (including me) with the BCcampus Open Textbook program for 2014-2015.

So it’s not the New York Times or even the CBC, but hey, it’s a start.

Authentic assessment and philosophy

In order to prepare for a meeting of the Scholarship of Teaching and Learning Community of Practice, I recently started reading a few articles on “authentic assessment.” I have considered this idea before (see short blog post here), but I thought I’d write a bit more about just what authentic assessment is and how it might be implemented in philosophy.

Authentic assessment–what

A brief overview of authentic assessment can be found in Svinicki (2004). According to Svinicki, authentic assessment “is based on student activities that replicate real world performances as closely as possible” (23). She also lists several criteria for assessments to be authentic, from Wiggins (1998):

 1. The assessment is realistic; it reflects the way the information or skills would be used in the “real world.”

2. The assessment requires judgment and innovation; it is based on solving unstructured problems that could easily have more than one right answer and, as such, requires the learner to make informed choices.

3. The assessment asks the student to “do” the subject, that is, to go through the procedures that are typical to the discipline under study.

4. The assessment is done in situations as similar to the contexts in which the related skills are performed as possible.

5. The assessment requires the student to demonstrate a wide range of skills that are related to the complex problem, including some that involve judgment.

6. The assessment allows for feedback, practice, and second chances to solve the problem being addressed. (23-24)

She points to an example of how one might assign a paper as an authentic assessment. Rather than just writing an essay about law generally (perhaps legal theory?), one might ask students to write an essay arguing for why a particular law should be changed. Or even better, write a letter to legislators with that argument (25).

Turns out there are numerous lists of what criteria should be used for authentic assessment, though (not surprising?). I have only looked at a few articles, and only those that are available for easy reading online (i.e., not books, or articles in books, or articles in journals to which our library does not have a digital subscription–I know this is lazy, but I’m not doing a major lit review here!). Here’s what I’ve found.

In Ashford-Rowe et al. (2014), eight questions are given that are said to get to the essential aspects of authentic assessment. These were first developed from a literature review on authentic assessment, then subjected to evaluation and discussion by several experts in educational design and assessment, and then used to redesign a module for a course upon which they gathered student and instructor feedback to determine whether the redesign solved some of the problems faced in the earlier design.

(1) To what extent does the assessment activity challenge the student?

(2)  Is a performance, or product, required as a final assessment outcome?

(3)  Does the assessment activity require that transfer of learning has occurred, by means of demonstration of skill?

(4)  Does the assessment activity require that metacognition is demonstrated?

(5)  Does the assessment require a product or performance that could be recognised as authentic by a client or stakeholder? (accuracy)

(6)  Is fidelity required in the assessment environment? And the assessment tools (actual or simulated)?

(7)  Does the assessment activity require discussion and feedback?

(8)  Does the assessment activity require that students collaborate? (219-220)

Regarding number 3, transfer of learning, the authors state: “The authentic assessment activity should support the notion that knowledge and skills learnt in one area can be applied within other, often unrelated, areas” (208). I think the idea here is that the knowledge and skills being assessed should be ones that can transfer to environments beyond the academic setting, which is the whole idea with authentic assessment I think.

Number 4, metacognition, has to do with self-assessment, monitoring one’s own progress, the quality of one’s work, reflecting on the what one is doing and how it is useful beyond the classroom, etc.

Number 6, regarding fidelity, has to do with the degree to which the environment in which the assessment takes place, and the tools used, are similar to what will be used and how, outside of the academic setting.

The point of number 8, collaboration, is that, as the authors state, “The ability to collaborate is indispensable in most work environments” (210). So having assessments that involve collaboration would be important to their authenticity for many work environments. [Though not all, perhaps. And not all authentic assessment needs to be tied to the workplace, right? Couldn’t it be that students are developing skills and attitudes that they can use in other aspects of their lives outside of an educational context?]

Gulikers et al. (2004) define authentic assessment as “an assessment requiring students to use the same competencies, or combinations of knowledge, skills, and attitudes, that they need to apply in the criterion situation in professional life” (69). They took a somewhat different approach to determining the nature of authentic assessments than that reflected in the two lists above. They, too, started with a literature review, but from that focused on five dimensions of authentic assessments, each of which can vary in their authenticity:

(a) the assessment task

(b) the physical context

(c) the social context

(d) the assessment result or form

(e) the assessment criteria (70)

Whereas the above two lists look at the kinds of qualities an assessment should have to count as “authentic,” this list looks at several dimensions of assessments and then considers what sorts of qualities in each dimension would make an assessment more or less authentic.

So, for example, an authentic task would be, given their definition of authentic assessment as connected to professional practice, one that students would face in their professional lives. Specifically, they define an authentic task as one that “resembles the criterion task with respect to the integration of knowledge, skills, and attitudes, its complexity, and its ownership” (71), where ownership has to do with who develops the problem and solution, the employee or the employer (I think that’s their point).

The physical context has to do with what sorts of physical objects people will be working on, and also the tools they will generally be using. It makes assessments less authentic if we deprive students of tools in academic settings that they will be allowed to use in professional settings, or give them tools in academic settings that they generally won’t have access to in professional settings. Time constraints for completing the task are also relevant here, for if professionals have days to complete a task, asking students to do it in hours is less authentic.

The social context has to do with how one would be working with others (or not) in the professional setting. Specifically, they specify that if the task in the professional setting would involve collaboration, then the assessment should do so, but not otherwise.

The assessment result or form has to do with the product created through the task. It should be something that students could be asked to do in their professional lives, something that “permits making valid inferences about the underlying competencies,” which may require more than one task, with a variety of “indicators of learning” (75).

Finally, the criteria for the assessment should be similar to those used in a professional setting and connected to professional competencies.

 

Authentic assessment and philosophy

Though Gulikers et al. (2004) tie authentic assessment pretty closely to professional life, and thus what they say might seem to be most relevant to disciplines where professional practice is directly part of courses (such as medicine, business, architecture, clinical psychology, and more), the overview in Svinicki (2004) suggests that authentic assessments could take place in a wide variety of disciplines. What could it look like in philosophy?

I think this is a somewhat tricky question, because unlike some other fields, where what one studies is quite directly related to a particular kind of activity one might engage in after receiving a degree, philosophy is a field in which we practice skills and develop attitudes that can be used in a wide variety of activities, both within and beyond one’s professional life. What are those skills and attitudes? Well, that’s a whole different issue that could take months to determine (and we’re working on some of that by developing program outcomes for our major in philosophy here at UBC), but for now let’s just stick with the easy, but overly vague answers like: the ability to reason clearly; to analyze problems into their component parts and see interrelationships between these; to consider implications of particular beliefs or actions; to make a strong case for one approach to a problem over another; to identify assumptions lying behind various beliefs, approaches, practices; to locate the fundamental disagreements between two or more “sides” to a debate and thereby possibly find a way forward; to communicate clearly, orally and in writing; to take a charitable attitude towards opponents and focus on their arguments rather than the persons involved; and more.

So what could it mean to do a task in philosophy in a similar way, with similar tools, for example, as what one might encounter in a work environment? Because the skills and attitudes developed in philosophy might be used in many different work environments, which one do we pick? Or, even more broadly, since many of these skills and attitudes can be practiced in everyday life, why restrict ourselves to what one might do in a work environment?

Perhaps, though, this means we have a lot more leeway, which could be a good thing. Maybe authentic assessments in philosophy could be anything that connects to what one might do with philosophical thinking, speaking and writing skills outside of the educational setting. And if several courses included them during a students’ educational career, they could perhaps see how philosophy can be valuable in many aspects of their lives, having done different sorts of authentic assessments applying those skills to different kinds of activities.

When I came up with a couple of possible authentic assessments in philosophy courses last summer, I believe I was thinking along these lines–something that the students would do that would mirror an activity they might engage in outside of class. One, which I implemented this year in my moral theory course, asked students to apply the moral theories we’re studying to a moral dilemma or issue of some kind. This isn’t exactly like an authentic assessment, though, because I’m not sure that I would expect anyone in their everyday lives to read Kant and Mill and then try to apply them to moral dilemmas they face. Maybe some people do, but I’m not really sure that’s the main value of normative moral theories (I’m still working on what I think that value is, exactly).

Another one of the suggested assignments from that earlier blog post was that students would reflect on how they use philosophical thinking or speaking or writing in their lives outside of the course. That one isn’t asking them to do so, though, so it’s not like mirroring a task they might use outside the class; it’s just asking them to reflect on how they already do so.

So I think I need to consider further just what an authentic assessment in philosophy might look like (the one from Svinicki (2004), above, about writing a letter to legislators to change a law is a good candidate), and how I might include one in a course I teach in the future. Possible ideas off the top of my head:

  • Take a discussion of a moral issue (for example) in the media and clearly lay out the positions on the various “sides” and what arguments underlie those. Evaluate those arguments. (We do this sort of thing all the time in philosophy, but not always by starting with media reports, which would be the sort of thing one might do in one’s everyday life.) Or, identify assumptions in those positions.
  • Write a letter to the editor or an op-ed piece about some particular moral or other issue, laying out clear arguments for your case.
  • Participate in or even facilitate a meeting of a Socrates Cafe, a philosophical discussion held in a public place for anyone who is interested to join.
  • Make a case to the university, or your employer, or someone else for something that you’d like to see changed. Give a clear, logical argument for why it should be changed, and how. Can collaborate with others on this project.

Okay, this is hard.

And it occurs to me that some of what we already do might be like an authentic activity, even if not an authentic assessment. For example, when we ask students to engage in philosophical discussion in small groups during class, this is the sort of thing they might also do in their lives outside of class (don’t know how many do, but we are giving them practice for improving such activities in the future).

Hmmm…gotta think more on this…

 

Any ideas are welcome, in the comments below!

 

Works Cited

Ashford-Rowe, K., Herrington, J. & Brown, C. (2014). Establishing the critical elements that determine authentic assessment. Assessment & Evaluation in Higher Education, 39(2), 205-222. DOI: 10.1080/02602938.2013.819566

Gulikers, J.T.M., Bastiaens, T.J., Kirschner, P.A. (2004). A five-dimensional framework for authentic assessment. Educational Technology Research and Development, 52(3), 67-86. Available on JSTOR, here: http://www.jstor.org/stable/30220391?

Svinicki, M. D. (2004). Authentic assessment: Testing in reality. New Directions in Teaching and Learning, 100, 23-29. Available behind a paywall, here: http://onlinelibrary.wiley.com/doi/10.1002/tl.167/abstract

Wiggins, G. (1998). Educative Assessment: Designing Assessments to Inform and Improve Student Performance. San Francisco: Jossey-Bass.

Closing the feedback loop

I attended the biannual meeting of the American Association of Philosophy Teachers July 30-Aug 2, 2014, and got some fantastic suggestions/ideas for future teaching, as I did the last time I attended this conference. The AAPT workshop/conference is easily one of my top favourite conferences: it is so friendly, inviting, supportive, and there are great people to talk to about teaching philosophy as well as about life in general. I haven’t laughed this much, for so many days in succession, for a long time. It’s too bad this meeting is only held every two years, as these are people I’d sure like to see more often!

I’m going to take a few of blog posts to write down some of the (many) things that inspired me at this conference, that I’d like to try in my own teaching one way or another. There were many more things than I’m going to write about here—I have pages and pages of notes that I typed out during the conference. But in this and a couple of future posts, I’ll focus on just a few.

Broken feedback loop: when did you not respond well to feedback?

Rebecca Scott from Loyola University Chicago facilitated a session on closing the feedback loop, which started off in a really helpful way: she asked us to consider (among other things) times when we received feedback from someone (whether in the context of our academic lives or other aspects of our lives) and didn’t respond in the way that we now think would be most helpful.

Kawazu Loop Bridge, Flickr photo shared by Tanaka Juuyoh, licensed CCBY 2.0

Kawazu Loop Bridge, Flickr photo shared by Tanaka Juuyoh, licensed CCBY 2.0

I won’t give details on either situation, but one of them had to do with feedback I received at the end of a course that utterly shocked and floored me. More than one student said that I did something that was so very far from who I think I am that I just couldn’t believe it was true. All I could think of was: “How could someone think I was doing that? There’s no way I did that! They must be wrong.” I didn’t entertain (at first) the idea that the feedback could be right in some way. It just didn’t fit with who I thought I was.

Remembering this situation helped put me into the mindset of students receiving critical feedback (or, at least, helped move me closer to that I hope), and not believe it, getting angry, indignant, even lashing out. When that happens you are not even allowing yourself to think that the feedback might be true; since it doesn’t fit with who you think you are, your own evaluation of the quality of your work, the truth must be that whoever said that is simply wrong. I’m reminded of Socrates who, at least in Plato’s texts, would show his interlocutors that they didn’t know what they thought they knew, and for some the reaction was to just assume that Socrates must be wrong and to get angry with him.

Why might feedback not be incorporated into future work?

We came up with numerous reasons during the session, which I wrote down:

  • Getting emotional; taking things too personally; losing sight of the goal of feedback
  • Not caring about the work, just trying to get credit
  • Too motivated by grade, not enough by learning
  • Not believing that the feedback is true; e.g., coming into class with mindset that one is an A student b/c have gotten A’s so far, so don’t believe the instructor who gives a lower mark
  • Distrust of the instructor, institution, due to larger social issues/context
  • Not thinking that you could do any better, that you’re capable of improving even with feedback; including: getting discouraged at how much they have to change and thinking they can’t
  • Not seeing work as formative process; thinking that when the assignment is done you are done and don’t need to revisit it, to learn from it
  • Professor and students seeing diff goals of feedback; students might think that feedback is there to explain why they got the grade they did, but for the prof it might be there to show ways to improve
  • Not understanding the feedback
  • Not connecting feedback from past to future situations
  • Thinking that just reading the comments is enough to improve for later
  • Not having a clear idea of what good work looks like to aim for
  • Too much feedback; overwhelmed; don’t know what to do with it

The one that I find hardest to deal with (though many are quite challenging) is the first: the emotional reaction. It kept me from addressing my situation as well as I could have, and I can see how student emotional reactions could lead them to not want to even look at the feedback again or think about it at all.

A reflective assignment to close the feedback loop

Rebecca shared with us an assignment she gives to students that asks them to reflect on their feedback, that forces them to read it and consider it and reflect on what they want to change for the future based on it. And the first item on that assignment is a question, asking them what their immediate reaction was on receiving the feedback. The idea is that maybe if they have an outlet to write it down, to let you know their emotional reaction, this might help them move past it.

But I think the rest of the assignment might help with that too. Because it goes on to ask students to

  • write down how many comments they got in each of several categories (to help them see which areas they need to work on, and to ensure that they read the comments or at least skim them),
  • what grade they expected, what grade they got and what do they think explains the difference between these
  • how much of the feedback do they feel they understand
  • what two things do they want to work on for the next assignment, and
  • whether they have any questions or comments about the feedback they received

How might all of this help with the emotional reaction issue? Besides making them continue to think about the feedback even if they get angry instead of just ignoring it, it also gives them a chance to give feedback on the feedback, to try to figure out what could explain the difference between the grade they expected and the grade they got, which could include thinking about the feedback and how it might suggest that the grade makes at least some sense. Or, if they disagree with the feedback, it gives them an outlet to do so, and the instructor can follow up with them later to discuss the issue.

How I’d like to adapt this assignment, and also address a couple of the other problems above

I like this idea of a reflection on the feedback that you submit to the instructor, but I also want them to have a kind of running record of the feedback they’ve received, the 2-3 things they want to work on for the next time, what they did well and want to keep doing, etc. In addition, I want to make sure that they have to look back at this feedback for the next paper they write.

So, here’s an idea.

1. For the Arts One course I teach, in which students write a paper every 2 weeks (12 over the course of a year), I think I’ll ask them to include on each new essay:

  • a list of at least two things they tried to do better on this one, based on feedback from the last one
  • at least one thing they themselves noticed from their previous essay that either they think was good or that they would like to improve on, that no one else pointed out
    • this is so that they don’t just look back at the feedback but also back at their previous essay and see what they themselves think, in order to do some self-assessment

2. I would also like to institute a policy in terms of my own feedback: that I will point out one or two instances of a certain type of mistake, and ask them to look for more instances (if I saw more in the essay, that is). Then, also on the next essay:

  • Point out at least one other place in the previous essay where one of the comments I made applies elsewhere too.
    • This is again so that they need to go do some self assessment of their work, and so I don’t need to go through and point out every single mistake. I think this could help with the issue of being overwhelmed by too much feedback

3. Finally, I think it would be great if they could keep a learning log, digitally, where they keep track of, for each essay: the comments they’ve gotten from peers, at least two things from me that they want to work on, the things they’re doing well and want to keep doing. That way they have a running record and periodically I can ask them to reflect on whether there are any patterns/repeated comments, or whether they are getting better because certain sorts of comments aren’t being said anymore.

These things could hopefully all help with the issue of not connecting feedback on previous work to later work. But I have to figure out how much of this is adding too much work for the students, or whether it is all so pedagogically valuable as to be worth it.

Back to when I didn’t respond well

At first, I just shut down. So I can understand when students do that. I didn’t want to think about it and just wanted to move past it. But I did eventually do something: I emailed all my students and asked them to fill in another feedback form, anonymously, that would just go to me. I asked them to be as specific as possible, because I didn’t get quite enough details on the first one. I got a few more details on this second round, which helped me understand some of the concerns expressed and how students may have come to the conclusion they did (and even that I might have been unconsciously doing some of what they thought, even though I’m still reluctant to believe that). But not entirely fully. I think there was some miscommunication somewhere that I just can’t rectify now.

All the more reason to give students more of a chance to give feedback during the course so problems can be solved earlier! (I just did it once, during the first term, and not at all during the second: lesson learned!)

 

Authentic assessments in two PHIL classes

For the blended learning course I’m taking on teaching a blended learning course, we were asked to design an “authentic assessment” for one of our courses. An authentic assessment, from what I understand, is one in which students are either simulating or doing the very sorts of activities you hope they will be able to do outside of class, after they take the course. In addition, according to some of the text of the course I’m taking,

According to Eisner (1993), authentic assessment projects should reveal how students go about solving the problems (process) and should have more than one correct solution. They should:

  • Promote ‘how’ knowledge as opposed to the ‘what’ knowledge measured in ‘traditional’ assignments;
  • Provide a way for students to develop an understanding of complex course material that will serve them beyond the classroom;
  • Encourage higher-order cognitive skills;
  • Involve students more extensively in the development of the assessment and the grading criteria.

PHIL 102: Introduction to Philosophy

Here is an idea for an authentic assessment activity for my Introduction to Philosophy course.

Rationale

In PHIL 102, Introduction to Philosophy, the main theme of the course is investigating what philosophy is, what philosophers do, and the value of these things, both by reading about what philosophers themselves have said about these questions, and by considering what the philosophers whose texts we are reading are doing with their lives and their writing.

One of the things I’d like students to be able to do by the end of the course is to recognize ways in which they themselves engage in philosophical activity, in their everyday lives.

Activity

Students will write a reflective blog post towards the end of the term in which they discuss two things they do in their lives that could show philosophical thinking or addressing of philosophical questions. They will also add a short summary of their post for a class wiki page on this question.

Learning objective addressed: “Explain at least two ways in which you yourself use philosophical thinking or address philosophical questions in your everyday life.”

Instructions

Now that the course is nearly over, you should have a pretty good idea of what philosophy is and what philosophers do. It’s  time to consider the ways in which you yourself engage in philosophy. This assignment consists of two parts:

1. Write a blog post on the class blog in which you do the following:

  • Discuss at least two ways in which you yourself use philosophical thinking or consider philosophical questions in your own life, your own day-to-day activities, your major life decisions, etc.
  • Explain why these could be considered “philosophical,” referring to at least one of the philosophers or texts or ideas we’ve discussed in class.
  • This blog post should be at least 300 words long, but no longer than 800 words

2. After you’ve completed your blog post, contribute your two ways to the class wiki page for this assignment [give URL for this here].

  • Write a one or two-sentence summary of each of the ways you engage in philosophical thinking or activity and put them as bullet points on the wiki page.
  • Christina will then organize these under general categories after they are posted, to make them easier to read through, and we’ll discuss the results in class

Marking criteria

This assignment will be marked using a three-level system:

1. Plus:

  • Your blog post discusses at least two ways in which you engage in philosophical thinking or address philosophical questions in your life
  • Your blog post adequately explains how these things are philosophical, referring to at least one of the philosophers/texts/ideas we’ve discussed in class.
  • Your blog post is between 300 and 800 words long.
  • You wrote a one- or two-sentence summary of each of the two things you discussed in your post, on the class wiki page.
  • Both the post and the wiki entry were completed by the due date and time.

2. Minus:

  • Your blog post discusses only one way in which you engage in philosophical thinking or address philosophical questions in your life, or
  • Your blog post does not adequately explain how this/these activities are philosophical, and/or doesn’t refer to at least one of the philosophers/texts/ideas we’ve discussed in class, or
  • Your blog post is less than 300 words or more than 800 words, or
  • Your blog post was fine, but you didn’t submit your one- or two-sentence summary of each point discussed in the post on the wiki page, or
  • Your blog post and/or wiki entry were submitted after the due date and time, but no later than six days afterwards.

3. Zero:

  • Your post and/or wiki page entry was not completed, or
  • Your blog post and/or wiki entry were completed seven or more days after the due date.

 

Thoughts/questions

I wanted this assignment to not only be useful for the students writing the posts themselves, to get them to think about how philosophy plays a role in their own lives, but also to others. That’s why I thought of having them post to a wiki page–there are often over 100 students in this course, and reading that many different blog posts will be too much for anyone else visiting the course (my courses are on open sites, on UBC Blogs, so anyone can visit them; students always have the option of posting under a pseudonym, or with a password so only the rest of the class can read, or private to me if they choose).

But just having a list of one- or two-sentence summaries on a wiki page is too messy too. So I thought I’d try to categorize them myself after they’re posted, and say something like: 15 people said x, 8 people said y, etc.

Of course, this is more work for me. Any ideas on how to make it so that we have a kind of summary document that might be useful for students in the class as well as others, without me having to go through and categorize all the entries? It’s okay if I have to do so (it’s just busy work, and easy), but if there are other ways I’d love to hear them!

 

PHIL 230: INTRODUCTION TO MORAL THEORY

Here is an idea for an authentic assessment for this course. Students will be writing in a “moral issue” journal throughout the course, starting with what they think about a particular moral issue, then comparing this with what they think each of the philosophers we study would say about it, and then concluding with their thoughts on the value of trying to come up with moral theories such as the ones we’ve studied. For this assignment, I’d like students to be able to take what they’re reflecting on in their moral issue journals and refine part of it into a formal essay.

This way, they’ll be using what they have learned in the course in thinking about moral issues they may face around them in their everyday lives.

Moral issue paper

For this paper, you’ll be using what you’ve reflected on in your moral issue journal and writing a formal paper. The idea here, as with the moral issue journal, is to apply the moral theories we’ve been studying to a moral issue that you might face in your life, or one that involves a larger group of people such as a community or nation. In this way, you’ll be making connections between what we’re studying in class and your life beyond.

Instructions

Using the moral issue you’ve been focusing on in your moral issue journal, write an argumentative paper that argues for how a consequentialist and a Kantian would approach the issue. Include also your own view on whether one approach is better than the other for this particular issue, and why (or why not; it may just be that they are very different and there’s no clear reason to choose one over the other).

Parts of the essay

Note from the Guidelines for essays handout that your essay should have an introduction with your thesis statement, a conclusion that wraps up the essay in some way, and body paragraphs that provide adequate arguments for the conclusion.

Your thesis should include (note that a thesis can be more than one sentence):

  • A summary statement of what a consequentialist and a Kantian would say about the issue
  • A summary of your view on whether or not one approach is better

Be sure to explain the moral issue you’re addressing, early on in the essay.

Length

The essay should be between 5 and 8 pages, typed, double-spaced, with margins between 0.75 and 1 inches, and font size between 11 and 12 points. [Or 2000-3000 words?]

Quotes, paraphrases, and citing sources

Quotes vs paraphrases: It’s usually best to have a mixture of both. You should use quotes where it’s important to give the author’s exact words, where the words themselves help you to make a point. This is often the case when a passage can be interpreted in more than one way, and you want to justify your interpretation with the words of the author. You can also use quotes where you need an extended passage to make your point (be sure to indent quotes over 4 lines long, 5 spaces on the left).

Citing sources in the paragraphs: Whether you give a quote or paraphrase a specific point from the text, you should give a page number or section/paragraph number to show where the information can be found in the text. You choose your favourite citation style, or you can just give the author’s last name plus the page or section number, in parentheses: (Kant 55). (This is the MLA style.) If you are citing more than one text by an author, give a shortened version of the title of the text in the parentheses as well: (Kant, Religion 99).

Citing sources at the end of the essay: Be sure to give a works cited page that includes all the texts you cited in parentheses in the essay. Again, you can use any citation style you wish, but be sure to include all the information that that citation style requires. For example, you can see how to create a Works Cited list in MLA style here [give URL].

Avoid plagiarism: It is the policy of the Instructor to prosecute plagiarism to the fullest extent allowed by UBC. Any use of another’s words, including just a sentence or part of a sentence, without citation, constitutes plagiarism. Use of another’s ideas without citation does as well. To avoid plagiarism, always give a citation whenever you have taken ideas or direct words from another source. Please see this page on the course website for information on how to avoid plagiarism, especially when you’re paraphrasing ideas or quoting from another source—quite a lot of plagiarism is not on purpose, just because students don’t understand the rules! http://blogs.ubc.ca/phil102/resources/

Depth of explanation and narrowness vs. breadth and superficiality: It’s usually best to focus your paper on a small number of claims and argue for them in some depth rather than trying to range widely over a very large number of claims that you then only have space to justify very quickly. Pick the strongest points for each, consequentialism and Kantianism, and focus on those.

Audience you should write for: Write this essay as if you were writing for someone who is in the class, has not read the texts, and has not attended the class meetings (say, a friend or family member). Explain your view, and the arguments of the philosophers you discuss, in as much depth as would be needed to make them clear to such an audience.

Marking: See the marking rubric posted here on the course website [give URL].

Late penalty: 5 points off per weekday late, unless otherwise agreed to by the Instructor (may require documentation). I do not generally give extensions due to students’ workloads, only for things that are unexpected and unavoidable such as medical issues; so plan ahead if you have multiple assignments due around the time that this essay is due!

“Students as Producers” Assignments in Intro to PHIL

For the blended learning course I’m doing on teaching a blended learning course, we were asked to think about possible assignments that could fit the “students as producers” model, where that involves projects that “encompass open-ended problems or questions, a authentic audience and a degree of autonomy” (according to the text in the course). Here’s a nice overview by Derek Bruff of the idea of “students as producers.”

 

Here are two ideas for “student as producer” assignments for my Introduction to Philosophy course (PHIL 102).

1. Shared notes on the reading

One person in each small group (of 4-5 students) is responsible for taking notes on the reading and posting them before any lecture on that section. Students will sign up for specific dates to finish their notes by.

Notes must include:

  • A statement of what you think the main point/main conclusion in this section of the reading is. If there is more than one, pick just one of the main conclusions in the reading. Refer to a page number where this conclusion can be found (or section and paragraph number, if the reading has no page numbers).
  • How the author argues for this point: give the reasons/premises the author gives to support the conclusion. Refer to page numbers where these premises can be found (or section and paragraph numbers, if the reading has no page numbers).
  • Give one or more comments about what you’ve discussed above: is there anything you disagree with? If so, why? Or, is there something in it that you find particularly interesting? How? Or, do you have any questions about it?

These notes must be typed and shared with the class, on the class blog [insert URL for where to share them]. Be sure to tag the post you’ve written with the last name of the author (e.g., Plato, Epicurus).

Anyone in the class can review the sets of notes for each author, which is a great resource for reviewing the text! Any student can respond to a question posed in one of the posts, or make a comment in response to what a student has said about the reading; you don’t have to just do it for the person from your small group.

 

Since the above is only partly open-ended (sections (a) and (b) are not very open-ended), I thought of another assignment as well.

 

2. What would it be like to live like an Epicurean or a Stoic?

For this activity, you will need to imagine what it would be like to live as either an Epicurean or a Stoic (choose one). You’ll need to describe some aspects of your current life and then how they would change if you lived as either an Epicurean or a Stoic. For example, you could consider how the following might be different (or anything else you deem relevant):

a. What you choose to study/what your career might be

b. What you spend your money on

c. What your day to day routine is like, the main choices you make each day and how they might change

Write a blog post on the class blog describing how your life would be different if you were an Epicurean or a Stoic. Discuss at least two ways that your life would be different. Include in your post a reflection on whether you think this would be a good way to live or not, and why.

  • Be sure to tag it either “Epicureanism” or “Stoicism,” and put it under the category “Live like a…”
  • Your blog post should be at least 400 words long, but no more than 900
  • Refer to the text with page numbers or section/paragraph numbers to show where the author says something that justifies why your life would be the way you say it would

This activity will be marked on a three-level scale:

  • Plus:
  • You have described at least two aspects of your life that would be different and why, with specific page or paragraph references to at least one of the texts we’ve read
  • You have included a reflection on whether you think this would be a good way to live or not, and why
  • the blog post is between 400 and 900 words long
  • Minus:
  • You have described only one aspect of your life that would be different, and/or
  • You have not adequately explained why your life would be different, and/or
  • You have not given specific references to the text(s) where needed to support your claims, and/or
  • You have not included a reflection on whether you think this would be a good way to live or not, and why
  • The post is less than 400 words or more than 900 words long, and/or
  • The post is late, without an acceptable excuse for being so (one to six days late)
  • Zero:
  • The post was not completed, or
  • It was completed seven or more days late

 

How are these related to the “student as producer” idea?

I was thinking of “student as producer” as having to do with students making things to share with a wider audience, producing content that would be useful to others. The first assignment does that for other students in the course; the second, if the blog posts are on a public site rather than a closed site (which my class blogs usually are), may provide information that could be interesting and useful to a wider audience trying to understand what Epicureanism and Stoicism are all about.

I was also thinking that the second assignment could be considered a kind of “authentic assignment,” in that many of the ancient philosophers thought that the purpose of philosophy was to change your life, to cause you to live in a better way, to be happier. I considered making them actually live like Epicureans or Stoics for a day, but I’m not sure one would get much out of just one day of doing so. Maybe a week would give you a taste, but that may be too much to ask! So I decided to do a simulation instead.

I’d love to hear anyone’s thoughts on how I might make either one of these assignments more useful to students or a wider audience, or more “authentic.” I considered adding a collaborative element to the second one, having them do it in groups, but I got stuck on whose life they would start with to consider how that life would change if lived as an Epicurean or Stoic, and then I got stuck on how they’d share the duties for writing the blog post about it. Any suggestions here would be great!