I hate grading, and as a result don’t think enough about the potent and vital role of assessment in learning. I admit it’s a shortcoming.
On the train back to Barcelona from the IN3 offices yesterday, I had a provocative conversation with Julià Minguillón about the problem of structuring assignment schedules in virtual courses. Essentially, how might it be possible to make assignment deadlines more flexible for distance students? If deadlines for ongoing assignments are simply extended deeper through to the end of the course, wouldn’t there be a natural tendency on the part of most students to hand in their assignments at the latest possible moment, to the detriment of their learning? My own experience as both student and teacher (as well as the virtual learner data that the UOC has analyzed) confirms this fear…
While we were talking, my mind went immediately to incentives for students to hand in their work earlier than is strictly necessary. Such as bonuses to students who get their work in first… But, are there any meaningful and justifiable reasons to turn admission submission into a race?
So what if we had students submit their work in a forum in which other students could see that work? Students submitting later would be able to build on that work, and perhaps improve on it – indeed, that would be an expectation… But if we expected students who submitted later rounds of assignments to read the work of their forerunners, and to incorporate it, and to cite it, we would see the work of the “original” work identified and rewarded (perhaps with a bonus of some kind), much as innovative work in an academic field is recognized by early publication and citation. This would provide an incentive to publish work ahead of others, yet would still allow those who publish later to do well. Those relative latecomers would, however, be expected to account for, synthesize, criticize and augment the work of those who had published before them.
Is this idea too facile? Is anyone using this model already? This strikes me as an approach to submitting work and grading that might: a) provide for a more ‘open’ model of assignment submission and online publication, and b) more accurately model a real-world economy of sharing ideas via publishing.
I don’t suggest this approach would work in every instance. And as I mentioned at the top of this post, I don’t think near enough about assessment, so surely I’m missing something here…
I like this idea, Brian…a lot… precisely because it reflects how we write (or should write) in the real world, taking into consideration thinkers before and around us, building and innovating simultaneously. I like it, too, because it gives students a choice–hand work in early without the benefit of all those other ideas swirling about, or hand it in later and pull them in. Students should absolutely see each other’s work. I also had students read the work of students from previous semesters and years.
There is a problem, though, and that has to do with the reality of the fuller semesterized system being unaware of its own parts. Some students carry ridiculously heavy loads due to major requirements, and their profs seemingly have no idea (or care) about what might be going on in other courses. Far too many courses punctuate a semester with the same pattern of due dates instead of allowing students more flexibility in deadlines. Your plan could just add more to a student’s plate if s/he chooses a later due date and has to take on the other students’ work.
I also think that not everyone needs to complete the same work. What if students could complete a certain percentage of assignments as first-wave writers (without taking into account their peers’ work) and a certain percentage as reviewers (responding to the ideas of their peers) and a certain percentage as second-wave writers (putting forth their own ideas as well as synthesizing and building on their peers).
In other words, what if every student had to complete some assignments early and some later, to gain experience with both kinds of writing? Students would be able to plan ahead by signing up for Option A or Option B at the beginning of the semester, weaving a realistic workload for themselves.
I would also like to see more undergraduate e-journals and online conferences to which students submit their work as part of the learning & evaluation process.
Then there’s the question of just who is assessing whom. I have long argued for the students, as a group, coming up with grading rubrics as part of the course, and to assess themselves in a narrative evaluation and–if the school requires it–in a grade that they have to propose and defend.
~bg
Very interesting ideas, Brian.
How would you address papers submitted close together? With not enough time between them, there would be difficulty in citing previous work, etc…
We’ve got a few projects that might benefit from this first-mover assessment.
One of our Computer Sciences courses has all of the students writing their term project papers in a wiki, from first draft (not final copies only).
eg, see http://wiki.ucalgary.ca/page/Courses/Computer_Science/CPSC_203/CPSC_203_Template/CPSC203_Term_Projects
And we have a science course with students writing articles on a community blog site, with peer reviews resulting in the “best” papers being included in an online magazine and eventually printed in rural newspapers.
http://sciespace.ucalgary.ca
But I don’t believe either of these have any form of formal rewards for publishing early… hmm… Thanks for the food for thought!
I liked what you wrote about assessment and meaning. I have always grappled with this not only as a student but as a teacher/learner. I find the whole process redundant especially when I know that the ‘product’ was more valued than the process and that most of us rote memorized and forgot much of what we memorized a few days later. Journals, video, and just plain old talking/collaboration….gives me way more information about myself and my students.
Brian…totally agree with the idea of building on ones learning…i know you are referring to the university setting but this relates to secondary students (and younger) as well. I am fascinated with the ‘project’ approach to learning…and feel that in my own education something like the anthropology, was missing in my schooling…subjects compartmentalized and segregated….as distinct entities…(i digress as usual-here’s a great book: http://www.amazon.com/gp/reader/080704623X/ref=sib_dp_pt#reader-link-oops long url-sorry)
ANyway, I’ve been trying for the past few years to figure out a fair and fruitful/meaningful way to assess student processes….(its like taking pictures of what you see and not what you think others want to see!)
I have a comment about deadlines and flexibility in online courses. When I first began teaching online I valued the ability for students to have all the assignments and complete them whenever. I’ve now changed my thinking on that. Because a huge element of my course values transparency and social learning, all work is online and open to all. If someone submits something at the end of the course, it penalizing the other students who did not have a chance to learn from that student. If assessment is really FOR learning, it must occur with enough time for the learner and others to learn from it. Otherwise we’re just talking about evaluation which is really not the same thing.
As someone who is currently going through the constant barrage of assessment/evaluation I agree that there is something massively wrong with what is being done at the moment.
I still think that the most useful motivator is real world contribution.
I also think that peer-evaluation (done properly) can be very powerful as in the long run it gives you the tools to look at your own work from the perspective of others.
hi,
as usual, there is no a universal solution; when you assignment is about solving some exercises with only one possible answer, you cannot allow such solution to be posted and discussed; if you are more in a discussion or debate based answer, then it is possible to take care of who is answering first, who is adding real comments and who is just copying and pasting, we are indeed already doing that
regarding students grading themselves, we allow them to do that when they work in groups: they grade themselves and the other group members, and they are very strict, we have seen that teacher should not be the only one grading students
Hi Brian,
I had thought trackback was auto on your blog for some reason.. but I blogged my response to this… now onto your OER post 🙂
Hi Brian and readers,
This is what I/we do at MITUPV Exchange (which is a social network established in 2000, it might look a little bit old-fashioned but it was there long before Facebook): there are several assignments: we take into account their level of participation and their replies to other people’s stuff. But the most important one is a final video they have to upload by the end of each semester (groupwork). They’re given a deadline. So what’s the problem with this? that most of them (or many) waited till the very end so there was this typical frantic flurry of very last minute activity. We’re changing our point of view: we’re telling them that we grade not only the final product (the video) but the amount of social conversation it triggers. This way they know that if they upload their stuff at the very last minute, chances are they won’t get that many comments from the other side (MIT or UPV) than those who upload their assignments long before the deadline. We want to compare results, so we can back it up with figures, but we can see there’s a change in their attitude.
I’ve been playing around with the idea of “social assessment” lately, a phrase that’s my own little cognitive mashup of oral examinations, online publication, peer review, and shared inquiry. I do think the teacher has a vital role to play in evaluating the work, but that role has to be in the context of the class’s 1) knowledge of itself as a group exploring the possibilities of communal mental activity (Bruner’s phrase–he still rocks my world) and b) use of itself as a collegial group (I got this from a presentation at the NITLE summit last year that insisted our students need to learn to be good colleagues for each other).
I tried something along these lines in my Film, Text, and Culture class in summer 06 and spring 07 when I had students post their final papers online *and* for the second half of these papers cite and link to specific blog posts their peers had written earlier in the term. I’ve spoken about this assignment several times but I should probably blog about it at length–aligning all the parts is a bit tricky to understand (and of course tricky to do as well). I’m certainly not satisfied with all the bits and pieces. For one thing, I didn’t do any marginal commentary on the essays; instead, I offered a paragraph-long narrative evaluation, which didn’t feel like quite enough to me. On the other hand, the folks who did well with the assignment did *extremely* well, and there were more of them than usual. Or so it seemed to me.
I did a presentation on “Proof That Matters” for the Prove It! area in the recent K12Online Conference. I hope to post those materials on my own blog soon, but for now, you can find them by searching k12online 2008. Or I could be less lazy and just put the link in! http://k12onlineconference.org/?p=301
I’ll be doing a presentation with Chuck Dziuban in February that will examine more of these questions. Taleb’s “The Black Swan” has been pretty crucial for both of us lately in thinking about this stuff.
ACK! Sorry to neglect these responses for so long. And I really don’t have the time or mental bandwidth to do them justice now. I wrote a simple post excerpting some of your excellent ideas and just posted it.
I recently came accross your blog and have been reading along. I thought I would leave my first comment. I dont know what to say except that I have enjoyed reading. Nice blog. I will keep visiting this blog very often.
Kate
http://educationonline-101.com
Pingback: It seems I’m always too slow when it comes to grading