Author Archives: chendric

Introduction: etmooc

I am taking part in a MOOC (massive, open, online course) focused on educational technology called etmooc. As part of that, I will have certain posts in the “etmooc” category that are designed for that mooc, specifically. I hope they will be interesting to my other readers as well!

This is just to introduce myself to the rest of the mooc: I’m Christina Hendricks, and I’m a Sr. Instructor in Philosophy at the University of British Columbia. A “Sr. Instructor” is a tenured, teaching-focused position (my merit and promotions are based on teaching and service, rather than research; though research on teaching counts!). I also teach in a first-year, interdisciplinary, team-taught course called Arts One. This is one of the three options for students in the Faculty of Arts to fulfill their first-year requirements. It is a year-long course, and students get 18 credits: 6 each in introductory English, History and Philosophy. I was Chair of that program from 2010-2012.

I am currently on sabbatical in Melbourne, Australia (July 2012-July 2013), and am doing research in the area of SoTL (the Scholarship of Teaching and Learning). Specifically, I’m interested in trying to analyze the value of the Arts One program for students. For more on my research this year, just look over my posts on this blog from the last 4-5 months! I’m particularly interested in peer assessment and feedback at the moment, but also in providing good feedback to students on writing, and how to effectively teach writing generally.

Looking forward to moving forward with the mooc, though I’m late in starting!

Problems with grading rubrics for complex assignments

In an earlier post I discussed a paper by D. Royce Sadler on how peer marking could be a means for students to learn how to become better assessors themselves, of their own and others’ work. This could not only allow them to become more self-regulated learners, but also fulfill roles outside of the university in which they will need to evaluate the work of others. In that essay Sadler argues against giving students preset marking criteria to use to evaluate their own work or that of other students (when that work is complex, such as an essay), because:

“Quality” is more of a global concept that can’t easily be captured by a set of criteria, as it often includes things that can’t be easily articulated.
As Sadler pointed out in a comment to the post noted above, having a set of criteria in advance predisposes students to look for only those things, and yet in any particular complex work there may be other things that are relevant for judging quality.
Giving students criteria in advance doesn’t prepare them for life beyond their university courses, where they won’t often have such criteria.

I was skeptical about asking students to evaluate each others’ work without any criteria to go on, so I decided to read another one of his articles in which this point is argued for more extensively.

Here I’ll give a summary of Sadler’s book chapter entitled “Transforming Holistic Assessment and Grading into a Vehicle for Complex Learning” (in Assessment, Learning and Judgement in Higher Education, Ed. G. Joughin. Dordrecht: Springer, 2009). DOI: 10.1007/978-1-4020-8905-3_4).

[Update April 22, 2013] Since the above is behind a paywall, I am attaching here a short article by Sadler that discusses similar points, and that I’ve gotten permission to post (by both Sadler and the publisher): Are we short-changing our students? The use of present criteria in assessment. TLA Interchange 3 (Spring 2009): 1-8. This was a publication from what is now the Institute for Academic Development at the University of Edinburgh, but these newsletters are no longer online.

Note: this is a long post! That’s because it’s a complicated article, and I want to ensure that I’ve got all the arguments down before commenting.

Continue reading →

On not living up to my open access values

Two things happened in the last couple of weeks that brought home to me how much more work I need to do to really be an open access advocate.

1. Hypatia announced their Book Reviews Online site: Hypatia Reviews Online

I just got an email in the last couple of days noting that the feminist philosophy journal Hypatia has set up a separate site for open access book reviews. The site states that from Fall 2012 all book reviews will be published online, open access (I don’t know if book reviews will also be available in print in the future or not). Previously, the Hypatia website had a book review archive that went from 2000-2004, and HRO extends that from 2004-the present (there are reviews from 2013 there already).

What a great idea! I thought, when I saw the new site. Of course! Why not have all your book reviews be open access, so everyone can get a good sense of various books in a field before they consider whether or not to buy them. It could even, potentially, help book sales (though that is pure speculation on my part).

I immediately sent an email offering to review books for Hypatia.

2. I received a copyright form for a book review I recently finished writing, as well as information on how to make that review open access.

The first thing I got was an email giving me a link to choose to make my book review open access. I knew it would cost money, of course, but I was shocked at the amount: $3250. Really? For a 1500-word book review?

I think if someone could break down those random-seeming numbers that appear in open access prices for articles to show where that money is going and why they need to charge that much for a single article, I might be less inclined to simply write off such amounts as absurd. All you get as an author is a number, and a choice to take it or leave it. No explanation is provided for where that number comes from.

Obviously I’m not going to pay that kind of money out of my own pocket, and I don’t have any grant money as I’m in a teaching rather than a research position. And even if I did, I wouldn’t pay that much for a book review, but would save it for a more original and substantive contribution to the scholarly literature that more people might want to read (hopefully).

So no open access for me on that one.

How about the copyright and author’s rights for this review? Fairly restrictive, it turns out. I can, at least, post a “preprint” of the review on my own website, as well as a copy of the version accepted for publication (after editing), so long as I don’t use the exact format they use in publishing (i.e., no final page numbers, no layout as it looks in the journal, etc.). But I can post the postprint on my website only after 18 months from publication.

And further, at no time can I post the pre- or postprint of this book review on a “systematic” basis, such as on a third-party database (wording: “not for commercial sale or for any systematic external distribution by a third party (for example a listserv or database connected to a public access server)”).

I’m assuming that means I can’t post on a site like Philpapers.org or Academia.edu, where I am in the process of posting many of my other works (subject, of course, to the copyright policies governing each one).

That’s the one that really gets me–it looks like I can’t make this widely accessible, just post it on my personal website (or my institution’s website or the institution’s “intranet”). Perhaps I’m reading that wrong, but sites like Philpapers and Academia.edu do distribute works on a systematic basis, don’t they? Do they count as databases “connected to a public access server”?

I can, at least, give out final, published copies to colleagues or students, but not in any “systematic” way.

My lesson

I didn’t pay any attention at all to the open access policies of the journal when I agreed to do this book review. That’s just downright embarrassing when I think of how much I’ve tweeted and blogged and talked about the value of open access. Okay, so it’s a small thing–a short book review–but the principle should be that if the policies for distributing the article are too restrictive, I should say no.

And in the meantime, I hopefully I’ll review a book or two for Hypatia.

Your thoughts

Might I have something wrong in reading the rules for preprint and postprint publication, given the quote above? Does it makes sense to say “no” to a journal’s request to review a book based on the publisher’s restrictive policies, even if the journal is one you like and the publisher’s policies are not really under their control?

Any other thoughts?

The Power of Space in the Classroom

Most of us know very well the importance of space in the classroom–how the room is set up can really change the dynamics of a class. For example, in a discussion course, I try to set up the room in as much of a circle as possible (which, given the configuration of some rooms, is sometimes impossible). Once I had a seminar-style class in a room where we simply could not put the tables and chairs into a circle, and had to leave them in rows (because there wasn’t enough room to put them in a circle). That was the worst term I’ve ever had for discussion.

A colleague of mine in the Arts One Program was even more innovative in her use of space than I’ve ever thought of being myself.

I have had the chance to view the classes of some of my colleagues in Arts One over the past few years. I wish I had more such chances to see others teach, since I always learn from what others are doing in their classes.

Arts One has two, 75-80 minute seminar-style discussion classes per week, with a maximum of 20 students, so most of the rooms we have allow for circular (actually rectangular) seating. There are tables arranged in a circle, with a big space in the middle of them. That works pretty well, since everyone can see everyone else.

Still, the professor usually sits at one of the “heads” of the table, on one of the shorter ends (we don’t have to do this, of course, but I’ve often seen it done). Subtly, then, we are still making ourselves the focal point by making sure most students can see us well (often students avoid sitting right next to the prof, and sit on the longer sides of the table instead).

This sort of setup is good for having books, paper and computers (if they’re allowed) out on the desk while engaging in discussion, but the tables with the big space in the middle cuts us off from one another in a sense, providing a pretty big distance from one another.

Continue reading →

Using SoTL to create guidelines for teaching

On the “Resources” page of the International Society for the Scholarship of Teaching and Learning (ISSOTL), I came across a link to “Guidelines on Learning that Inform Teaching,” a project by Adrian Lee, Emeritus Professor at the University of New South Wales, in Sydney, Australia. Through his work in his position as Pro Vice Chancellor (Education & Quality Improvement) at the University of NSW, Lee decided to work with others to create a set of guidelines, based on research, on what “works” in teaching.

The main ideas behind the project are stated as follows:

As academics, our task is to help students learn.

There is a vast research literature on how students learn and examples of good teaching based on this research.

As we claim to be research intensive institutions should not our teaching be based on this research?

But, as Lee notes, most faculty are very busy, and don’t have time to look into the relevant research on teaching, on top of their own research and teaching activities (I am lucky to be able to spend my sabbatical time doing lots of research into teaching, given that I’m in a teaching position, but most people don’t have that luxury). So he decided to distill the research into a set of guidelines, with links to relevant research papers.

Lee’s paper, “From Teaching to Learning,” linked at the bottom of the front page of the website, provides information (towards the end) on how the guidelines were developed. It also provides a great summary of some other excellent work done at UNSW on professional development for faculty re: teaching and learning–some things to learn from there, for institutions!

There are 16 guidelines, each with its own dedicated page. On those pages are quotes from various research papers that are meant to illustrate the guideline. I found these somewhat helpful, but would liked to have seen more narrative description of each guideline and what kind of research supports it. Perhaps one or two disciplinary examples would have helped illustrate the guideline a bit better, though those are all collected in a different section (see below). There are also links to research papers related to that particular guideline, for those who are interested in seeing some of the sources of the guideline. Of course, this effort is hampered by the fact that so many SoTL articles are not open access, so they can’t be easily linked to. Or rather, he could have put links to them, with notes that they are subscription only–so those whose institutions do have subscriptions could read them. Still, interested faculty could look up some of the non-linked papers themselves if their institutions have subscriptions.

There is also a section that has “Discipline-specific exemplars,” which has links to ways people are putting the guidelines into action (whether they are consciously aware of these specific guidelines or not!) in different disciplines. The “humanities” section is fairly sparse at this point, but Lee asks for exemplars and provides his email address.

Finally, each guideline has a “toolkit” designed to encourage reflection on each one by individual teachers (linked at the bottom of each guideline page). It asks instructors to think of examples, to engage in reflection on the guideline, to consider obstacles to implementing it, and more. I find the toolkit a bit sparse, and think that more instruction could be helpful–more specific questions to guide reflection, for example. And perhaps a prompt to ask instructors to come up with a way they could use it in their own teaching in the future.

Lee’s larger project is to encourage institutions to set up their own guidelines, for a similar purpose–to help faculty think about the current research in a distilled fashion, and see examples of work being done at their own institution and elsewhere that conforms to the guidelines. On the homepage he links to three universities that have used the 16 guidelines as a model for their own set of guidelines. For example, MIT has a page listing exemplars of the guidelines from MIT itself.

I think this is a very important concept–for those who don’t have the time to go to workshops on teaching where they might get similar information (or those for whom such workshops are not available), it’s helpful to have research-based information on quality teaching in higher ed available. Of course, a project like this requires upkeep, and some of the links on the site are currently broken (including the link to one of the three universities with their own, similar guidelines, University of Bedfordshire-UK). In addition, the latest papers linked to on the site are from the mid-2000s, and it would be nice to have later papers available as well. Still, what Lee is really hoping will happen is that universities create their own guidelines and sites, and keep these up to date; and that’s an excellent idea.

Do you know of any colleges or universities that have similar sorts of guidelines, backed by research? If so, please give links in the comments and I’ll send them to Adrian Lee! Or, if you have some discipline-specific exemplars for one or more of the guidelines, please post those below.
P.S. I’m off and on out of town for the next month, as it is summer here in Australia (where I am for sabbatical), so I may be a little slow in responding to comments!

MOOCs in Humanities: too massive?

I recently tweeted about an article I heard about from The Guardian (newspaper) higher education twitter feed: @GdnHigherEd: “Could online courses be the death of the humanities?” by Aurélien Mondon and Gerhard Hoffstaedter, co-founders of Melbourne Free University. I want to discuss that article briefly, and then give some thoughts on the benefits and drawbacks of scale in MOOCs (Massive Open Online Courses).

In the article noted above, Mondon and Hoffstaedter are commenting on a previous article, “Do online courses spell the end for the traditional university?,” by Carole Cadwalladr, in The Observer. There, Cadwalladr discusses how open education, and free online courses, could have an impact on traditional university education. Why pay thousands of dollars when you can get the same content taught by the same professors for free? Of course, you don’t get degrees or credit (yet), but for those who just want to learn something, MOOCs are likely a better option than signing up for a face-to-face class that you have to pay for.

In their response to this article, Mondon and Hoffstaedter suggest that the expansion of MOOCs could spell the death of humanities, specifically.

Continue reading →

How does giving comments in peer assessment affect students? (Part 3)

This is the third post in a series summarizing empirical studies that attempt to answer the question posed in the title. The first two can be found here and here. This will be the last post in the series, I think, unless I find some other pertinent studies.

Lundstrom, K. and Baker, W. (2009) To give is better than to receive: The benefits of peer review to the reviewer’s own writing, Journal of Second Language Writing 18, 30-43. Doi: 10.1016/j.jslw.2008.06.002

This article is focused on students in “L2” classes (second language, or additional language), and asks whether students who review peers’ papers do better in their own (additional language) writing than students who only receive peer reviews and attempt to incorporate the feedback rather than giving comments on peers’ papers.

Participants were 91 students enrolled in nine sections of additional language writing classes at the English Language Center at Brigham Young University. The courses were at two levels out of a possible five: half the students were in level 2, “high beginning,” and half were in level 4, “high intermediate” (33). The students were then divided into a control and experimental group:

The first group was composed of two high beginning and three high intermediate classes, (totaling forty-six students). This group was the control group (hereafter ‘‘receivers’’) and received peer feedback but did not review peers’ papers (defined as compositions written by students at their same proficiency level). The second group was composed of two high beginning classes and two high intermediate classes, (totaling forty-five students), and made up the experimental group, who reviewed peer papers but did not receive peer feedback (hereafter ‘‘givers’’). (33; emphasis mine)

Research questions and procedure to address them

Research questions:

1. Do students who review peer papers improve their writing ability more than those who revise peer papers (for both beginning and intermediate students)?

2. If students who review peer papers do improve their writing ability more than those who revise them, on which writing aspects (both global and local) do they improve? (32)

Continue reading →

How does giving comments in peer assessment impact students? (Part 2)

This is the second post looking at published papers that use empirical data to answer the question in the title. The first can be found here. As noted in that post, I’m using “peer assessment” in a broad way, referring not just to activities where students give grades or marks to each other, but more on the qualitative feedback they provide to each other (as that is the sort of peer assessment I usually use in my courses).

Here I’ll look at just one article on how giving peer feedback affects students, as this post ended up being long. I’ll look at one last article in the next post (as I’ve only found four articles on this topic so far).

Lu, J. and Law, N. (2012) Online peer assessment: effects of cognitive and affective feedback, Instructional Science 40, 257-275. DOI 10.1007/s11251-011-9177-2. This article has been made open access, and can be viewed or downloaded at: http://link.springer.com/article/10.1007%2Fs11251-011-9177-2

In this study, 181 13-14 year old students in a Liberal Studies course in Hong Kong participated in online peer review of various parts of their final projects for the course. They were asked to both engage in peer grading and give peer feedback to each other in groups of four or five. The final project required various subtasks, and peer grading/feedback was not compulsory — students could choose which subtasks to give grades and feedback to their peers about. The grades were given using rubrics created by the teacher for each subtask, and both grades and feedback were given through an online program specially developed for the course.

Research Questions

Are peer grading activities related to the quality of the final project for both assessors and assessees?

Are different types of peer … feedback related to the quality of the final projects for both assessors and assessees? (261)

Continue reading →

Learnist boards on SoTL and Open Access

During the past week or so I’ve been working on finishing a couple of things before their deadlines, and during breaks in working on those I’ve been having fun with Learnist: http://learni.st/. Learnist is sort of like Pinterest but focused more on learning about new things. The idea is that someone who has some knowledge about something designs a “learnboard” about it, and collects things from the web (including PDFs, Google docs and books, Prezi presentations, Kickstarter campaigns, and more), plus their own materials if they like, and puts them together in an order that helps others learn the topic. That person also adds commentary to help explain the artifact posted. There are places to comment on each artifact or on the whole board, and others can suggest new things to be put on the board.

Right now Learnist is in beta, and I am a bit frustrated by the fact that there is nothing on the front page to explain it (though there are learnboards on Learnist itself that give you an overview–you just have to type in “learnist” on the search bar at the top to find them).

You have to send a request to them if you want an account to create your own boards or make comments, but you don’t have to have an account to see all the existing boards. You can see mine by following the links below. They are still in progress, so more will likely be added later (esp. to the first and third ones).

1. What is the Scholarship of Teaching and Learning and How Can I Get Involved?

2. Open Access Journals in the Scholarship of Teaching and Learning

3. Open Access Scholarly Publishing (and Why It’s Important)

And you could also check out my personal page on Learnist, which lists any new learnboards I may have added since writing this post, as well as the boards I myself “like” and “follow.”

Update Dec. 6, 2012

I received an email from someone over at Grockit, the company that runs Learnist, asking for some suggestions to try to address the problems I noted above with being confused about how it all works when you arrive at the page for the first time. Nice to hear they are concerned and open to suggestions!

They also let me know I could invite others to use Learnist and they could bypass the usual waiting period through that means. So if you want an invite, send me an email (I may have to ask you a couple of questions to make sure you’re a real person and really interested in Learnist, since I’ll personally be inviting you!). Find my email address on my personal website at: https://blogs.ubc.ca/christinahendricks

How does giving comments in peer assessment impact students? (Part 1)

Some colleagues and I are brainstorming various research we might undertake regarding peer assessment, and in our discussions the question in the title of this post came up. I am personally interested in the comments students can give to each other in peer assessment, more than in students giving marks/grades to each other. Students engaging in giving comments on each others’ work are not only impacted by receiving peer comments, of course, but through the process of giving them as well. How does practice in giving comments and evaluating others’ work affect students’ own work or the processes they use to produce it?

I’ve already looked at a couple of articles that address this question from a somewhat theoretical (rather than empirical) angle (see earlier posts here and here). As discussed in those posts, it makes sense to think that practice in evaluating the work of peers could help students get a better sense of what counts as “high quality,” and thus have that understanding available to use in self-monitoring so as to become more self-regulated.

In this post I summarize the findings of two empirical articles looking at the question of whether and how providing feedback to others affects the quality of students’ own work. I will continue this summary in another post, where I look at another few articles.

(1) Li, L., Liu, X. and Steckelberg, A.L. (2010) Assessor or assessee: How student learning improves by giving and receiving peer feedback, British Journal of Educational Technology 41:3, 525-536. DOI: 10.1111/j.1467-8535.2009.00968.x

In this study, 43 undergraduate teacher-education students engaged in online peer assessment of each others’ WebQuest projects. Each student evaluated the projects of two other students. They used a rubric, and I believe they gave both comments and marks to each other. Students then revised their projects, having been asked to take the peer assessment into account and decide what to use from it. The post-peer assessment projects were marked by the course instructor.

Continue reading →