Author Archives: chendric

Your Turn (#h817open, Activity 3)

I’m trying to participate in another MOOC (massive, open, online course), though being away on holiday for a couple of weeks is making it difficult. It’s the “Open Education” course from the Oen University.

This is the sort of MOOC where it makes sense to say I’m “behind,” because there is a clear weekly schedule with weekly assignments to complete. I’m about a week behind. This structure is not appealing to me at the moment, after ETMOOC, which was much more open to participants doing whatever they felt most meaningful in response to the presentations, Twitter chats, shared links, and more. But I’m giving it a go because I really want to learn more about open education. I’ll probably do a few of the assigned activities, and then for the rest just write blog posts about what strikes me as most interesting.

Here’s one of the activities for the first week. We were asked to read Martin Weller’s article entitled “The Openness-Creativity Cycle in Education” and Terry Anderson’s slides from a Keynote for Alt-C in 2009. (Alt-C is the conference for the Association for Learning Technology). The activity was to then create a visual representation of open education, based on what we had read. “The key is to provide a representation that draws together the key concepts of openness as you perceive them,” according to the activity instructions.

I didn’t actually do that. Instead, I focused on one aspect of open education, or rather, one aspect of “openness”: that it’s more than just being available for reading/viewing for free, but is also available for remixing, reworking, taking pieces and doing something entirely different with them.

Part of the reason I did this was because I wanted to take on the challenge of creating a visual representation that didn’t just replicate a textual one. And if I tried to pull together all the strands of open education from the two readings (and elsewhere), I’d just end up putting into visual form the same things I would have otherwise written in text. That’s pretty much all I do in my work: write and speak narrative text. And I wanted to try to get beyond that a bit by using a picture that speaks. As you’ll see with the image, though, I couldn’t get away from text entirely.

This is not my thing, usually, so I won’t be surprised if it doesn’t entirely work, and if what seems good today seems simplistic tomorrow.

Since I seem incapable of doing without textual explanations, I will just say this. Multiple pieces made by others can appear as useless scatter, or as raw materials.

P.S. The other reason I did this is because one of my Twitter peeps created Ruschagram and this was a great excuse for me to play around with it.

 

 

Goodbye, #etmooc

ETMOOC is finishing up next week, and I’m about to leave town and be very sleep deprived for the next 3-4 days or so, so this is kind of my last hurrah for ETMOOC. I was trying to think about how/why my experience in it has been so important, so much so that I’m very sad it’s nearly over.

Instead of writing my response to that, I decided to try my first vlog–as a final project for ETMOOC, something else I’ve never done before (though I did do a “true story of openness” for Alan Levine).

Not surprisingly, it’s probably too long–just like my blog posts, my articles, and my plans for class meetings.

And I had only a couple of hours today to get it done, so it’s not terribly polished. I have a lot to learn about video editing, such as how to not only edit out portions of the video but also the audio that went with them (I tried to get rid of some “ummm’s” and “so’s,” and the video got deleted but not the audio. Hmph.). And that handmade heart at the beginning and end? Yeah, that should have been a nice title for the video…didn’t have time to do anything but cut one out and write on it.

So this is my love letter to ETMOOC. So sad to see you go.

[P.S. Is it possible to change the “still” shots from videos (such as the one below) so they aren’t those horribly strange facial expressions that come up when you’re in the middle of saying something?]

Why do I care if I’m attributed?

During one of the Twitter chats for the ETMOOC topic on “The Open Movement – Open Access, OERs & Future of Ed,” Pat Lockley Tweeted this:

 

We were talking about sharing our educational or other work, why some people find this difficult, the difference between “open access” and things being open in a wider sense, and more.

During the chat Pat’s Tweet kind of just went past me, but as I went back to the #etmchat Tweets for that day to add some to my Storify board on my ETMOOC experience, I came across it again and became curious as to what he meant. Thus started a fairly long conversation about copyright, licenses, public domain, and more. You can see it all here.

There’s a lot I’d like to think about further in this conversation, but what is really standing out for me at the moment is this:

 

 

Why am I using a CC-BY license on my work? Why do I care if I’m attributed when someone uses something from my blog, or some “open educational resource” I create? Pat brought up an important point:

 

 

Why not make one’s work public domain instead of using something like CC-BY? In the current legal climate, apparently it’s rather complicated: some places, like Canada and the U.S. (and probably other places too–I haven’t done enough research to list them), grant copyright simply through creating a work, and this may not actually be easy (or possible?) to give up (see, e.g., re: the U.S., Wikipedia on granting work into the public domain, and this post from the Public Domain Sherpa, and the last section of this page from Copyfree). One can, though, try to state as clearly as possible that one gives up all copyright and related rights to whatever extent allowed by law, and if not allowed, to give a license to anyone to use the work however they wish, without requirement of attribution. That’s what Creative Commons CC0 is meant to do. Copyfree has a list of various licenses that conform to their standard of “free use,” “free distribution,” free modification and derivation,” “free combination” and “universal application,” and CC0 is one of them (as is the Nietzsche public license, which is rather a personal favourite).

So, getting back to the original question and modifying it a bit: why not just use CC0 or something similar, thus releasing one’s work for any use by anyone, without attribution? Why care about attribution?

As Pat Lockley noted, it would be good to know that others find my work useful and that they reuse, repurpose and/or rework it. This would be helpful, if for no other reason than to validate for yourself what you’re doing. It could help you do more of it, perhaps. Knowing this would probably also be a way to improve one’s work through finding out what others have done with it. Not to mention it could be a way to potentially connect with others, which might even lead to collaborations.

In my own situation, on a pragmatic level, if I could discover and document how others have used my work, this could provide evidence that what I am doing has influence in the wider educational community, which might be one of several ways to support a claim of “educational leadership” or “distinction in the field of teaching and learning” for the new Professor of Teaching rank at UBC.

So yes, there are plenty of good reasons to be able to know what others are doing with your work.

But all of this requires what is NOT happening with CC-BY (and possibly not with other licenses…I haven’t done enough research to specify): notifying the attributed person that their work is being reused. If another blog links to your blog, you may get a pingback (maybe not; depends on the settings of your blog and the other blog, I think). And it’s a good practice to let other people know when you’ve used their work, if there’s an easy way to do it (such as leaving a comment on a photo posted on Flickr). I try to do that, but too often I forget (I’m working on this).

As noted towards the end of the Storified conversation with Pat, what’s missing, in order to get the benefits noted above, is some systematic way to notify people as to how you’ve used their work. I don’t even know how such a thing could work–the technological hurdles seem huge–but theoretically, it seems a good idea. Now, like any such things, one wouldn’t have to choose such a license (an attribution + notification license?), but for some it would provide a useful way to not just be attributed, but to know what uses their work is being put to. Perhaps it is too difficult/too much of a hassle to bother with. But it’s an intriguing idea.

“Attribution,” by fotogail (see below)

Of course, there are good arguments for making work as free as possible, without restrictions on what you have to do once you’ve accessed it–like attributing the author/creator, or telling him/her what you’re doing with it. So I’m undecided whether I, personally, would want to require more of the people using my work than just attribution. I might not even recommend this to others. But some might want to do it, and it could be useful.

But until and unless something like this happens, I’m back to my original question: Why do I care about attribution? If, for the most part, I won’t get the above benefits, what am I getting out of knowing that perhaps, somewhere out there, is a piece of work with my name attached?

One might think that it’s kind of like citation in academia; except again, citations are tracked whereas use of my CC-BY work (unless it’s a publication) is not. So really, it’s just a sense that other people know I created something. Why should I care about this?

Add to this the point that much of my work is not, perhaps, really “mine” in a deep sense because it is a culmination of so many other influences, work by so many other people that I have read or otherwise interacted with, and the question becomes even more pressing.

Okay, maybe it will come back to me at some point; maybe I’ll discover my work being used somewhere with my name, and then I can realize some of the good things noted previously. But maybe not (and perhaps most likely not). Or perhaps someone will find something with my name on it and decide to connect with me–thus leading to a connection through effort on someone else’s part rather than mine. These things might happen, but is that enough to require attribution for my work? I’m not yet sure.

I don’t have an answer, and you can’t answer for me of course, but maybe you have some ideas on why asking others to attribute one’s work might be a good idea, rather than just letting it go free into the wild. I’m thinking not so much for people who have to rely on their work to make a living, to make money off of it, but for people like me who are getting a salary from a university and could just share their blog writings, their photos, their OERs for free and without restrictions.

Help me out here?

Image credit: “Attribution,”  flickr photo (CC-BY) shared by fotogail

Introduction for #h817open

Even though ETMOOC still has another 1.5 weeks to go, and though I’m about to leave town for two weeks, I still decided to enrol in another open course: Open Education from the Open University. I will not be able to participate to the level I have in ETMOOC, given that I have so many things to do before my sabbatical ends and I leave Australia at the end of June. But I think it will be useful, given that it’s focused on various aspects of open education, including discussing just what “open” means, talking about OERs, MOOCs, and more. So some of my blog posts in the next seven weeks will be focused on activities related to that course, but I expect they’ll be of interest to at least some current readers as well.

For fellow participants in the open education course (h817open): my name is Christina Hendricks, and I’m a Sr. Instructor in Philosophy at the University of British Columbia, which is a tenured position focused on teaching (rather than research). My About.me page has a good deal of info, as does my Google+ profile. This is me on Twitter.

I’m currently on sabbatical in Melbourne, Australia, trying to jump start a new research focus in the Scholarship of Teaching and Learning (SoTL). I’ve done quite a bit of reading on learning communities (as I teach in one at UBC, and Chaired it for two years: Arts One) as well as peer assessment–if you’re interested, you could click on the “Scholarship of Teaching and Learning” category on the right, as I think I’ve put all (or at least most) of my posts related to these issues into that category. I’ve also got a set of posts called “Research Reviews,” in which I summarize the main points of some of the SoTL articles I’m reading and respond to them–those can be found at the menu on the top.

Even though I really don’t have time to participate in h817open as I’d like, I decided to enrol because through my experience in another MOOC I’m in the process of finishing, ETMOOC (Educational Technology and Media MOOC), I’ve become very interested in open education and really want to learn more. I had an idea for opening up my own courses in Philosophy or Arts One: see my earlier posts here and here for explanations.

I may not post every week, but I’ll try to do so at least every two weeks. And I’m an avid Twitter user, so I’ll definitely be following and contributing to the #h817open hashtag. I look forward to connecting!

 

(etmooc) On openness and panopticism

“Panopticon,” cc licensed ( BY ) flickr photo shared by chad_k

A year or two ago a student came into my office and told me about some podcasts he had been listening to, which consisted of some lectures by a well-known philosopher as part of one of his university courses. The student then asked me why I didn’t put my lectures out on podcasts, or make them public in some other way.

I don’t remember what I said. But I do remember what I felt: apprehension. And some fear. I couldn’t imagine, at the time, doing such a thing.

Now I can, and largely through my experience in ETMOOC I’ve become very interested in the idea of “openness” in education and want to start doing some of this myself. Of course, “open” means different things in different contexts (here’s a nice post explaining some of them, and here’s an even larger list of various “opens”) , but I’m considering things such as posting and licensing many of my course materials for re-use, as well as possibly opening up a course to outside participants the way Bryan Jackson did with his high school Philosophy course.

The value of open education

There are plenty of good things about opening up your teaching and learning materials, space, interactions, etc. Bryan Jackson explains something good that happened as a result of having an open Philosophy course, in this video. Barbara Ganley had an interesting experience from a writing assignment in her class posted publicly on a blog (see “A Writing Assignment Gets Personal,” on this site).

David Wiley, in a presentation on open education called “Openness, Disaggregation, and the Future of Education” (the keynote for the 2009 Penn State Symposium for Teaching and Learning) gave several examples of things he had done recently in his courses to make them more open. Among them:

  • He required that all students’ written work must be made public on the course blog. One result of this was that Stephen Downes—a prominent Canadian researcher, blogger, cMOOC facilitator, and editor the popular newsletter OLDaily (online learning daily)–had read some of the work and highlighted a few posts, sending them out to thousands of his followers in the OLDaily newsletter. Wiley noted that the following week, much of the students’ writing got longer, better, and more thoughtful. Such improvement came much better this way than just encouraging students to write more carefully and address issues more deeply through the instructor’s comments.
  • He wrote up a script for a fake sitcom (situation comedy) tv show, to show differing viewpoints on opening up “learning objects” (what are now called open educational resources, I think). He put this up on a course wiki, and some of the graduate students in the course started writing in new characters in order to give even more perspectives. They hadn’t asked or said they were going to do it, but just did. This was, he stated in the talk, a great way to get students involved in creating learning materials for the course itself.

My experiences in ETMOOC are good evidence as well: I now have a much wider network of people to talk to about teaching and learning, and educational technology, because this course is open to anyone who wants to join and participate. I have more comments on my blog, many more twitter interactions, more people to help answer questions (I just ask the Twittersphere and answers come quickly), more links to helpful resources for my own thinking and teaching and learning, and more.

These are just a few examples of good things that can come from opening up education. I’m certain there are many more. 

In addition, ETMOOC-ers said some good things about the value of openness in a recent Twitter chat:

I can see many benefits to opening up my teaching and learning more than I’m already doing, and I expect there are more that I can’t even currently imagine.

So was I reticent before, when my student asked about podcasting my classes, only because I didn’t see these benefits then? I don’t think so.

Fear and Openness

There are many ways of making one’s courses more “open,” including just posting one’s course materials for others to see (e.g., written materials, digital presentations, video or audio of lectures); giving the materials a Creative Commons license that allows others to reuse, repurpose, and build on them; live streaming your class meetings publicly; all the way to opening out the course to any participants who want to join (see Alec Couros‘ Social Media & Open Education course as an example, as well as Bryan Jackson’s high school philosophy course noted above). The concerns I bring up below apply to all of these, but mostly to the last two.

The apprehension I felt at the idea of podcasting my lectures wasn’t just the usual fear of being in front of a camera or having one’s voice go out into the wider world; I was a college radio DJ in university and grad school, and am don’t mind speaking into the void with the knowledge that many people (or none) might be listening. Video is still a little tough for me, but I’m quickly getting over that.

It wasn’t just a lack of confidence, a sense that no one would want to listen to my lectures when they have access to those of people who are much more expert than me on the topics they’re discussing (though there was some of that too).

There was something about potentially being watched, being observed, at any time, by anyone; but mostly, by those who could have significant influence over my future. It’s not that I worry my teaching isn’t very good, or that I think bad things would happen if those who can affect my employment see most or all of what I do in class. I actually have (and have had) fantastic colleagues, and every time I’ve had a peer visit a class it has ended up being a very positive experience, complete with helpful advice–much of which I still vividly remember and use.

I think it was partly that in having my courses be “open” it’s as if I could be undergoing a peer review of teaching at any time, all the time. 

Which means, of course, (being a Foucault scholar) that I thought of Foucault.

Panopticism

Panopticon

Panopticon, Jeremy Bentham [Public domain], via Wikimedia Commons


In Discipline and Punish (1975), Foucault wrote a great deal about the “disciplinary society” being a “panoptic” one, referring to Jeremy Bentham’s idea for a panoptic design for a prison. Section 3.3 (“History of the Prison”) of the Stanford Encyclopedia of Philosophy’s entry on Foucault is a nice, concise explanation of Foucault’s discussion of panopticism and discipline. And here is a post that connects panopticism to social media, and starts to get to the concern I’m working towards here.

It’s not just a concern about possibly being observed at any given moment. Nor is it only that there could be a potential danger to this vis-à-vis power relations in one’s place of employment. It’s also that this situation of potentially being observed at any given moment can pressure one to change one’s own behaviour in order to bring it more in line with dominant norms. We police ourselves, rather than having to be policed. There doesn’t even have to be anyone watching for this to happen.

Now, this isn’t always necessarily bad. I agree with Alec Couros’s tweet, above, that knowing others might see my work would spur me to make it as good as possible. Plus, of course, if others saw it and commented, this could help me improve it even more.

But the potential downside is that one might be less likely to try radically new things, to experiment, to risk doing things that don’t fit with dominant views of how education is “done.” Clearly this isn’t true for everyone; there are people doing innovative things openly (e.g., many of the conspirators in ETMOOC)–though even then one usually has a community with its own norms that one is part of.

The issue would be prominent especially for those who don’t have tenured or otherwise semi-permanent positions–it’s often (though not always) in their best pragmatic interest to police themselves not to take too many risks if their work is open, though some risk-taking might be seen as positive.

So one reason some people might not be willing to be more open in their teaching and learning might be because of vulnerability. They could be vulnerable in the sense of not having a stable position, or in the sense of having a particular department or school climate that makes it such that opening their teaching could be dangerous to their position (because their colleagues may not agree with what they’re doing, e.g.).

I am fortunate in that neither of these situations applies to me, but that’s a bit of a luxury, and there are many people who don’t have it.

One more thing

I wonder if making my courses more open, in the sense of recording the sessions, would change how I conduct some of my class meetings. A fair number of them are unscripted, experimental forays into topics through (sometimes haphazard) discussion that may or may not come to a clear end point (usually not). I think of these as part of a work-in-progress, a long-term work in which I and the students are moving towards better understanding of certain issues, questions, arguments, texts. Or at least, different understanding that brings up fruitful, new ways of thinking about and approaching these things, showing further dimensions that were hidden before. This work-in-progress may last for a few weeks or months, a few years, or a lifetime. The courses, for me, are just a very small part of this process. In some ways I like that the class meetings are evanescent, short-lived; they aren’t final products in any sense and aren’t meant to be. I wouldn’t want anyone to watch one or two such meetings and get the sense that what I or anyone else says there represents anything more than a provisional test of a thought or argument. It will always change later.

Somehow, recording one’s course sessions seems to me to be making them more permanent, which goes against the way I think of the meetings. I want them to be memories only, things that change when you revisit them, just as the ideas do.

Of course, these issues exist with writing and publishing too–writing is never permanent, and one’s arguments can change radically over the course of a few years. But writing already seems more stable than a class discussion that takes place orally.

Conclusion?

I don’t have one. I just wanted to explore why I might have been reticent to be open, and why others might be. These thoughts on panopticism and sharing things publicly are anything but new, but they may be factors for some.

As with anything, there are benefits and drawbacks to being open in teaching and learning. I think the benefits, in my own personal situation, outweigh the risks of being open (as well as the concern about “permanency” noted above). But that may not be true for everyone, and it may for reasons other than a desire to keep one’s work to oneself, or out of a lack of confidence.

 

Summary of research on modes of peer assessment

I have been doing quite a few “research reviews” of articles on peer assessment–where I summarize the articles and offer comments about them. Lately I’ve been reading articles on different modes of peer assessment: written, oral, online, face to face, etc. And here, I am going to try to put together what that research has said to see if anything can really be concluded about these issues from it.

In what follows, I link to the blog posts discussing each article. Links to the articles themselves can be found at the bottom of this post.

I created PDF tables to compare/contrast the articles under each heading. They end up being pretty small here on the blog, so I also have links to each one of them, below.

Peer feedback via asynchronous, written methods or synchronous, oral, face to face methods

This is the dichotomy I am most interested in: is there a difference when feedback is given asynchronously, in a written form, or when given synchronously, as spoken word face to face? Does the feedback itself differ? Might one form of feedback be more effective than another in terms of being taken up in later revisions of essays?

Do the comments differ in the two modes of peer feedback, and are they used differently in later drafts?

The PDF version of the table below can be downloaded here.

van den Berg, Admiraal and Pilot (2006) looked at differences in what was said in peer feedback on writing assignments when it was written (on standardized peer feedback forms, used for the whole class) and when it was given in oral, face to face discussions. They found that written feedback tended to be more focused on evaluating the essays, saying what was good or bad about them, and less on giving explanations for those evaluative comments or on providing suggestions for revision (though this result differed between the courses they analyzed). In the oral discussions, there was more of a balance between evaluating content, explaining that evaluation, and offering revisions. They also found that both written and oral feedback focused more on content and style than on structure, though there were more comments on structure in the written feedback than in the oral. The authors note, though, that in the courses in which peer feedback took place on early drafts or outlines, there was more feedback on structure than when it took place on later drafts. They conclude: “A combination of written and oral feedback is more profitable than written or oral feedback only” (146).

Hewett (2000) looked at differences in peer feedback between an oral, face to face environment and an electronic, text-based environment. She found that the talk in the oral communication was much more interactive, with students responding to each others’ comments, giving verbal cues that they were following along, and also working together to generate new ideas. The text-based, online feedback was much less like a conversation, with students commenting on the papers at hand but not interacting very much with each other. Perhaps unsurprisingly, then, while the feedback in the written environment was mostly focused on the content of the essay being evaluated, the discussion in the oral environment ranged more widely. Hewett also analyzed essay drafts and peer comments from both environments to see if the peer discussion and comments influenced later drafts of essays. She found that in the oral environment, there was more use in students’ work of ideas that came up in the peer discussion about others’ essays, or that one had oneself said. Hewett concludes that a combination of oral discussion and asynchronous, written comments would be good, using the former for earlier stages of writing–since in oral discussion there can be more talk in which students speculate about wider issues and work together to come up with new ideas–and the latter for revisions focused more on content.

What are students’ views of each mode?

A PDF version of the following table can be downloaded here.

Figl et al. (2006) surveyed students in a computer science course who had engaged in peer assessment of a software project in both the face to face mode as well as through an online, asynchronous system that allows for recording of criticisms as well as adding comments as in a discussion board. There wasn’t a clear preference for one mode over another overall, except in one sense: about half of the students preferred using the face to face mode for discussion within their own teams, and with their partner teams (those they are giving feedback to and receiving feedback from). There was not as much discussion of the feedback, whether within the team or with the partner teams, in the online format, students reported, and they valued the opportunity for that discussion. Figl et al. conclude that it would be best to combine online, asynchronous text reviews with face to face activities, perhaps even with synchronous chat or voice options.

The study reported in Guardardo & Shi 2007 focused on asynchronous, written feedback for the most part; the authors recorded online, discussion-board feedback on essays and compared that with a later draft of each essay. They wanted to know if students used or ignored these peer comments, and what they thought of the experience of receiving the asynchronous, written feedback (they interviewed each student as well). All of the students had engaged in face to face peer feedback before the online mode, but the face to face sessions were not recorded so the nature of the comments in each mode was not compared. Thus, the results from this study that are most relevant to the present concern are those that come from interviews, in which the students compared their experiences of face to face peer feedback with the online, written, asynchronous exchange of feedback. Results were mixed, as noted in the table, but quite a few students said they felt more comfortable giving feedback without their names attached, while a significant number of students preferred the face-to-face mode because it made interacting with the reviewer/reviewee easier. The authors conclude that “online peer feedback is not a simple alternative to face-to-face feedback and needs to be organized carefully to maximize its positive effect” (458).

Cartney 2010 held a focus group of ten first-year students in a social work course who had engaged in a peer feedback exercise in which essays and comments on essays, as well as follow up discussion, was to take place over email. Relevant to the present concern is that the focus group discussion revealed that several groups did not exchange feedback forms via email but decided to meet up in person instead in order to have a more interactive discussion. Some groups did exchange written, asynchronous, online feedback, citing discomfort with giving feedback to others to their “faces.” The author concludes that there may be a need to use more e-learning in curricula in order for students to become more accustomed to using it for dialogue rather than one-way communication. But I also see this as an indication that some students recognized a value in face to face, oral, synchronous communication.

Peer feedback via electronic, synchronous text-based chat vs. oral, face to face methods

This dichotomy contrasts two sorts of synchronous methods for peer feedback and assessment: those taking place online, through text-based systems such as “chats,” and those taking place face to face, orally.

Do comments given synchronously through text-based chats differ from those given orally, face to face? And do these two modes of commenting affect students’ revisions of work differently?

A PDF version of both of the tables below can be downloaded here.

Sullivan & Pratt 1996  looked at two writing classes: in one class all discussions and peer feedback took place through a synchronous, electronic, text-based chat system and in the other discussions and peer feedback took place face to face, orally. They found that writing ability increased slightly more for the computer-assisted class over the traditional class, and that there were differences in how the students spoke to each other in the electronic, text-based chat vs. face to face, orally. The authors stated that the face to face discussion was less focused on the essay being reviewed than in the online chats (but see my criticisms of this interpretation here). They also found that the electronic chats were more egalitarian, in that the author did not dominate the conversation in them in the same way as happened with the face to face chats. The authors conclude (among other things) that discussions through online chats may be beneficial for peer assessments, since their study “showed that students in the computer-assisted class gave more suggestions for revision than students in the oral class” (500), and since there was at least some evidence for greater writing improvement in the “chat” class.

Braine 2001 (I haven’t done an earlier summary of this article in my blog) looked at students in two different types of writing classes in Hong Kong (in English), similar to those discussed in Sullivan & Pratt (1996), in which one class has all discussions and peer assessment taking place orally, and the other has these taking place on a “Local Area Network” that allows for synchronous, electronic, text-based chats. He looked at improvement in writing between a draft of an essay and a revision of that essay (final version) after peer assessment. Braine was testing students’ ability to write in English only, through the “Test of Written English.” He found that students’ English writing ability improved a bit more for the face-to-face class than the computer-mediated class, and that there were significant differences in the nature of discussions in the two modes. He concluded that oral, face-to-face discussions are more effective for peer assessment.

Liu & Sadler 2003  contrasted two modes of peer feedback in two composition classes, one of which wrote comments on essays by hand and engaged in peer feedback orally, face to face, and the other wrote comments on essays digitally, through MS Word, and then engaged in peer discussion through an electronic, synchronous, text-based chat during class time. The authors asked about differences in these  modes of commenting, and whether they had a differential impact on later essay revisions. Liu & Sadler were not focused on comparing the asynchronous commenting modes with the synchronous ones, but their results show that there was a higher percentage of “global” comments in both of the synchronous modes, and a higher percentage of “local” comments in the asynchronous ones. They also found that there was a significantly higher percentage of “revision-oriented” comments in the oral discussion than in the electronic chat. Finally, students acted more often on the revision-oriented comments given in the “traditional” mode (handwritten, asynchronous comments plus oral discussion) than in the computer-mediated mode (digital, asynchronous comments plus electronic, text-based chat). They conclude that for asynchronous modes of commenting, using digital tools is more effective than handwriting (for reasons not discussed here), and for synchronous modes of commenting, face to face discussions are more effective than text-based, electronic chats (219-221). They suggest combining these two methods for peer assessment.

Jones et al 2006  studied interactions between peer tutors in an English writing centre in Hong Kong and their clients, both in face to face meetings and in online, text-based chats. This is different from the other studies, which were looking more directly at peer assessment in courses, but the results here may be relevant to what we usually think of as peer assessment. The authors were looking at interactional dynamics between tutors and clients, and found that in the face-to-face mode, the relationship between tutors and clients tended to be more hierarchical than in the electronic, online chat mode. They also found that the subjects of discussion were different between the two modes: the face-to-face mode was used most often for “text-based” issues, such as grammar and word choice, while in the electronic chats the tutors and clients spoke more about wider issues such as content of essays and process of writing. They conclude that since the two modes differ and both serve important purposes, it would be best to use both modes.

Implications/discussion

This set of studies is not the result of a systematic review of the literature; I did not follow up on all the other studies that cited these, for example. A systematic review of the literature might add more studies to the mix. In addition, there are more variables that should be considered (e.g., whether the students in the study underwent peer assessment training, how much/what kind; whether peer assessment was done using a standardized sheet or not in each study, and more).

Nevertheless, I would like to consider briefly if these studies provide any clarity for direction regarding written peer assessment vs. oral, face-to-face.

For written, asynchronous modes of peer assessment (e.g., writing on essays themselves, writing on peer assessment forms) vs. oral, face-to-face modes, the studies noted here (van den Berg, Admiraal and Pilot (2006) and Hewett (2000)) suggest that in these two modes students give different sorts of comments, and for a fuller picture peer assessment should probably be conducted in both modes. Regarding student views of both modes (Figl et al. (2006), Guardardo & Shi (2007), Cartney (2010)), evidence is mixed, but there are at least a significant number of students who prefer face-to-face, oral discussions if they have to choose between those and asynchronous, written peer assessment.

For written, synchronous modes of peer assessment (e.g., electronic, text-based chats) vs. oral, face-to-face, the evidence here is all from students for whom English is a foreign language, but some of the results might still be applicable to other students (to determine this would require further discussion than I can engage in now). All that can be said here is that the results are mixed. Sullivan & Pratt (1996) found some, but not a lot of evidence that students using e-chats improved their writing more than those using oral peer assessment, but Braine (2001) found the opposite. However, they were using different measures of writing quality. Sullivan & Pratt also concluded that the face-to-face discussions were less focused and effective than the e-chat discussions, while Braine concluded the opposite. This probably comes down in part to interpretation of what “focused” and “effective” mean.

Liu & Sadler (2003) argued that face-to-face modes of synchronous discussion are better than text-based, electronic, synchronous chats–opposing Sullivan & Pratt–because there was a higher percentage of “revision-oriented” conversational turns (as a % of total turns) in the face-to-face mode, and because students acted on the revision-oriented comments more in the traditional class (both writing comments on paper and oral, face-to-face peer discussion) than in the computer-mediated class (digital comments in MS Word and e-chat discussions). Jones et al. (2006) found that students and peer tutors talked about different types of things, generally, in the two modes and thus concluded that both should be used. But that study was about peer tutors and clients, which is a different situation than peer assessment in courses.

So really, little can be concluded, I think, from looking at all these studies, except that it does seem that students tend to say different types of things in different modes of communication (written/asynchronous, written/synchronous, oral/face-to-face/synchronous), and that those things are all valuable; so perhaps what we can say is that using a combination of modes is probably best.

Gaps in the literature

Besides more studies to see if better patterns can emerge (and perhaps they are out there–as noted above, my literature search has not been systematic), one gap is that no one, so far, has considered video chats, such as Google Hangouts, for peer assessment. Perhaps the differences between those and face-to-face meetings might not be as great as between face-to-face meetings and text-based modes (whether synchronous chats or asynchronous, written comments). And this sort of evidence might be useful for courses that are distributed geographically, so students could have a kind of face-to-face peer assessment interaction rather than just giving each other written comments and carrying on a discussion over email or an online discussion board. Of course, the problem there would be that face-to-face interactions are best if supervised, even indirectly, so as to reduce the risk of people treating each other disrespectfully, or offering criticisms that are not constructive.

So, after all this work, I’ve found what I had guessed before starting: it’s probably best to use both written, asynchronous comments and oral, face-to-face comments for peer assessment.

 

Works Cited

Braine, G. (2001) A study of English as a foreign language (EFL) writers on a local-area network (LAN) and in traditional classes, Computers and Composition 18,  275–292. DOI: http://dx.doi.org/10.1016/S8755-4615(01)00056-1

Cartney, P. (2010) Exploring the use of peer assessment as a vehicle for closing the gap between feedback given and feedback used, Assessment & Evaluation in Higher Education, 35:5, 551-564. DOI: http://dx.doi.org/10.1080/02602931003632381

Figl, K., Bauer, C., Mangler, J., Motschnig, R. (2006) Online versus Face-to-Face Peer Team Reviews, Proceedings of Frontiers in Education Conference (FIE). San Diego: IEEE. See here for online version (behind a paywall).

Guardado, M., Shi, L. (2007) ESL students’ experiences of online peer feedback, Computers and Composition 24, 443–461. Doi: http://dx.doi.org/10.1016/j.compcom.2007.03.002

Hewett, B. (2000) Characteristics of Interactive Oral and Computer-Mediated Peer Group Talk and Its Influence on Revision, Computers and Composition 17, 265-288. DOI: http://dx.doi.org/10.1016/S8755-4615(00)00035-9

Jones, R.H., Garralda, A., Li, D.C.S. & Lock, G. (2006) Interactional dynamics in on-line and face-to-face peer-tutoring sessions for second language writers, Journal of Second Language Writing 15,  1–23. DOI: http://dx.doi.org/10.1016/j.jslw.2005.12.001

Liu, J. & Sadler, R.W. (2003) The effect and affect of peer review in electronic versus traditional modes on L2 writing, Journal of English for Academic Purposes 2, 193–227. DOI: http://dx.doi.org/10.1016/S1475-1585(03)00025-0

Sullivan, S. & Pratt, E. (1996) A comparative study of two ESL writing environments: A computer-assisted classroom and a traditional oral classroom, System 29, 491-501. DOI: http://dx.doi.org/10.1016/S0346-251X(96)00044-9

Van den Berg, I., Admiraal, W.,  & Pilot, A. (2006) Designing student peer assessment in higher education: analysis of written and oral peer feedback, Teaching in Higher Education, 11:2, 135-147.  DOI: http://dx.doi.org/10.1080/13562510500527685

Peer Assessment: Face to face vs. online, synchronous (Part 2)

Here I look at one last study I’ve found that focuses on the nature of student peer feedback discussions when they take place in a synchronous, online environment (a text-based chat). Part 1 corresponding to this post can be found here.

Jones, R.H., Garralda, A., Li, D.C.S. & Lock, G. (2006) Interactional dynamics in on-line and face-to-face peer-tutoring sessions for second language writers, Journal of Second Language Writing 15,  1–23. DOI: http://dx.doi.org/10.1016/j.jslw.2005.12.001

This study is rather different than the ones I looked at in Part 1 of face to face vs. online, synchronous peer assessment, because here the subjects of the study are students and peer tutors in a writing centre rather than peers in the same course. Still, at least some of their results regarding the nature of peer talk in the tutor situation may still be relevant for peer assessment in courses.

The Tutoring Centre, by Tulane Public Relations, on Flickr. CC-By licensed.

Participants and data

The participants in this study were five peer tutors in a writing centre in Hong Kong, dedicated to helping non-native English speakers write in English. For both tutors and clients, English was an additional language, but the tutors were further along in their English studies and had more proficiency in writing in English than the clients. Data was collected from transcripts of face to face consultations of the tutors with clients, as well as transcripts of online, text-based chat sessions of the same tutors, with many of the same clients.

Face to face tutoring was only available in the daytime on weekdays, so if students wanted help after hours, they could turn to the online chat. Face to face sessions lasted between 15 and 30 minutes, and students “usually” emailed a draft of their work to the tutor before the session. Chat sessions could be anywhere from a few minutes to an hour, and though tutors and clients could send files to each other through a file exchange system, this was only done “sometimes” (6). These details will become important later.

Model for analyzing speech

To analyze the interactions between tutors and clients, the authors used a model based on “Halliday’s functional-semantic view of dialogue (Eggins & Slade, 1997; Halliday, 1994)” (4). In this model, one analyzes conversational “moves,” which are different than “turns”–a “turn” can have more than one “move.” The authors explain a move as “a discourse unit that represents the realization of a speech function” (4).

In their model, the authors use a fundamental distinction given by Halliday into “initiating moves” and “responding moves”:

Initiating moves (statements, offers, questions, and commands) are those taken independently of an initiating move by the other party; responding moves (such as acts of acknowledgement, agreement, compliance, acceptance, and answering) are those taken in response to an initiating move by the other party. (4-5)

They then subdivide these two categories further, some of which is discussed briefly below.

Results

Conversational control

In the face to face meetings, the tutors exerted the most control over the discussions. Tutors had many more initiating moves (around 40% of their total moves, vs. around 10% of those for clients), whereas clients had more responding moves (around 33% of clients’ total moves, vs. about 14% for tutors). In the chat conversations, on the other hand, initiating and responding moves were about equal for both tutors and clients (7).

Looking more closely at the initiating moves made by both tutors and clients, the authors report:

In face-to-face meetings, tutors controlled conversations primarily by asking questions, making statements, and issuing directives. In this mode tutors asked four times more questions than clients. In the on-line mode, clients asked more questions than tutors, made significantly more statements than in the face-to-face mode, and issued just as many directives as tutors. (10)

Types of questions

However, the authors also point out that even though the clients asserted more conversational control in the online chats, it was “typical” of the chats to consist of questions by students asking whether phrases, words, or sentences were “correct” (11). They did not often ask for explanations, just a kind of check of their work from an expert and a quick answer as to whether something was right or wrong. On the other hand, when tutors controlled the conversations with their questions, it was often the case that they were using strategies to try to get clients to understand something themselves, to understand why something is right or wrong and to be able to apply that later. So “control” over the conversation, and who asks the most questions or issues the most directives, are not the only important considerations here.

The authors also divided the questions into three different types. Closed questions: “those eliciting yes/no responses or giving the answerer a finite number of choices; open questions: “those eliciting more extended replies”; rhetorical questions: “those which are not meant to elicit a response at all” (12)

In the face to face sessions, tutors used more closed questions (about 50% of their initiating questions) than open questions (about 33%); the opposite was true in the online chats: tutors used more open questions (about 50% of their initiating questions) than closed (about 41%).

Continue reading →

MOOCs and humanities, revisited

In the last post I discussed how I have come to learn about the different kinds of MOOCs through my participation in etmooc. I also said that through learning about a new kind of MOOC, the cMOOC or “network-based” MOOC, I was reconsidering my earlier concerns with MOOCs. Might the cMOOC do better for humanities than the xMOOC?

A humanities cMOOC

“Roman Ondák”, cc licensed ( BY ) flickr photo shared by Marc Wathieu

I haven’t yet decided whether or not one could do a full humanities course, such as a philosophy course, through a cMOOC structure. Brainstorming a little, though, I suppose that one could have a philosophy course in which:

  • Common readings are assigned
  • Presentations are given by course facilitators and/or guests, just as in etmooc
  • Participants are encouraged to blog about the readings and presentations and comment on each others’ blogs (through a course blog hub, like etmooc and ds106 have)
  • Dedicated Twitter hashtag, plus a group on a social network like Google+, and a group on a social bookmarking site like Diigo (see etmooc’s group site on Diigo)
  • Possibly a YouTube channel, for people to do vlogs instead of blogs if they want, or share other videos relevant to the course

Would this sort of structure be more likely to allow for teaching and practice of critical thinking, reading and writing skills, as I discussed in my earlier criticism of MOOCs (which was pretty much a criticism of xMOOCs)? I suppose it depends on what is discussed in the presentations, in part. The instructors/facilitators could model critical reading and thinking, through explaining how they are interpreting texts and pointing out potential criticisms with the arguments. They could talk about recognizing, criticizing, and creating arguments so that participants could be encouraged to present their own arguments in blogs as clearly and strongly as possible, as well as offering constructive criticisms of works being read–as well as each others’ arguments (though the latter has to be undertaken carefully, just as it is in a face to face course).

This would involve, effectively, peer feedback on participants’ written work. Rough guidelines for blog posts (at least some of them) could be given, so that in addition to reflective pieces (which are very important!) there could also be some blog posts that are focused on criticizing arguments in the texts, some on creating one’s own arguments about what’s being discussed, etc.

What you wouldn’t be able to do well with this structure are writing assignments in the form of argumentative essays. These take a long time to learn how to do well, and ideally should have more direct instructor/facilitator feedback rather than only peer feedback, in my view. Peer feedback is important too, but could lead to problems being perpetuated if the participants in a peer group share misconceptions.

Another thing you can’t do well with a cMOOC is require that everyone learn and be assessed on a particular set of facts, or content. A cMOOC is better for creating connections between people so that they can pursue their own interests, what they want to focus on. Each person’s path through a cMOOC can be very different. Thus, as noted in my previous post, there is not a common set of learning objectives; rather, participants decide what they want to get out of the course and focus on that.

One would need to have a certain critical mass of dedicated and engaged participants for this to work. If it’s a free and open course, then people will participate when they can, and can flit in and out of the topics as their time and interest allows. That’s fantastic, I think, though if there are few participants that might mean that for some sections of the course little is happening. So having a decent sized participant base is important. (How many? No idea.)

I envision this sort of possibility as a non-credit course for people who want to learn something about philosophy and discuss it with others. Why not give credit? There would have to be more focus on content and/or more formal assessments, I think (at least in the current climate of higher education).

A cMOOC as supplement to an on-campus course

Even if a full cMOOC course in philosophy or another humanities subject may not work, I can see a kind of cMOOC component to philosophy courses, or Arts One. In addition to the campus-based, in-person course, one could have an open course going alongside it. This is what ds106 is like. One could have readings and lectures posted online (or at least, links to buy the books if the readings aren’t readily available online), and then have a platform for students who are off campus to engage in a cMOOC kind of way.

Then, those off campus can participate in the course through their blog posts and discussions/resource sharing on the other platforms, like we do in etmooc. Discussion questions used in class could be posted for all online participants.  Students who are on campus could be blogging and tweeting and discussing with others outside the course as well as inside the course.

Frankenstein engraved

Frontispiece to Mary Shelley’s Frankenstein (1831),by Theodor von Holst [Public domain], via Wikimedia Commons. One of the texts on Arts One Digital.

Discussions would expand to include many more people with many more backgrounds and things to contribute, which is likely to enrich the learning experience. There might get to be too much for each individual to follow, but then one just has to learn to pick and choose what to read and comment on (more on this, below). All participants could make connections and continue discussions beyond the course itself.

Arts One has already started to move in this direction, with a new initiative called Arts One Digital. So far, there are some lectures posted, links to some online versions of texts, twitter feed, and blog posts. This is a work in progress, and we’re still figuring out where it should go. I think extending the Arts One course in the way described above might be a good idea.

Again, the main problem with this idea (beyond the fact that yes, it will require more personnel to design and run the off-campus version of the course) is getting a high number of participants. It won’t work well if there aren’t very many people involved–a critical mass is needed to allow people to find others they want to connect with in smaller groups, to engage in deeper discussions, to help build their own personal learning network.

Looking back at previous concerns with (x)MOOCs

Besides general worried about their ability to help students develop critical skills, I was also concerned in my earlier post with the following:

  • In the Coursera Course on reasoning and argumentation (“Think Again”) that I sat in on briefly, I found myself getting utterly overwhelmed by the number of things posted in the discussion board. I complained that I could scroll and scroll just to get through the comments on one post, to get down to the next post, and repeat for each of the thousands of posts. Even for one topic there were just too many posts.
  • I felt that the asynchronous discussion opportunities weren’t as good as synchronous ones, which allow for groups to be in the same mind space at the same time, feeding off each others’ ideas and coming up with new ideas. With asynchronous discussions, one might not get a response to one’s idea or comment until long after one has been actively thinking about it, and then at that point one may not be as interested in discussing it anymore (or at the very least, the enthusiasm level may be different).
  • The synchronous option of Google Hangouts seems to be a promising way to address the previous point, but I noted in my earlier post that there had been some reports of disrespectful behaviour in one or two of those in the “Think Again” course. I said I thought a moderator would be needed for such discussions, just as we have in face to face courses to ensure students treat each other respectfully.

Can a cMOOC address these concerns?

  1. From my experience with etmooc, the discussion does not have to get overwhelming. The thing is, each person focuses on what they want to focus on from the presentations, or from what others have said in their blogs, or from resources shared by others. There is no single “curriculum” that we all have to follow, so it’s not the case that everything posted by each person is relevant to everyone else’s interests and purposes for the course. This could be true of a philosophy or Arts One cMOOC as well, so it could be easier to pick and choose what, amongst the huge stream of things to read and think about, one wants to focus on.
  2. Synchronous discussions are difficult in a large group. In etmooc we have some opportunities for them in the presentations, which allow for people to write on the whiteboard, engage in a backchannel “chat,” and also take the mic and ask questions/offer comments. One could have the presentations have more time for discussion, perhaps, which could take place in part on the chat and in part via audio. It’s not as good as face to face discussions, though–much more fragmented.
  3. Google Hangouts are an alternative, though I haven’t tried doing one in etmooc. Some have, though, and reported success. However, the people taking etmooc are mostly professionals, both teachers and businesspeople, and they are both highly motivated and responsible/respectful. Having Google Hangouts where anyone in the world can show up could be inviting trouble. I don’t see a cMOOC addressing this problem.

cMOOCs in humanities–what’s not to love?

What other problems might there be with trying to do a cMOOC in humanities, whether on its own or as a supplement to another course? Or, do you love the idea? Let us know in the comments.

UPDATE: I just found, in that wonderfully synergistic way that etmooc seems to work, this blog post by Joe Dillon, which explains how well a cMOOC like etmooc stacks up to a face to face course. It’s just one example, but it can provoke some further thought on whether a cMOOC for humanities might be a good thing.

A MOOC by another name

 cc licensed ( BY ) flickr photo shared by Cikgu Brian

Last October I posted some criticisms of moocs (massive, open, online courses) in humanities as too massive to really deal well with promoting critical skills in learners.  Recent experience has made me change my mind, but it’s going to take two blog posts to explain. This is the first. (The second is here.)

Part of the issue with MOOCs that I expressed in my earlier post was that they were too content-focused, and seemed most conducive to topics in which that content can be machine-assessed (with multiple-choice or other automate-able question/answer formats). I wondered whether critical thinking, reading, writing and discussing skills could really be done well in a MOOC.

The problem is, at the time I wrote that I fell into the common trap of thinking that MOOCs are a monolithic type of entity. I may, perhaps, be forgiven this as most of the press about MOOCs is about the Coursera/EdX/Udacity type (as Alan Levine notes in a blog post–see below). It was only through participating in etmooc, a mooc about educational technology and media, that I found that there are other options.

Not all MOOCs are equal

One way of distinguishing types of MOOCs (at least at the moment…things are always changing) is to break them down into two categories: xMOOC and cMOOC. What do these categories mean? The “c” in cMOOC stands for “connectivist,” but I am not sure what the “x” in xMOOC stands for. [Update May 27, 2013: This Google+ post by Stephen Downes says he started calling them xMOOCs because of the “x” used in things like EdX–which stands for the course being an extension of regular university course offerings].

See here, here and here for some explanations of the differences between cMOOCs and xMOOCs. [update March 17, 2013:] Here’s an even more detailed discussion of the differences, by George Siemens. Lisa M. Lane has come up with three categories for MOOCs, though I’m not familiar enough with the “task-based” MOOCs to really comment on them.

Alan Levine has a thought-provoking blog post on the numerous experiments in open learning (should we call them MOOCs?) that are going on at the moment, and how they are very different from the xMOOC model. The range of possibilities in courses that are open to anyone and everyone is astounding.

The etmooc course I’ve been participating in since Jan. 2013 is in the cMOOC category (or, in Lane’s three categories, it’s a “network-based” mooc). The “connectivist” aspect of it is obvious, as it seems clear that one of the main points of the course is to help people forge connections in order to learn from each other. There is a set of topics, one every two weeks, with presentations by various people working in those fields (all archived here). But the emphasis is not at all on learning content. Rather, participants are encouraged to watch the presentations they are interested in, and then (and mostly) to interact with the rest of the community in various ways: through twitter (#etmooc), a Google+ community, a community-curated list of links on Diigo, and posting and commenting on blogs (syndicated in an etmooc blog hub, though many of us read them on an RSS reader). We also have a weekly twitter chat (#etmchat) in which we discuss issues related to the topic for the week.

There really is no single “place” where the course is; it exists in the discussions we have with each other, the blog posts and digital stories we create and share, the connections we make with others and the conversations (about etmooc and teaching/learning generally, and other things) that we have. I haven’t watched all the presentations, and don’t plan to. Nor is it encouraged. Over and over we are reminded by the course “conspirators” and other participants that etmooc is driven by our own interests (and our own schedules…some have more time than others), and that there is no such thing as being “behind” in etmooc. You dive in when and where you want, and the most important part is to engage in discussion when you can. Blog, comment on others’ blogs, participate in Twitter and G+, or whichever of those you feel you can do.

Among other things, the “about” page for the course says:

Sharing and network participation are essential for the success of all learners in #etmooc. Thus, we’ll be needing you to share your knowledge, to support and encourage others, and to participate in meaningful conversations.

Without the various conversations going on in and around etmooc, there really wouldn’t be a course at all. It exists in our connections and discussion, in the things we share and the comments we make.

In addition to forging connections, etmooc, and other cMOOCs from what I understand, are focused on content creation rather than passive learning of content. In etmooc we contribute to content creation by writing in our own blogs and commenting on those of others. Recently we did a segment on digital storytelling and we created numerous digital stories (see my blog post here for links to a few examples). Right now we are talking about digital literacy and are invited to participate in Mozilla’s work to develop a framework for web literacy (open to anyone to contribute).

Etmooc also requires self-directed learning–participants must choose what to focus on, what to read, what to write about, whether to keep up on twitter and G+ or not, etc. There is no set of course objectives that are decided in advance, as explained in this conversation about learning objectives and cMOOCs on Storify. Rather, as Alec Couros puts it in that Storify conversation, participants are to develop their own learning objectives. Different people will engage with the course for different reasons, pursue different paths. And that’s the point.

The value of a cMOOC

Does it work? Do people learn? All I have at the moment is anecdotal evidence.

I have learned more in the last few weeks in etmooc than I ever did in any other professional development opportunity. It’s because of the connections and discussions: I read others’ blog posts (only a few a week, really; don’t have time for more), comment, and get conversations going. And the same thing happens now on my blog. My twitter lists have expanded widely, and I am getting so many links to articles, blog posts and other resources that are useful for topics I’m interested in.

I agree with Michelle Franz, though I’d say it’s not just twitter I’m learning from in etmooc:

See also Paul Signorelli’s mid-term reflections on etmooc, where he gives this list of what he has done and learned so far (among other things):

I have become an active part of a newly formed, dynamic, worldwide community of learners; continue to have direct contact with some of the prime movers in the development of MOOCs; had several transformative learning experiences that will serve me well as a trainer-teacher-learner involved in onsite and online learning; and have learned, experientially, how to use several online tools I hadn’t explored four weeks ago.

MOOCs and feedback, interaction

Ted Curran notes in a recent article (found via @jackiegerstein) that MOOCs–or rather, xMOOCs–are “the internet-scale version” of huge  introductory courses at large universities with hundreds of students: “massive, impersonal, and uninspiring exercises.” He notes that this model works well if you want to save money (more students, fewer faculty), but it doesn’t work very well pedagogically. What is needed for both the online and in-person teaching and learning platforms, according to Curran, is more emphasis on faculty interaction with students: “personalized timely feedback and frequent interaction with the teacher is more important to student success than the quality of lecturer, the quality of the textbooks, or the use of technology in courses” (emphasis in original). What MOOCs, and online learning in general, can do is to allow faculty

to automate the less effective activities (lecturing, exams, grading) so they can spend more time interacting with students (discussions, online office hours, targeted interventions when students fail assignments.) In short, online teaching tools let teachers spend more time on students and less time regurgitating content.

I agree that faculty/student interaction in courses can be important; it’s one of the most-cited things that students in Arts One said in a recent survey that they valued about the course. But realistically, is this possible in a MOOC that has thousands of participants? How many faculty can actually interact in a meaningful way with students in a course whose enrollment is upwards of 10,000 students or more?

Enter the cMOOC.

Must the interaction that is necessary to student success come from the instructor? Why not set up and foster a space in which interaction is encouraged amongst participants–indeed, where interaction and discussion are as much of (or more of) the focus as content delivery?

I don’t think the discussion boards on most or all xMOOC courses are enough. Discussion boards are limited as a technology: for example, I think blogs are better for posting lengthy reflections, including links and photos/videos, etc. Following blogs and Twitter feeds also promotes more lasting connections to foster learning after the course is finished. Encouraging participants to blog, comment on blogs, and interact in other ways such as Twitter and Google+ (or similar) has, in my experience with etmooc, worked very well.

The experience is still huge–there are far too many blog posts, tweets, G+ posts to follow. But the conspirators and participants are constantly reminding each other that keeping up with it all is not the point. Again, diving in where and when you want is. That, and creating smaller groups organically, through creating connections–deciding which blogs and twitter accounts to follow regularly, for example. Or creating your own smaller group within the larger group, with its own wiki, as another example.

In etmooc the “conspirators” tweet regularly, join in on some discussions in G+, comment on a few blogs here and there, but they don’t even try to interact with everyone. Instead, they have managed to create a space where participants engage mostly with each other.

Now, a purely connectivist mooc won’t work for all purposes; I’m not arguing for replacing xMOOCs with cMOOCs entirely. After all, in some disciplines there is a certain amount of content that simply must be grasped before one can really engage in meaningful discussions with others about the field. Further, for participants to thrive in a cMOOC, they have to be self-directed learners, as noted above, and not everyone is comfortable with this sort of learning.

But why couldn’t xMOOCs take some ideas from the successes of cMOOCs and incorporate more connectivist principles and practices alongside the traditional methods of learning they tend to use?

MOOCs and the media

Alan Levine points out, in the post linked above, that in mainstream media outlets you won’t hear about many of the “experiments in open courses” that some cMOOCs could be called (including etmooc). While drafting the first part of this post I was also engaging in a Twitter conversation with Rolin Moe (@RMoeJo) about how the hype about MOOCs in the media focuses on one type of MOOC only, even though there are at least two. As he noted, the “connectivist” MOOCs tend to be popular amongst educators, academics, and a few others, and they aren’t winning the PR battle.

The other problem, as we discussed in our twitter conversation, is that cMOOCs are often run by volunteers, because they believe in open learning, and there isn’t much in the way of trying to monetize the efforts. That doesn’t make for interesting news, apparently.

A different name?

Since mainstream media has hijacked “MOOC” to mean xMOOC, perhaps it’s time to call the cMOOC something else? Which is ironic, since apparently the whole idea of MOOCs started with cMOOCs (see “connectivist MOOCs” here).

Nevertheless, would a new name help to avoid the confusion? Or is it enough to try to push the xMOOC vs cMOOC distinction?

*** Update March 14, 2013 *****

I just found this blog post by David Kernohan that points to a third option: open boundary courses, in which an on-campus course is opened to outside participants (usually not for credit). It seems to me the “open boundary” courses could be either more like cMOOCs or more like xMOOCs in structure.

Peer assessment: Face to face vs. online, synchronous (Part 1)

This is another post in the series on research literature that looks at the value of doing peer assessment/peer feedback in different ways, whether face to face, orally, or through writing (mostly I’m looking at computer-mediated writing, such as asynchronous discussion boards or synchronous chats). Earlier posts in this series can be found here, here, here and here.

In this post I’ll look at a few studies that focus on peer assessment through online, synchronous discussions (text-based chats).

1. Sullivan, S. & Pratt, E. (1996) A comparative study of two ESL writing environments: A computer-assisted classroom and a traditional oral classroom, System 29, 491-501. DOI: http://dx.doi.org/10.1016/S0346-251X(96)00044-9

 38 second-year university students studying English writing for the first time (where English was an additional language) participated in the study. They were distributed in two classes taught by the same professor, where all the teaching materials were the same except that in one class all class discussions and peer evaluation discussions were held orally, face to face, and in the other all class discussions and peer group discussions were held online, in a synchronous “chat” system. In the computer-assisted class, students met often in a computer lab, where they engaged in whole-class discussions and peer group discussions using the chat system.

[I see the reason for doing this sort of thing, so that students don’t have to spend time outside of class doing online chats, but I do always find it strange to have a room full of students and the teacher sitting together but only communicating through computers.]

Research questions:

(1) Are there differences in attitudes toward writing on computers, writing apprehension, and overall quality of writing between the two groups after one semester?; and

(2) Is the nature of the participation and discourse in the two modes of communication different?

In what follows I will only look at the last part of question 1 (the overall quality of writing), as well as question 2.

Writing scores

At the beginning of the term, students produced a writing sample based on a prompt given by the instructor. This was compared with a similar writing sample given at the end of the term. These were “scored holistically on a five point scale by two trained raters” (494).

In the oral class, strangely, the writing scores went down by the end of the term: at the beginning the mean was 3.41 (out of 5), with a standard deviation of 0.77, and at the end it was 2.95 with a SD of 0.84. The authors do not comment on this phenomenon, though the difference (0.46) is not great. In the computer class, the writing scores went up slightly: from a mean of 3.19 (SD 0.77) at the beginning to 3.26 (SD 0.70) at the end. The authors note, though, that “[t]he students in the two classes did not differ significantly (below the 0.05 probability level) at the beginning nor at the end of the semester” (496).

They did find a some evidence that the students in the computer assisted class did improve their writing:

However, some evidence was found for improved writing in the computer-assisted class by comparing the writing score changes of the two classes (computer-assisted classroom’ s gain (+0.07) to oral classroom’s loss (-0.46)). A t-test showed the difference to be significant at the 0.08 probability level. (496)

The authors conclude, however, that the data does not support saying one environment is better than another in terms of improving writing (nor, incidentally, for the rest of research question (1), above).

Discourse patterns in peer group discussions 

[The authors also looked at discourse patterns in the whole-class discussions, but as I don’t plan to do whole-class discussions via chats in the near future, I’m skipping that portion of the article here.]

There were more comments made in the oral class, during peer assessment discussions, than in the online chat groups: 40-70 turns per group for the oral discussions and 14-25 turns per group for the online chats (498). However, the authors found that the discussion in the oral class was, as they put it, “less focused” (498), in the sense that there were more interjections of personal narratives and repetitions of what other students had said. In the computer class, the talk was more “focused on the task of criticizing the writing rather than conversing with their fellow students while on the network” (499).

The tone of the article here indicates that the talk in the online chat was better than that in the oral discussion. But as noted in Hewett (2000), the sort of talk that might be interpreted as “unfocused” could also be interpreted as an important part of participating in an interactive discussion. Repetitions indicate that one is listening, following along, and being an engaged participant in a discussion. Personal narratives can both help to make a point as well as forge some connections between discussion group members, perhaps bringing them closer together and thereby helping them feel more comfortable (which could contribute to more productive peer evaluation).

In addition, in the oral groups the author of the paper being discussed often dominated the discussion, while the author spoke less in the online chats, making for more equal participation.

Continue reading →