Monthly Archives: February 2013

Peer Assessment: Face to face vs. online, synchronous (Part 2)

Here I look at one last study I’ve found that focuses on the nature of student peer feedback discussions when they take place in a synchronous, online environment (a text-based chat). Part 1 corresponding to this post can be found here.

Jones, R.H., Garralda, A., Li, D.C.S. & Lock, G. (2006) Interactional dynamics in on-line and face-to-face peer-tutoring sessions for second language writers, Journal of Second Language Writing 15,  1–23. DOI: http://dx.doi.org/10.1016/j.jslw.2005.12.001

This study is rather different than the ones I looked at in Part 1 of face to face vs. online, synchronous peer assessment, because here the subjects of the study are students and peer tutors in a writing centre rather than peers in the same course. Still, at least some of their results regarding the nature of peer talk in the tutor situation may still be relevant for peer assessment in courses.

Participants and data

The participants in this study were five peer tutors in a writing centre in Hong Kong, dedicated to helping non-native English speakers write in English. For both tutors and clients, English was an additional language, but the tutors were further along in their English studies and had more proficiency in writing in English than the clients. Data was collected from transcripts of face to face consultations of the tutors with clients, as well as transcripts of online, text-based chat sessions of the same tutors, with many of the same clients.

Face to face tutoring was only available in the daytime on weekdays, so if students wanted help after hours, they could turn to the online chat. Face to face sessions lasted between 15 and 30 minutes, and students “usually” emailed a draft of their work to the tutor before the session. Chat sessions could be anywhere from a few minutes to an hour, and though tutors and clients could send files to each other through a file exchange system, this was only done “sometimes” (6). These details will become important later.

Model for analyzing speech

To analyze the interactions between tutors and clients, the authors used a model based on “Halliday’s functional-semantic view of dialogue (Eggins & Slade, 1997; Halliday, 1994)” (4). In this model, one analyzes conversational “moves,” which are different than “turns”–a “turn” can have more than one “move.” The authors explain a move as “a discourse unit that represents the realization of a speech function” (4).

In their model, the authors use a fundamental distinction given by Halliday into “initiating moves” and “responding moves”:

Initiating moves (statements, offers, questions, and commands) are those taken independently of an initiating move by the other party; responding moves (such as acts of acknowledgement, agreement, compliance, acceptance, and answering) are those taken in response to an initiating move by the other party. (4-5)

They then subdivide these two categories further, some of which is discussed briefly below.

Results

Conversational control

In the face to face meetings, the tutors exerted the most control over the discussions. Tutors had many more initiating moves (around 40% of their total moves, vs. around 10% of those for clients), whereas clients had more responding moves (around 33% of clients’ total moves, vs. about 14% for tutors). In the chat conversations, on the other hand, initiating and responding moves were about equal for both tutors and clients (7).

Looking more closely at the initiating moves made by both tutors and clients, the authors report:

In face-to-face meetings, tutors controlled conversations primarily by asking questions, making statements, and issuing directives. In this mode tutors asked four times more questions than clients. In the on-line mode, clients asked more questions than tutors, made significantly more statements than in the face-to-face mode, and issued just as many directives as tutors. (10)

Types of questions

However, the authors also point out that even though the clients asserted more conversational control in the online chats, it was “typical” of the chats to consist of questions by students asking whether phrases, words, or sentences were “correct” (11). They did not often ask for explanations, just a kind of check of their work from an expert and a quick answer as to whether something was right or wrong. On the other hand, when tutors controlled the conversations with their questions, it was often the case that they were using strategies to try to get clients to understand something themselves, to understand why something is right or wrong and to be able to apply that later. So “control” over the conversation, and who asks the most questions or issues the most directives, are not the only important considerations here.

The authors also divided the questions into three different types. Closed questions: “those eliciting yes/no responses or giving the answerer a finite number of choices; open questions: “those eliciting more extended replies”; rhetorical questions: “those which are not meant to elicit a response at all” (12)

In the face to face sessions, tutors used more closed questions (about 50% of their initiating questions) than open questions (about 33%); the opposite was true in the online chats: tutors used more open questions (about 50% of their initiating questions) than closed (about 41%).

Continue reading

MOOCs and humanities, revisited

In the last post I discussed how I have come to learn about the different kinds of MOOCs through my participation in etmooc. I also said that through learning about a new kind of MOOC, the cMOOC or “network-based” MOOC, I was reconsidering my earlier concerns with MOOCs. Might the cMOOC do better for humanities than the xMOOC?

A humanities cMOOC

“Roman Ondák”, cc licensed ( BY ) flickr photo shared by Marc Wathieu

I haven’t yet decided whether or not one could do a full humanities course, such as a philosophy course, through a cMOOC structure. Brainstorming a little, though, I suppose that one could have a philosophy course in which:

  • Common readings are assigned
  • Presentations are given by course facilitators and/or guests, just as in etmooc
  • Participants are encouraged to blog about the readings and presentations and comment on each others’ blogs (through a course blog hub, like etmooc and ds106 have)
  • Dedicated Twitter hashtag, plus a group on a social network like Google+, and a group on a social bookmarking site like Diigo (see etmooc’s group site on Diigo)
  • Possibly a YouTube channel, for people to do vlogs instead of blogs if they want, or share other videos relevant to the course

Would this sort of structure be more likely to allow for teaching and practice of critical thinking, reading and writing skills, as I discussed in my earlier criticism of MOOCs (which was pretty much a criticism of xMOOCs)? I suppose it depends on what is discussed in the presentations, in part. The instructors/facilitators could model critical reading and thinking, through explaining how they are interpreting texts and pointing out potential criticisms with the arguments. They could talk about recognizing, criticizing, and creating arguments so that participants could be encouraged to present their own arguments in blogs as clearly and strongly as possible, as well as offering constructive criticisms of works being read–as well as each others’ arguments (though the latter has to be undertaken carefully, just as it is in a face to face course).

This would involve, effectively, peer feedback on participants’ written work. Rough guidelines for blog posts (at least some of them) could be given, so that in addition to reflective pieces (which are very important!) there could also be some blog posts that are focused on criticizing arguments in the texts, some on creating one’s own arguments about what’s being discussed, etc.

What you wouldn’t be able to do well with this structure are writing assignments in the form of argumentative essays. These take a long time to learn how to do well, and ideally should have more direct instructor/facilitator feedback rather than only peer feedback, in my view. Peer feedback is important too, but could lead to problems being perpetuated if the participants in a peer group share misconceptions.

Another thing you can’t do well with a cMOOC is require that everyone learn and be assessed on a particular set of facts, or content. A cMOOC is better for creating connections between people so that they can pursue their own interests, what they want to focus on. Each person’s path through a cMOOC can be very different. Thus, as noted in my previous post, there is not a common set of learning objectives; rather, participants decide what they want to get out of the course and focus on that.

One would need to have a certain critical mass of dedicated and engaged participants for this to work. If it’s a free and open course, then people will participate when they can, and can flit in and out of the topics as their time and interest allows. That’s fantastic, I think, though if there are few participants that might mean that for some sections of the course little is happening. So having a decent sized participant base is important. (How many? No idea.)

I envision this sort of possibility as a non-credit course for people who want to learn something about philosophy and discuss it with others. Why not give credit? There would have to be more focus on content and/or more formal assessments, I think (at least in the current climate of higher education).

A cMOOC as supplement to an on-campus course

Even if a full cMOOC course in philosophy or another humanities subject may not work, I can see a kind of cMOOC component to philosophy courses, or Arts One. In addition to the campus-based, in-person course, one could have an open course going alongside it. This is what ds106 is like. One could have readings and lectures posted online (or at least, links to buy the books if the readings aren’t readily available online), and then have a platform for students who are off campus to engage in a cMOOC kind of way.

Then, those off campus can participate in the course through their blog posts and discussions/resource sharing on the other platforms, like we do in etmooc. Discussion questions used in class could be posted for all online participants.  Students who are on campus could be blogging and tweeting and discussing with others outside the course as well as inside the course.

Frankenstein engraved

Frontispiece to Mary Shelley’s Frankenstein (1831),by Theodor von Holst [Public domain], via Wikimedia Commons. One of the texts on Arts One Digital.

Discussions would expand to include many more people with many more backgrounds and things to contribute, which is likely to enrich the learning experience. There might get to be too much for each individual to follow, but then one just has to learn to pick and choose what to read and comment on (more on this, below). All participants could make connections and continue discussions beyond the course itself.

Arts One has already started to move in this direction, with a new initiative called Arts One Digital. So far, there are some lectures posted, links to some online versions of texts, twitter feed, and blog posts. This is a work in progress, and we’re still figuring out where it should go. I think extending the Arts One course in the way described above might be a good idea.

Again, the main problem with this idea (beyond the fact that yes, it will require more personnel to design and run the off-campus version of the course) is getting a high number of participants. It won’t work well if there aren’t very many people involved–a critical mass is needed to allow people to find others they want to connect with in smaller groups, to engage in deeper discussions, to help build their own personal learning network.

Looking back at previous concerns with (x)MOOCs

Besides general worried about their ability to help students develop critical skills, I was also concerned in my earlier post with the following:

  • In the Coursera Course on reasoning and argumentation (“Think Again”) that I sat in on briefly, I found myself getting utterly overwhelmed by the number of things posted in the discussion board. I complained that I could scroll and scroll just to get through the comments on one post, to get down to the next post, and repeat for each of the thousands of posts. Even for one topic there were just too many posts.
  • I felt that the asynchronous discussion opportunities weren’t as good as synchronous ones, which allow for groups to be in the same mind space at the same time, feeding off each others’ ideas and coming up with new ideas. With asynchronous discussions, one might not get a response to one’s idea or comment until long after one has been actively thinking about it, and then at that point one may not be as interested in discussing it anymore (or at the very least, the enthusiasm level may be different).
  • The synchronous option of Google Hangouts seems to be a promising way to address the previous point, but I noted in my earlier post that there had been some reports of disrespectful behaviour in one or two of those in the “Think Again” course. I said I thought a moderator would be needed for such discussions, just as we have in face to face courses to ensure students treat each other respectfully.

Can a cMOOC address these concerns?

  1. From my experience with etmooc, the discussion does not have to get overwhelming. The thing is, each person focuses on what they want to focus on from the presentations, or from what others have said in their blogs, or from resources shared by others. There is no single “curriculum” that we all have to follow, so it’s not the case that everything posted by each person is relevant to everyone else’s interests and purposes for the course. This could be true of a philosophy or Arts One cMOOC as well, so it could be easier to pick and choose what, amongst the huge stream of things to read and think about, one wants to focus on.
  2. Synchronous discussions are difficult in a large group. In etmooc we have some opportunities for them in the presentations, which allow for people to write on the whiteboard, engage in a backchannel “chat,” and also take the mic and ask questions/offer comments. One could have the presentations have more time for discussion, perhaps, which could take place in part on the chat and in part via audio. It’s not as good as face to face discussions, though–much more fragmented.
  3. Google Hangouts are an alternative, though I haven’t tried doing one in etmooc. Some have, though, and reported success. However, the people taking etmooc are mostly professionals, both teachers and businesspeople, and they are both highly motivated and responsible/respectful. Having Google Hangouts where anyone in the world can show up could be inviting trouble. I don’t see a cMOOC addressing this problem.

cMOOCs in humanities–what’s not to love?

What other problems might there be with trying to do a cMOOC in humanities, whether on its own or as a supplement to another course? Or, do you love the idea? Let us know in the comments.

UPDATE: I just found, in that wonderfully synergistic way that etmooc seems to work, this blog post by Joe Dillon, which explains how well a cMOOC like etmooc stacks up to a face to face course. It’s just one example, but it can provoke some further thought on whether a cMOOC for humanities might be a good thing.

A MOOC by another name

 cc licensed ( BY ) flickr photo shared by Cikgu Brian

Last October I posted some criticisms of moocs (massive, open, online courses) in humanities as too massive to really deal well with promoting critical skills in learners.  Recent experience has made me change my mind, but it’s going to take two blog posts to explain. This is the first. (The second is here.)

Part of the issue with MOOCs that I expressed in my earlier post was that they were too content-focused, and seemed most conducive to topics in which that content can be machine-assessed (with multiple-choice or other automate-able question/answer formats). I wondered whether critical thinking, reading, writing and discussing skills could really be done well in a MOOC.

The problem is, at the time I wrote that I fell into the common trap of thinking that MOOCs are a monolithic type of entity. I may, perhaps, be forgiven this as most of the press about MOOCs is about the Coursera/EdX/Udacity type (as Alan Levine notes in a blog post–see below). It was only through participating in etmooc, a mooc about educational technology and media, that I found that there are other options.

Not all MOOCs are equal

One way of distinguishing types of MOOCs (at least at the moment…things are always changing) is to break them down into two categories: xMOOC and cMOOC. What do these categories mean? The “c” in cMOOC stands for “connectivist,” but I am not sure what the “x” in xMOOC stands for. [Update May 27, 2013: This Google+ post by Stephen Downes says he started calling them xMOOCs because of the “x” used in things like EdX–which stands for the course being an extension of regular university course offerings].

See here, here and here for some explanations of the differences between cMOOCs and xMOOCs. [update March 17, 2013:] Here’s an even more detailed discussion of the differences, by George Siemens. Lisa M. Lane has come up with three categories for MOOCs, though I’m not familiar enough with the “task-based” MOOCs to really comment on them.

Alan Levine has a thought-provoking blog post on the numerous experiments in open learning (should we call them MOOCs?) that are going on at the moment, and how they are very different from the xMOOC model. The range of possibilities in courses that are open to anyone and everyone is astounding.

The etmooc course I’ve been participating in since Jan. 2013 is in the cMOOC category (or, in Lane’s three categories, it’s a “network-based” mooc). The “connectivist” aspect of it is obvious, as it seems clear that one of the main points of the course is to help people forge connections in order to learn from each other. There is a set of topics, one every two weeks, with presentations by various people working in those fields (all archived here). But the emphasis is not at all on learning content. Rather, participants are encouraged to watch the presentations they are interested in, and then (and mostly) to interact with the rest of the community in various ways: through twitter (#etmooc), a Google+ community, a community-curated list of links on Diigo, and posting and commenting on blogs (syndicated in an etmooc blog hub, though many of us read them on an RSS reader). We also have a weekly twitter chat (#etmchat) in which we discuss issues related to the topic for the week.

There really is no single “place” where the course is; it exists in the discussions we have with each other, the blog posts and digital stories we create and share, the connections we make with others and the conversations (about etmooc and teaching/learning generally, and other things) that we have. I haven’t watched all the presentations, and don’t plan to. Nor is it encouraged. Over and over we are reminded by the course “conspirators” and other participants that etmooc is driven by our own interests (and our own schedules…some have more time than others), and that there is no such thing as being “behind” in etmooc. You dive in when and where you want, and the most important part is to engage in discussion when you can. Blog, comment on others’ blogs, participate in Twitter and G+, or whichever of those you feel you can do.

Among other things, the “about” page for the course says:

Sharing and network participation are essential for the success of all learners in #etmooc. Thus, we’ll be needing you to share your knowledge, to support and encourage others, and to participate in meaningful conversations.

Without the various conversations going on in and around etmooc, there really wouldn’t be a course at all. It exists in our connections and discussion, in the things we share and the comments we make.

In addition to forging connections, etmooc, and other cMOOCs from what I understand, are focused on content creation rather than passive learning of content. In etmooc we contribute to content creation by writing in our own blogs and commenting on those of others. Recently we did a segment on digital storytelling and we created numerous digital stories (see my blog post here for links to a few examples). Right now we are talking about digital literacy and are invited to participate in Mozilla’s work to develop a framework for web literacy (open to anyone to contribute).

Etmooc also requires self-directed learning–participants must choose what to focus on, what to read, what to write about, whether to keep up on twitter and G+ or not, etc. There is no set of course objectives that are decided in advance, as explained in this conversation about learning objectives and cMOOCs on Storify. Rather, as Alec Couros puts it in that Storify conversation, participants are to develop their own learning objectives. Different people will engage with the course for different reasons, pursue different paths. And that’s the point.

The value of a cMOOC

Does it work? Do people learn? All I have at the moment is anecdotal evidence.

I have learned more in the last few weeks in etmooc than I ever did in any other professional development opportunity. It’s because of the connections and discussions: I read others’ blog posts (only a few a week, really; don’t have time for more), comment, and get conversations going. And the same thing happens now on my blog. My twitter lists have expanded widely, and I am getting so many links to articles, blog posts and other resources that are useful for topics I’m interested in.

I agree with Michelle Franz, though I’d say it’s not just twitter I’m learning from in etmooc:

See also Paul Signorelli’s mid-term reflections on etmooc, where he gives this list of what he has done and learned so far (among other things):

I have become an active part of a newly formed, dynamic, worldwide community of learners; continue to have direct contact with some of the prime movers in the development of MOOCs; had several transformative learning experiences that will serve me well as a trainer-teacher-learner involved in onsite and online learning; and have learned, experientially, how to use several online tools I hadn’t explored four weeks ago.

MOOCs and feedback, interaction

Ted Curran notes in a recent article (found via @jackiegerstein) that MOOCs–or rather, xMOOCs–are “the internet-scale version” of huge  introductory courses at large universities with hundreds of students: “massive, impersonal, and uninspiring exercises.” He notes that this model works well if you want to save money (more students, fewer faculty), but it doesn’t work very well pedagogically. What is needed for both the online and in-person teaching and learning platforms, according to Curran, is more emphasis on faculty interaction with students: “personalized timely feedback and frequent interaction with the teacher is more important to student success than the quality of lecturer, the quality of the textbooks, or the use of technology in courses” (emphasis in original). What MOOCs, and online learning in general, can do is to allow faculty

to automate the less effective activities (lecturing, exams, grading) so they can spend more time interacting with students (discussions, online office hours, targeted interventions when students fail assignments.) In short, online teaching tools let teachers spend more time on students and less time regurgitating content.

I agree that faculty/student interaction in courses can be important; it’s one of the most-cited things that students in Arts One said in a recent survey that they valued about the course. But realistically, is this possible in a MOOC that has thousands of participants? How many faculty can actually interact in a meaningful way with students in a course whose enrollment is upwards of 10,000 students or more?

Enter the cMOOC.

Must the interaction that is necessary to student success come from the instructor? Why not set up and foster a space in which interaction is encouraged amongst participants–indeed, where interaction and discussion are as much of (or more of) the focus as content delivery?

I don’t think the discussion boards on most or all xMOOC courses are enough. Discussion boards are limited as a technology: for example, I think blogs are better for posting lengthy reflections, including links and photos/videos, etc. Following blogs and Twitter feeds also promotes more lasting connections to foster learning after the course is finished. Encouraging participants to blog, comment on blogs, and interact in other ways such as Twitter and Google+ (or similar) has, in my experience with etmooc, worked very well.

The experience is still huge–there are far too many blog posts, tweets, G+ posts to follow. But the conspirators and participants are constantly reminding each other that keeping up with it all is not the point. Again, diving in where and when you want is. That, and creating smaller groups organically, through creating connections–deciding which blogs and twitter accounts to follow regularly, for example. Or creating your own smaller group within the larger group, with its own wiki, as another example.

In etmooc the “conspirators” tweet regularly, join in on some discussions in G+, comment on a few blogs here and there, but they don’t even try to interact with everyone. Instead, they have managed to create a space where participants engage mostly with each other.

Now, a purely connectivist mooc won’t work for all purposes; I’m not arguing for replacing xMOOCs with cMOOCs entirely. After all, in some disciplines there is a certain amount of content that simply must be grasped before one can really engage in meaningful discussions with others about the field. Further, for participants to thrive in a cMOOC, they have to be self-directed learners, as noted above, and not everyone is comfortable with this sort of learning.

But why couldn’t xMOOCs take some ideas from the successes of cMOOCs and incorporate more connectivist principles and practices alongside the traditional methods of learning they tend to use?

MOOCs and the media

Alan Levine points out, in the post linked above, that in mainstream media outlets you won’t hear about many of the “experiments in open courses” that some cMOOCs could be called (including etmooc). While drafting the first part of this post I was also engaging in a Twitter conversation with Rolin Moe (@RMoeJo) about how the hype about MOOCs in the media focuses on one type of MOOC only, even though there are at least two. As he noted, the “connectivist” MOOCs tend to be popular amongst educators, academics, and a few others, and they aren’t winning the PR battle.

The other problem, as we discussed in our twitter conversation, is that cMOOCs are often run by volunteers, because they believe in open learning, and there isn’t much in the way of trying to monetize the efforts. That doesn’t make for interesting news, apparently.

A different name?

Since mainstream media has hijacked “MOOC” to mean xMOOC, perhaps it’s time to call the cMOOC something else? Which is ironic, since apparently the whole idea of MOOCs started with cMOOCs (see “connectivist MOOCs” here).

Nevertheless, would a new name help to avoid the confusion? Or is it enough to try to push the xMOOC vs cMOOC distinction?

*** Update March 14, 2013 *****

I just found this blog post by David Kernohan that points to a third option: open boundary courses, in which an on-campus course is opened to outside participants (usually not for credit). It seems to me the “open boundary” courses could be either more like cMOOCs or more like xMOOCs in structure.

Peer assessment: Face to face vs. online, synchronous (Part 1)

This is another post in the series on research literature that looks at the value of doing peer assessment/peer feedback in different ways, whether face to face, orally, or through writing (mostly I’m looking at computer-mediated writing, such as asynchronous discussion boards or synchronous chats). Earlier posts in this series can be found here, here, here and here.

In this post I’ll look at a few studies that focus on peer assessment through online, synchronous discussions (text-based chats).

1. Sullivan, S. & Pratt, E. (1996) A comparative study of two ESL writing environments: A computer-assisted classroom and a traditional oral classroom, System 29, 491-501. DOI: http://dx.doi.org/10.1016/S0346-251X(96)00044-9

 38 second-year university students studying English writing for the first time (where English was an additional language) participated in the study. They were distributed in two classes taught by the same professor, where all the teaching materials were the same except that in one class all class discussions and peer evaluation discussions were held orally, face to face, and in the other all class discussions and peer group discussions were held online, in a synchronous “chat” system. In the computer-assisted class, students met often in a computer lab, where they engaged in whole-class discussions and peer group discussions using the chat system.

[I see the reason for doing this sort of thing, so that students don’t have to spend time outside of class doing online chats, but I do always find it strange to have a room full of students and the teacher sitting together but only communicating through computers.]

Research questions:

(1) Are there differences in attitudes toward writing on computers, writing apprehension, and overall quality of writing between the two groups after one semester?; and

(2) Is the nature of the participation and discourse in the two modes of communication different?

In what follows I will only look at the last part of question 1 (the overall quality of writing), as well as question 2.

Writing scores

At the beginning of the term, students produced a writing sample based on a prompt given by the instructor. This was compared with a similar writing sample given at the end of the term. These were “scored holistically on a five point scale by two trained raters” (494).

In the oral class, strangely, the writing scores went down by the end of the term: at the beginning the mean was 3.41 (out of 5), with a standard deviation of 0.77, and at the end it was 2.95 with a SD of 0.84. The authors do not comment on this phenomenon, though the difference (0.46) is not great. In the computer class, the writing scores went up slightly: from a mean of 3.19 (SD 0.77) at the beginning to 3.26 (SD 0.70) at the end. The authors note, though, that “[t]he students in the two classes did not differ significantly (below the 0.05 probability level) at the beginning nor at the end of the semester” (496).

They did find a some evidence that the students in the computer assisted class did improve their writing:

However, some evidence was found for improved writing in the computer-assisted class by comparing the writing score changes of the two classes (computer-assisted classroom’ s gain (+0.07) to oral classroom’s loss (-0.46)). A t-test showed the difference to be significant at the 0.08 probability level. (496)

The authors conclude, however, that the data does not support saying one environment is better than another in terms of improving writing (nor, incidentally, for the rest of research question (1), above).

Discourse patterns in peer group discussions 

[The authors also looked at discourse patterns in the whole-class discussions, but as I don’t plan to do whole-class discussions via chats in the near future, I’m skipping that portion of the article here.]

There were more comments made in the oral class, during peer assessment discussions, than in the online chat groups: 40-70 turns per group for the oral discussions and 14-25 turns per group for the online chats (498). However, the authors found that the discussion in the oral class was, as they put it, “less focused” (498), in the sense that there were more interjections of personal narratives and repetitions of what other students had said. In the computer class, the talk was more “focused on the task of criticizing the writing rather than conversing with their fellow students while on the network” (499).

The tone of the article here indicates that the talk in the online chat was better than that in the oral discussion. But as noted in Hewett (2000), the sort of talk that might be interpreted as “unfocused” could also be interpreted as an important part of participating in an interactive discussion. Repetitions indicate that one is listening, following along, and being an engaged participant in a discussion. Personal narratives can both help to make a point as well as forge some connections between discussion group members, perhaps bringing them closer together and thereby helping them feel more comfortable (which could contribute to more productive peer evaluation).

In addition, in the oral groups the author of the paper being discussed often dominated the discussion, while the author spoke less in the online chats, making for more equal participation.

Continue reading

Resources on rhizomatic learning

For anyone interested in rhizomatic learning, as discussed in my earlier blog posts (here, and here), you might also be interested in the following.

I recently came across this glossary entry for “rhizome,” via a tweet by George (@reticulatrix). It is from a Theories of Media Keywords glossary, which appears to have been created by students in a course from 2004. I found this discussion of rhizomes extremely clear and grounded in theory. Especially helpful is the contrast between rhizomatic models and “tree” models.

In addition, through a comment on one of my blog posts, I found this blog, by Keith Hamon, called Communications and Society. It’s subtitled, “A blog to support Keith Hamon’s explorations of the rhizome,” and there are many, many posts there about rhizomatic learning. He discusses numerous theorists whose views are relevant to this topic. I plan to spend some serious time exploring Keith’s posts. He has continued the conversation started on my blog over at his, and he recently joined etmooc. I look forward to learning more with him!

Finally (though this does not exhaust the resources out there, I’m sure), I found this article by Tanya Sasser over at Hybrid Pedagogy. In this article, entitled “Bring Your Own Disruption: Rhizomatic Learning in the Composition Class,” Sasser argues for a rhizomatic learning approach to first year composition courses. It’s a good example of how to apply rhizomatic learning to a writing course. I have to think about it more carefully, but I plan to comment on this article soon. I am in agreement with the basic idea, but still am a little hesitant. Probably that’s because I’ve been fully immersed in, and convinced by, the idea of using rubrics and step-by-step learning for teaching writing. Still, I’m questioning at least the rubrics part–see this post.

Do you have any other rhizomatic learning resources you’d like to share? Please post them in the comments!

(etmooc) Digital Storytelling, you’re looking better every day

In a recent post I explained that I just haven’t been very into digital storytelling, the second topic in etmooc. While many of the other participants have been busy creating animated gifs, 5 card stories, photo stories and more, I just wasn’t engaged enough to try to do much myself.

But then something happened. Well, Cogdog (Alan Levine) happened.

He gave a presentation on digital storytelling for etmooc, which I was able to join live. I’m not sure what was so inspiring about it, really–he introduced some tools, talked about how to write stories, asked some of the participants to play pechaflickr during the session. But somehow, partway through, I started getting excited.

Probably it was Cogdog’s enthusiasm. He just is so into storytelling, and digital storytelling, that I thought, well, there must be something to this. His excitement was infectious. I caught it.

The part of the presentation that really got me, though, was when he talked about how professional writing could be more like storytelling, that we could provide information, but do it in a more engaging way. He cited a book by Randy Olson called Don’t be Such a Scientist, which discusses the need for scientists to reach a broader audience and the power of storytelling to help do so. Olson was a professor at a university and then moved into filmmaking, and argues that scientists could learn a lot from the world of storytellers, in order to make what they do more accessible.

So could philosophers

And it hit me that this could be a great way to try to make my class lectures, the presentations I do for classes more engaging. I already try to ensure I don’t do too much lecturing and also have a good deal of activities for students to engage in during class time, discussions, working together in groups, etc. But why not find a way to make the lectures themselves more like stories?

This is challenging, but it’s a challenge I’m suddenly wanting to take on. I just needed to find something that I felt passionate about, and getting students as excited as I am about philosophy is that something.

Why not start small, by trying to incorporate some of the aspects of good storytelling practice in some lectures (it will take awhile to change many or all of them!). Why not, for example, start with a hook, something that draws people in, present an obstacle, resolve it, and then set up for a new story? (As discussed here, where storytelling meets math.) This could be done fairly easily without requiring too much in the way of time or learning new technological tools.

But there’s more

Somehow I also got excited about the digital part of digital storytelling. I mean, I started to want to spend time with some of the tools. I started coming up with ideas for stories–like telling the story of a recent trip to New Zealand (some of the photos are posted on flickr, though the ones with people are private), or the story behind the name of this blog–and I was motivated to look around Cogdog’s 50+ ways to tell a digital story site to find tools that would work.

My previous reluctance was due to numerous reasons, but partly because I didn’t want to put a lot of time into learning a new tool and creating something with it, only to discover that in a couple years’ time the tool would disappear. It’s hard to know which of these applications will stick around and which will die off. It seemed a waste of time.

But then in his presentation Cogdog pointed out: sure, some of the tools will disappear, but you will still have all your source photos, video, text, transcripts, etc., and it’s not that hard to create the story again in something new. Good point. I’m still worried about making things for my son that will still be viewable 20 or 30 years down the road, so I’m making a photo book that will be printed; that way, technology obsolescence won’t destroy it (though dirt, water, and forgetting it in a box might).

A true story

So I got up this morning and re-recorded my “true story of open sharing” for Cogdog’s collection. I tried to start with something that was a little more engaging … “I got a comment on my blog.” Okay, that’s not very exciting in itself, but it could make you think about what sort of comment on my blog could lead me to want to tell a story. It might get people wondering.

The rest of the story is rather like it was before, but at least it’s a start. And I played around with iMovie (an application that comes with Mac computers) to add in a couple of titles, at the beginning and end, and put in some transitions from the titles to the video.

 I spent a good deal of time trying to lessen the background noise–an airplane, and my husband trying to get the pilot light on the gas fireplace lit. (I was originally going to film this in front of our fireplace, with the gas flames going, but it’s summer here in Australia and we turned off the pilot light. Turned out there was a trick to getting it back on and it took awhile to figure out! So I just filmed outside instead). I couldn’t really get the background noise gone completely without making my voice sound very, very strange. But it is better than it was.

Then, I put the video into Mozilla Popcorn maker, because I wanted to include some relevant links (e.g., to my home page, to my blog). Here’s the result.

Okay, so it took me a couple hours longer than I thought it would, but now I have the hang of Popcorn Maker. And special thanks to Glenn Hervieux (@SISQITMAN), who came to my aid on Twitter when I ran into a problem with it!

Peer assessment: face to face vs. online, asynchronous (Pt. 2)

This is part of a series of posts in which I summarize and comment on research literature about different methods of doing peer assessment. Earlier posts in this series can be found here and here, and part 1 corresponding to this particular post is here.

In this post I summarize, as briefly as I can, a complex study on differences between how students speak to each other when doing peer assessment when it’s in person versus on a discussion board (mostly asynchronous, but students also did some posting to the discussion boards in a nearly synchronous environment, during class time).

Hewett, B. (2000) Characteristics of Interactive Oral and Computer-Mediated Peer Group Talk and Its Influence on Revision, Computers and Composition 17, 265-288. DOI: 10.1016/S8755-4615(00)00035-9

This study looked at differences between ways peers talk in face to face environments and computer-mediated environments (abbreviated in the article as CMC, for computer-mediated communication). It also looked at whether there are differences in the ways students revise writing assignments after these different modes of peer assessment and feedback.

There were several research questions for the study, but here I’ll focus just on this one:

How is peer talk that occurs in the traditional oral and in the CMC classroom alike and different? Where differences exist, are they revealed in the writing that is developed subsequent to the peer-response group sessions? If so, how? (267)

Participants and data

Students in two sections of an upper-level course (Argumentative writing) at a four-year university participated; one section engaged in face to face peer assessment, and the other used computer-mediated peer assessment, but otherwise the two course were the same, taught by the same instructor. The CMC course used a discussion board system with comments organized chronologically (and separated according to the peer groups), and it was used both during class, synchronously (so students were contributing to it while they were sitting in a class with computers) and outside of class, asynchronously.

Peer group conversations were recorded in the face to face class, and the record of conversations from the CMC class could just be downloaded. The author also collected drafts of essays on which the peer discussion took place. Data was collected from all students, but only the recordings of conversations in one peer group in each class (oral and CMC) were used for the study. I’m not sure how many students this ended up being–perhaps 3-4 per peer group? [update (Feb. 28, 2013)] Looking at the article again, a footnote shows that there were four students in each group.

One of those groups, the CMC group, engaged in both computer-mediated peer discussion as well as oral discussion at a later point–so this group provides a nice set of data about the same people, discussing together, in two different environments. Below, when talking about the “oral” groups, the data include the group that was in the oral only class, plus the CMC group when they discussed orally.

Results

Nature of the talk in both environments

Not surprisingly, the student discussion in the face to face groups was highly interactive; the students’ statements often referred to what someone else had said, asked questions of others, clarified their own and others’ statements, and used words and phrases that cued to others that they were listening and following along, encouraging dialogue (e.g., saying “yes,” “right,” “okay,” “exactly”) (269-270).

In the CMC discussions, the talk was less interactive. Multiple threads of discussion occurred on the board, and each students’ comments could pick up on several at a time. This created a “multivocal tapestry of talk” that individuals would have to untangle in order to participate (270). At times, students in a peer group would respond to the paper being discussed, but not to each other (271), so that the comments were more like separate, standalone entities than part of an interactive conversation.

In addition, the possibility for asynchronous communication, though it could be convenient, also left some students’ comments and questions unanswered, since others may or may not return to the board after the synchronous group “chat” time had ended.

Subjects of the talk in each environment

Hewett found that face to face discussion had more talk about ideas, wider issues raised in the papers, and information about the contexts surrounding the claims and issues discussed in the papers, than in the CMC discussion (276). The CMC groups tended to focus more on the content of what was written, and showed less evidence of working together to develop new ideas about the topics in the essays. Hewett suggests: “Speculative thinking often involves spinning fluid and imperfectly formed ideas; it requires an atmosphere of give-and-take and circumlocution,” which is more characteristic of oral speech (276).

Continue reading

Peer assessment: face to face vs. asynchronous, online (Pt. 1)

I have been doing a good deal of reading research on peer assessment lately, especially studies that look at differences and benefits/drawbacks of doing peer assessment face to face , orally, and through writing–both asynchronous writing in online environments (e.g., comments on a discussion board) and synchronous writing online (e.g., in text-based “chats”). I summarized a few studies on oral vs. written  peer assessment in this blog post, and then set out a classification structure for different methods of peer assessment in this one.

Here, I summarize a few studies I’ve read that look at written, online, asynchronous peer feedback. In another post I’ll summarize some studies that compare oral, face to face with written, online, synchronous (text-based chats). I hope some conclusion about the differences and the benefits of each kind can be drawn after summarizing the results.

1. Tuzi, F. (2004) The impact of e-feedback on the revisions of L2 writers in an academic writing course, Computers and Composition 21, 217–235. doi:10.1016/j.compcom.2004.02.003

This study is a little outside of my research interest, as it doesn’t compare oral feedback to written (in any form). Rather, the research focus was to look at how students revised essays after receiving e-feedback from peers and their teacher. Oral feedback was only marginally part of the study, as noted below.

20 L2 students (students for whom English was an additional language) in a first-year writing course at a four-year university participated in this study. Paper drafts were uploaded onto a website where other students could read them and comment on them. The e-feedback could be read on the site, but was also sent via email to students (and the instructor). Students wrote four papers as part of the study, and could revise each paper up to five times. 97 first drafts and 177 revisions were analyzed in the study. The author compared comments received digitally to later revised drafts, to see what had been incorporated. He also interviewed the authors of the papers to ask what sparked them to make the revisions they did.

Tuzi combined the results from analyzing the essay drafts and e-feedback (to see what of the feedback had been incorporated into revisions) with the results of the interviews with students, to identify the stimuli for changes in the drafts. From these data he concludes that 42.1% of the revisions were instigated by the students themselves, 15.6% from e-feedback, 148.% from the writing centre, 9.5% from oral feedback (from peers, I believe), and for 17.9% of the revisions, the source was “unknown.” He also did a few finer-grained analyses, showing how e-feedback fared in relation to these other sources at different levels of writing (such as the punctuation, word, sentence, paragraph), in terms of the purpose of the revision (e.g., new information, grammar) and more. In many analyses, the source of most revisions was the students themselves, but e-feedback ranked second some (such as working at the sentence, clause and paragraph levels, and adding new information). Oral feedback was always low on the list.

In the “discussion” section, Tuzi states:

Although e-feedback is a relatively new form of feedback, it was the cause of a large number of essay changes. In fact, e-feedback resulted in more revisions than feedback from the writing center or oral feedback. E-feedback may be a viable avenue for receiving comments for L2 writers. Another interesting observation is that although the L2 writers stated that they preferred oral feedback, they made more e-feedback-based changes than oral-based changes.

True, but note that in this study oral feedback was not emphasized. It was something students could get if they wanted, but only the e-feedback was focused on in the course. So little can be concluded here about oral vs. e-feedback. To be fair, that wasn’t really the point of the study, however. The point was simply to see how students use e-feedback, whether it is incorporated into revisions, and what kinds of revisions e-feedback tends to be used for. And Tuzi is clear towards the end: “Although [e-feedback] is a useful tool, I do not believe it is a replacement for oral feedback or classroom interaction …”. Different means of feedback should be available; this study just shows, he says, that e-feedback can be useful as one of them.

Continue reading

(etmooc) Digital Storytelling, I’m just not that into you …

… and I’m not sure why. So I decided to work on a post to try to write my way to understanding my reticence.

We’re now into topic two of etmooc, on digital storytelling. What is digital storytelling?

Storytelling, by Surian Soosay (Flickr; links below)

According to a University of Houston site,

Digital Storytelling is the practice of using computer-based tools to tell stories. … [A]s the name implies, digital stories usually contain some mixture of computer-based images, text, recorded audio narration, video clips and/or music.

Storytelling is an art

It’s not just about putting together a video through, e.g., Mozilla popcorn maker or putting some photos to music with Animoto. It’s not just about finding and learning how to use the many, many tools available for digital storytelling.

As Alan Levine (@cogdog) notes in his reply to a tweet during a recent etmooc twitter chat (#etmchat), the tools aren’t the most important thing:

The difficult part of digital storytelling is storytelling itself (as noted in this post). Determining what story you want to tell and how to tell it (regardless of the media used) are the hard parts. Finding the digital tools to use to tell it generally come after you’ve figured out what you want to tell (though Amy Burvall intriguingly suggests the opposite may work as well). But first, you have to know how to create art, not crap. And that’s hard to do. (One might start by looking at the Digital Storytelling Cookbook by Joe Lambert, a partial view of which is available for free on this site.)

I think that’s one of the main reasons I’m not really as engaged in this part of etmooc as I was in the “connected learning” section–I feel like I would need to know more about how to not create crap, yet all I think I’m really getting so far is a list of tools and examples of how they’ve been used. I suppose one can learn a lot from just watching examples of good stories being told, absorbing ideas on how to do it through those. But to be quite honest, I just am not into spending the time to do that. For it does take a lot of time–not just to learn the various tools to figure out which one you might want to use for a story, but also to figure out how to tell a story well. This could be a real time sink, and I have too many other things on my plate at the moment.

The other issue is that I don’t see myself using these methods and tools in my philosophy courses very much. I’m willing to spend the time needed for things I foresee using in courses, but I’m not seeing that for digital storytelling. I really value the written, argumentative essay format that we tend to focus on in philosophy courses. I don’t think it’s the only thing that anyone should ever use to make a point, but it can be a very effective way to make clear, concise arguments. And it takes a lot of practice to do well, which is why I emphasize a good deal of writing in my courses (and am working on how to teach it better, with more scaffolding, so that students work through the process step by step).

Value of digital storytelling

This isn’t to say that I can’t imagine using digital storytelling to make arguments in philosophy. On the contrary, I can imagine students telling stories like this one, or this one, or this one in courses where we discuss current events, ethical, social and political issues. I agree with Alan Levine when he notes in this video that “if you want to motivate, or inspire, persuade, stand out, be different, touch people, then tell a story.” I think that some philosophical arguments can be made even more strongly, perhaps, with a story (and a digital story wouldn’t hurt, because it can be shared so easily across the web). I see that.

I also see Alec Couros point here that

Storytelling may give voice to individuals and groups who have been oppressed by a culture of literary dominance.

This is important–some people’s voices may get lost if we emphasize only reading and writing, if all we ever produce are written documents to make our points.

My apology

And yet, I’m still reticent. I think it’s because:

  • Storytelling is hard
  • I don’t know how I’d teach students to do it, plus use tools, in a 13 week course that is too short already
  • Storytelling is hard and I’m not sure I know how to teach it

That last point is easy, right? Just read some things, watch a lot of digital stories, and practice, practice, practice yourself and you’ll figure it out. That’s probably true; but somehow, even after recognizing all of the above, I still just don’t feel excited enough to want to put the time in to do all that.

And I still really like and want to teach students how to do well at writing argumentative papers. Regardless of all of the above, that is still a valuable skill to have, and it’s something I know a good deal about already. I feel like I want to let others focus on digital storytelling in their courses if it grabs them as something they want to do, and I’ll still focus on writing. And that way, hopefully, students can get different options in different courses.

But why do I feel I must apologize for this, explain it, defend it? I suppose it feels like the written argument is already overdone, something that has been emphasized over and over in education, and now it’s time for something new, better and more inclusive. Writing argumentative essays does leave some people’s voices out, those who aren’t good, for whatever reason, at expressing themselves this way. And it is not always as good at capturing and inducing emotional reactions as something like digital storytelling. So it’s not good for every purpose.

But neither is digital storytelling, of course (as I expect those who promote it will also recognize). And for some people, it just doesn’t resonate as much as a good, clear, strong argument–e.g., for me.

I am a little into you

Of course, being not “that” into something can also mean one is still a little into it. I did play around with some six word stories (okay, one of them was three words) during the past week or so, and I really enjoyed that medium (note: no need to spend time learning new tools, since I already know how to use Twitter).

 

This one was about trying to put our rubbish bag in a place where animals couldn’t get to it. Impossible: either the possums climb the tree or the wallabies jump to the bag. You can imagine the results.

This one isn’t, strictly speaking, true–I did have conversations on my blog and Twitter before etmooc. But I have many, many more conversations now. So I used “soliloquy” for dramatic effect.

[This superhero pic and story were added Feb. 18, 2014] I haven’t done this myself, but I’ve also really enjoyed the six-word stories that others have been doing with images as well as text. Here’s one of my favourites, from Margaret A. Powers (see the blog post about the origin of this story herethese superheroes are window washing at a children’s hospital).

Six word story from Margaret Powers

Also, I plan to use Mozilla Popcorn Maker to add some things to a true story of openness video I created for Alan Levine’s collection. I’ll upload that here later, when it’s done. [Actually, I ended up putting it into another blog post.]

Finally, I have a kind of digital story of my own etmooc experience (ongoing, in process) over at Storify.

Warming a little more

In addition, by writing this post and watching some more digital stories in order to find good ones to link to this post, I have come to recognize that using digital stories would be a much better medium that written essays for making philosophical arguments that might actually have more of an impact on large numbers of people. Since I do have a great interest in philosophical discussions and philosophical thinking and reflection not being just something academics engage in, then creating more digital arguments myself, and encouraging students to do so, would be a good thing. Then our conversations about philosophical issues could more easily extend beyond the classroom. I can envision students creating digital stories, posting them online, and engaging in conversations with others who see them. I can imagine using digital stories created by others in my courses and then having students respond to them, comment, create their own stories as part of a conversation. It could be the start of an interesting conversation for the students and for others. I can certainly see the potential.

This won’t work for every course, though; sometimes you just have to write a clear argument to explain just why and how a certain part of someone else’s argument doesn’t work, referring exactly to the words said, page numbers, others’ arguments, etc. And I won’t give up on the emphasis on writing argumentative essays, either; it’s just that I can see the value in asking students to produce a digital story for one of their assignments in addition to writing papers.

Back to the beginning

Still, that adds on more work for them, too, in an already-crowded, 13-week term. They’d have to learn a new tool and learn how to tell a story well. And it would mean I’d have to figure out myself what makes for a story that is not crap, which I am still not yet excited enough to take the time to learn. So I’m back to my original problem.

Thus, for now, I am ambivalent. Perhaps I am in the minority, in that telling and watching digital stories just isn’t that engaging for me. And perhaps that is either my personality or years and years of training in text-based philosophy.

Does anyone else fell similarly? Or perhaps you can offer me more arguments why it would be worth it to try to push past my ambivalence?

 

Photo credit: Storytelling, CC-BY license (2.0), shared by ssoosay

 

Peer feedback: oral, written, synchronous, asynchronous…oh my

In an earlier post I summarized a few studies of peer assessment/peer feedback, and since then I’ve done some more research and found several more studies. What I also realized is that “oral vs written peer feedback” is not an adequate description of the myriad options. There is also synchronous vs. asynchronous, and face-to-face vs. mediated (often computer mediated).

So, written feedback can be asynchronous or synchronous (such as with online “chats”), and done on paper or computer mediated (typed digitally and emailed or uploaded into some online system that allows students to retrieve the comments on their papers).

Oral feedback, in turn, can be face-to-face or computer mediated, and the latter can be asynchronous (such as recorded audio or video) or synchronous (such as real-time video or audio chatting).

Thus, the possibilities are (at least insofar as I understand them at this point):

The situations in two of the boxes are quite rare:

  1.  Oral, asynchronous, face-to-face feedback situation are probably unlikely to happen very often, such as showing videos of one’s feedback to another person in person (or doing the same with an audio recording).
  2.  Written, face-to-face, synchronous feedback by itself is likely also rare, since it’s more probable that students will be writing comments on each others’ papers in each others’ presence while also discussing comments together–in which case the situation would be a blend of written, face-to-face, synchronous and oral, face-to-face, synchronous.

Also, I’m not really sure about the written, face-to-face, asynchronous box; that is only face-to-face insofar as the comments are given to the peer face-to-face, on paper.

The reason why I’m taking the time to distinguish these various permutations is that the literature I’ve been reading lately falls into various boxes on the table. For example, Reynolds and Russell (2008) (see this post) would fall into the oral, computer-mediated, asynchronous box. Most of the literature that talks about “oral” feedback is talking about oral, face-to-face, synchronous feedback (as I was using the term “oral” before now).

So now I guess I’ll need to come up with a new naming convention, and it will likely be an ugly set of abbreviations, such as:

Oral, face-to-face (assumed synchronous): OFTF
— though maybe written face-to-face is rare enough that this could just be FTF?

Oral, computer mediated, synchronous: OCMS

Oral, computer mediated, asynchronous: OCMA

Etc. Quite ugly.

In the next few days I’ll summarize and comment on a few more articles that fall into these various boxes, and then see if I can come up with any conclusions about the different types of oral vs. written peer feedback from those summaries and the ones in the post linked at the beginning, above. Does the research allow for any stable conclusions at this point? I’ll have to see after I think through the various papers I’m about to summarize in the next few days…

 

Works cited

Reynolds, J. & Russell, R. (2008) Can You Hear Us Now?: A comparison of peer review quality when students give audio versus written feedback, The WAC Journal 19(1), 29-44. http://wac.colostate.edu/journal/vol19/index.cfm