Searching for philosophy on OER repositories

One of the activities in week 2 of the “open education” course at the Open University is to find open educational resources for a fictional course on “digital skills” we should imagine ourselves creating. Since I have pretty much no interest in (nor expertise for) creating a course on digital skills (whatever that means), I decided to look at some OER repositories to see what they have in the way of materials for philosophy.

This won’t allow me to do the activity as suggested, since I expect there are much fewer OERs on philosophy than on “digital skills,” which is probably why they suggested the latter. But it does allow me to do something that might be of more use to me in the future.

The instructions for the activity said we should look at several OER repositories:

I find it puzzling that Xpert isn’t on the list, but I’ll look at that as well.

Now, rather than trying to find OERs for an entire course, which I find too time-consuming and unnecessary for my purpose of just trying to get to know the strengths and weaknesses of these repositories from an instructor perspective, I decided to just search for a particular topic. I thought about things that I wish I could find good online resources for, to help students in my courses when my explanations aren’t enough (often it’s good to have resources that approach topics from multiple perspectives and with multiple modes). I also thought it might be helpful to compare the repositories when searching for the same thing on each, to see what the search and recovery experience is for each one.

Many things came to mind, including Plato’s view of “forms” or “ideas,” Foucault’s view of power and resistance (or of the relationship between knowledge and power, or of biopower, or, well, of most things), Kant’s categorical imperative, and more. But I figured there’d be more resources on philosophers whose views are taught most often, so that eliminated Foucault. And since “forms” and “ideas” are pretty vague terms they might be likely to mess with the search results. So I was left with Kant.

I’ll discuss the repositories I searched in alphabetical order.

“Search and rescue swimmers train in a pool…” cc licensed ( BY ) flickr photo shared by Official U.S. Navy Imagery

Ariadne

The first thing I noticed on the Ariadne site is that there is no information on this repository on the site itself, though through the link on the front page to the Ariadne Foundation I found this page, which seems to explain a fair bit about how the repository works. However, it’s not written for people new to OER and metadata to understand, so it left me still pretty clueless.

The sidebar shows “providers,” which lists (I think) the other OER collections that Ariadne searches, including MERLOT, OER Commons, OER Africa, and others I am not familiar with.

I appreciate how the sidebars allow you to sort results by language (OERs in many languages seem accessible here), format (such as PDF, html, xml, powerpoint, video, audio), context (educational context, such as postsecondary, CGEP, primary, secondary, training), and type (such as presentation, syllabus, image, lecture notes, project).

The search box at the top of the repository page doesn’t have an option for “advanced search,” so I just typed in “Kant categorical imperative,” and got…nothing. “No results found.” Okay, bad start.

A search for “Kant” turned up some resources in Engligh (many more in other languages), though they were mostly full courses that had some Kant in them. I clicked on “type” on the sidebar to focus only on “lecture notes,” just as an example, and all I got still were full courses. Turns out you have to click on the link to the course, which takes you to another repository where you have to click on the link to the course, then finally get to the online course and dig around to find what you want. Nonideal.

I guess I hoped you could just get right to the individual resources. Possibly you could on Ariadne, if you were looking for other topics for OER–I don’t know.

Connexions

The Connexions site explains itself as “a place to view and share educational material made of small knowledge chunks called modules that can be organized as courses, books, reports, etc.” So there won’t be whole courses here. The “about” page notes that:

  • “Connexions content is modular for easy remixing. This makes it easier and more cost effective to update and adapt content.”
  • all modules are in a standardized XML format (but can be downloaded in various formats)
  • all content is licensed as CC-BY
  • quality control is provided through a “lensing system by which trusted/knowledgeable vetters review and endorse content,” and those visiting the repository can view material through these “lenses”

It’s possible to browse the content by subject, language, popularity, and author/title/keyword. I tried “subject: humanities,” which led me to a page where all the humanities subjects were listed alphabetically, and I had to go through several pages of “more” to get to philosophy. Not good. But you can do a search from that page that is limited by subject, if you want.

Again, the search bar doesn’t offer an “advanced search” option, so “Kant categorical imperative” it was. This time I got five results, all in English, two of which actually had most of the same content, and all of which were separate “modules” rather than full courses.

They were all texts, which I think all OERs in Connexions are (a downside). All were also highly contextualized in the sense that the discussion was mostly focused on the particular context of the module. These were nearly all from applied ethics contexts, such as business ethics, ethics for engineers, and ethics for administrators. There were some parts that were mostly theory, but most of those were fairly superficial. I didn’t find anything useful for my purposes, which requires fairly in-depth philosophical discussion.

But if I had, I could have saved it to a “favourites” list, or put it in my own workspace in order to remix it. The ability to remix materials on site sounds kind of cool, though I haven’t tried it to see how it works. 

Finally, on the front page there is a direct link and instructions on how to contribute to Connexions, which is nice–it’s not clear how to contribute to Ariadne (don’t think you can, directly).

Jorum

The front page of the Jorum site indicates that resources found there are only from the UK: “Through Jorum, you can find and share learning and teaching resources, shared by the UK Further and Higher Education community.” If that’s the case, it seems unnecessarily narrow.

Like Connexions, there is an explicit invitation on the front page for contributing to Jorum, but yeah, you have to either be a UK educator or a “trusted depositor” to do so (or you can contact them to ask if you can). Hmmmmm. Why? Probably because it was funded by JISC, which is focused on education in the UK. But still, is it really useful even for educators in the UK to have and contribute to a resource that has OER from only their area of the world?

You can browse by subject, date, author, title or keyword. Jorum does have an “advanced search” option, which allows you to specify search terms in “full text,” “author,” “title,” “keyword,” “date” (creation or insertion), “language,” “type,” and more. Turns out this doesn’t help my particular search much, so I just did “Kant categorical imperative” again in the simple search bar.

This time I got 92 results, but it was hard to tell how many are really relevant. All I got was a title and an author, sometimes with an institutional affiliation and sometimes not. How can I tell without clicking on it whether or not a resource titled “Zero Chance? Aiming for Zero in Weapons Control” is relevant to my purposes? (Probably not, but there could be an in-depth discussion of Kant’s categorical imperative in there somewhere.) Some were obviously not relevant, such as “Statistics for Geography and Environmental Science: An Introduction in R,” and “Getting Started with SPSS.” But what about one called, simply, “Newspapers”? (How is that title helpful in any way?)

Since it was clear this search did not yield good results, I went back to the Advanced search and entered “Kant” in “full text” “AND” “categorical imperative” in “full text.” This time I got 3 resources only, which cut out some of the earlier ones that were relevant, and included one that wasn’t (“School Geography: Exploring a Definition”). There was one PowerPoint presentation on the Categorical Imperative that was not bad, but not detailed enough for my purposes.

Back to the simple search, and I found:

  • Some courses on philosophy from the Saylor Foundation–same problem as with Ariadne, except Jorum doesn’t even tell me whether to look for lecture notes, assignments, videos, or some other resource type with Kant in it, so this is even worse.
  • Some lectures by Peter Millican  on “general philosophy” for first-year students at Oxford, but when I click on the link for the feed I get something that goes straight to Google Reader, which is a problem on so many levels. I’m pretty sure this is a function of how I set my browser to handle feeds, but I don’t know how to fix it. Then, When I click on the second link to try to find these lectures, I am taken to a page with all of the podcasts from Oxford–more searching.
  • A module on “moral theories” in the context of health care ethics, without an author identified (hmmm…not sure why having no author specified bugs me, but it does)
  • Some resources from an “introduction to moral philosophy” course of some kind, but: (a) I had to view a number of PDFs before I found one on Kant, (b) these were all copyrighted–could I even put them on a course web page without asking for permission first? And (c) it still wasn’t detailed enough for my purposes.

And that was it for what looked relevant from title and author alone.

MERLOT

The MERLOT site could use a serious visual makeover. It is far too busy–and the moving type on the top is just puts it even more over the top.

According to this page, MERLOT has some value-added features, including comments provided by users and “learning exercises” provided by the contributor, to go along with some of the resources. MERLOT also subjects OERs to peer review, through discipline-specific editorial boards. The “about” section of the MERLOT site is the most extensive I have seen, providing quite a bit of information on policies and procedures, and much more.

I started with the “communities” portals, which apparently can provide not only OERs related to different disciplines, but also journals, conferences, etc. But no–nothing for philosophy. Under the “communities” site there is also a link to “compass,” which has a link to the “pedagogy” portal. I went there, and found a sizable list of resources related to things like writing assignments, active learning, inquiry-guided learning, and more. I didn’t have time to look at these carefully at this point, however.

MERLOT has an advanced search screen, which allows for searches by keyword, title, description, language, type, author, license, date, and more. Under “keywords,” which I used for “Kant categorical imperative,” you can choose “any words,” “all words,” or “exact phrase.” It’s possible to sort results by relevance, rating, date, and more.

However, my search came up entirely empty of results. So I searched for “Kant” instead. This got me 15 results, only one of which had been peer reviewed, a series of lectures on Justice by Michael Sandel at Harvard. One nice thing about MERLOT is that, like Ariadne, you can narrow your results by “type” of resource. You can’t seem to do so by language, once you’ve got the results list (you’d have to do that at the “search” stage). You can also narrow by “category,” which seemed to be a list of different disciplines or other contexts, such as humanities, arts, social sciences, and “Academic support services.”

The vast majority of results were full courses, with the same problem as noted above with Jorum: unlike Ariadne, I can’t tell what parts of these courses are related to Kant–lecture notes, assignments, or other. And after a good deal of time searching through the courses, unfortunately I came out with nothing useful for what I’m looking for. This isn’t the fault of the repository, necessarily…I’m beginning to wonder whether there is actually anything available for what I’m looking for.

MIT Open Courseware

MIT Open Courseware was one of the first (if not the first) repositories of open courses. Their “about” page says they publish “virtually all MIT course content.” Many of the repositories discussed above and below link to courses from MIT OCW.

You can search courses by topic, department, language (some have been translated into languages other than English), and more. There is an “advanced search” option that lets you search “all” of some words, an “exact phrase,” “at least one of the words,” and “none of these words”–a fairly robust way to handle search terms. You can also limit your search to things like syllabi, lecture notes, assignments, videos, animations, and more. You can sort results by relevance or date only.

I searched “all the words” “Kant categorical imperative” and didn’t limit to any particular type of materials. I got 18 results, most of which were PDFs. Some of them were the same materials I found through other repository searches, which is not surprising (and probably it would be a good thing if there were more that were the same–it’s best if you don’t have to go through six repositories or more to find results, because they find essentially the same things).

The results screen lists titles, URLs, and a passage from the resource with the search terms in it. No indication of what type of resource it is (except sometimes that’s in the title, sometimes not), and there are not always authors (sometimes in the title, sometimes not).  Sometimes the authors are listed on the resource itself, sometimes not (annoying). Here’s what I found:

Again, I don’t necessarily blame the site for not having what I’m looking for, but the results list could be improved by having more than just the title and a few words with the search terms in them. A “type” of resource, perhaps? An “author”? At least. Maybe even a description?

Finally, it’s hard to find what license(s) these materials have. Connexions states explicitly that everything is CC-BY, Jorum and MERLOT have the licenses on the landing page when you click on one of the search items (better if it were in the results list itself), and with Ariadne it depends on what other repository the resource is in, where you can find the license for the materials. For MIT OCW, you have to go to the very bottom of each page, with those tiny menus, and find “Help and FAQs,” then click around on various links to find where the heck the CC license is for MIT OCW. Finally, I found here a notice that they are using a CC-BY-NC-SA license. Shouldn’t that be more prominent? And only through that page did I find a link to the “terms of use page.” 

Now, to be fair, there’s a link to those terms on each resource I found, once you click on it and view it. But for those who want to know before they go through a search and find materials, it would be better to make that information easier to find.

Open Learn

 I originally thought the Open Learn site from the Open University would just be full courses, a kind of “open university” concept in which people take courses for free. The front page doesn’t help explain much what there is, in truth: it says you can “dip into insights from OU academics” (what does that mean?), try “free extracts from OU course materials” (aren’t they all free anyway?), and then go into deeper study with OU (does that mean actually take a course?). So I went to the “about” section at the bottom of the front page, and found this page, which told me I could not only take full courses but browse articles, videos and games from these courses.

The more I poked around in the site, the more I realized that it all looks the same…dreadfully, boringly the same. The same headers appear on each page, and sometimes it’s hard to tell you’re on a new page. But hey, I already knew that from the Open Education course that prompted this blog post; I just hoped that was the case only inside particular courses themselves. Nope.

There is only a simple search bar at the top of all the pages, so I tried “Kant categorical imperative” again. One result: from what I can tell out of context, it seems to be a discussion between Stephen Pinker and some others about one of Pinker’s books. Not relevant. So I tried just “Kant” and got 48 results. Once I got to the results page, then I got an option for an “advanced search,” which is strange. I could search for all words, a phrase, the usual; also approximate searches, like “starts with” or “approximate spelling”; also by file type, date, language, and many more. None were helpful for my purposes, however.

The results page allowed me to narrow my search, by parts of the Open Learn site, by people (authors? subjects of the resources?), and by organization (Open University, YouTube, BBC, and more). I narrowed by “Immanuel Kant” and got 22 results. The results page lists title, date, and author, as well as a brief, few-word excerpt that is sometimes helpful, sometimes not. I found, among other things:

  • A tag cloud that presumably led me to resources tagged with those philosophers’ names; clicking on “Kant” didn’t lead me to anything relevant
  • A video by Michael Sandel on Kant and the capacity for reason, which was interesting but not quite what I was looking for
  • A game about lying or not lying, which looks like it might be cool but again, takes too long to load
  • A podcast about the wrongness of killing, which features some of Kant’s views and is otherwise quite interesting

Nothing that suits my needs, but what I like is that there are quite a few different types of resources here, including videos, podcasts, and games. A quick perusal of the full 48 results from the “Kant” search showed nothing else of relevance.

What about licenses for the materials? Again, you have to dig to find this information. Sometimes there is a clear copyright symbol on materials, sometimes there is a CC license, and sometimes there is nothing (indicating fully copyrighted materials, since this page says that unless stated otherwise, everything is under copyright).

So I suppose Open Learn is good for learners who want to access materials, but many of the materials are copyrighted, and thus not terribly useful for teachers who want to reuse or remix them.

Xpert

 Xpert has a very clean home page, which, after MERLOT and Open Learn and some of the other repositories, is a nice break (Ariadne does too). The “about” page is written a bit beyond a novice’s understanding, but what I can gather is that there is an open source tool for authoring OERs that one can use, and then an RSS feed from one’s OER creations uploads them automatically to Xpert. Xpert is also JISC (UK) funded, but there’s nothing to indicate that the resources are only from UK educators.

You can subscribe to some RSS feeds, if you want–things like the latest CC-licensed resources, the latest MP3s, podcasts, and videos.

The “browse” function seems resource-heavy; it took ages to load, and froze my browser with a message saying a script on the page was not responding. I would blame this on my slow internet connection at the moment, except Xpert is the only thing I’ve been using for the past couple of hours that is acting like this. Eventually, when the page loaded I found that I could browse by institution, author, type, language, publisher, license, and keywords. I chose to use the “advanced search” option, which was kind of like “browse” but with a search bar. You could enter your search terms and then narrow by the choices above. Again, extremely slow.

When I searched for “Kant categorical imperative” and tried to narrow by language to english, I was puzzled by the fact that there seemed to be several choices for English: “en,” including “en-AU,” “en-US,” and more; and also “eng,” including “eng-GB,” “eng-US” and more; and also just “English.” Which to choose? Will I get different results for each? Hmmmm. I chose “English.” I clicked a box for CC-licensed materials only (there was also an option for UKOER, which I think is OER from the UK, but I didn’t check that box).

I got 40 results, and the results page gave titles, descriptions and authors, as well as license information and a link to “related content” (if there was any). It’s great to see the license information right on the search results page, and to be able to limit results to CC-licensed materials. What I would have also liked to see is something indicating the type of each resource (a course, an audio or video recording, etc.). I could tell from the description that most of the results were courses, but it was unclear from the titles and descriptions just what some of the results were.

Many results here, as with Jorum, were clearly way off the mark, such as an MIT course on programming languages, a course from the Johns Hopkins School of Public Health on Statistics for Psychosocial Researchers, an Open Learn course on Psychology in the 21st century. I’m guessing something about “categorical imperative” is messing with the results, so I tried just “Kant” instead.

Again, I ran into issues with Xpert–kept getting messages that the server was reset while trying to load the page. Eventually I got 17 results for just “Kant.” Most of these clearly had something about Kant in them, but many of those were full courses in which one has to hunt to find where Kant is–same problem as with Ariadne, above. I also got a good number of links that went to the Open Learn site, but got me only an “oops, something went wrong” page–broken links.

I did discover, however, (thanks to Pat Lockley, who explained what the page did when I couldn’t understand it on the face of it) that Xpert allows you to search for images, and it then attributes them. Use the search box at the top of this page. You also get results for “sounds” and “videos.” That’s all quite nice. I was going to use an image of Kant I found through this service for this post, but strangely I got a bunch of images of people in costume as well as some of women in lingerie.

Interim conclusion

I’m going to write another post in a few days summarizing the problems I ran into during this experiment, and some suggestions for OER repositories (given, of course, that I know nothing about the technological–and likely other–difficulties involved). All I’ll say for now is that I didn’t find what I was looking for, but I don’t blame the repositories for that. It’s probably the case that there either isn’t anything online that discusses the categorical imperative in a detailed way, or that whatever there is hasn’t been linked to a repository…and there could be various reasons for that. But I did find some issues when searching the repositories, and I’ll summarize those in my next post.

 

Issues with OER (#h817open, activity 7)

For week two of Martin Weller’s Open Education course at the Open University, one of the activities was to read three articles from a suggested reading list on open educational resources (OERs) and discuss three issues related to OERs and how they are being addressed.

I read several articles, but am focusing here on three:

  1. Albright, P. (2005). UNESCO (IIEP): Final forum report. Available at: http://learn.creativecommons.org/wp-content/uploads/2008/03/oerforumfinalreport.pdf
  2. Downes, S. (2007). Models for sustainable open educational resources. Interdisciplinary Journal of Knowledge and Learning Objects, 3, 29-44. Available at http://ijklo.org/Volume3/IJKLOv3p029-044Downes.pdf
  3. Smith, M.S. & Casserly, C.M. (2006) The promise of open educational resources. Change: The Magazine of Higher Learning, 38(5), 8–17. A pre-publication version is available here: http://learn.creativecommons.org/wp-content/uploads/2008/03/changearticle.pdf

Personally, I am most interested in the question of motivations for faculty to contribute OER–for me, it seems obviously a good thing to do, but I may be an anomaly. In what follows I also consider possible motivations for educational institutions to support OER, as well as the issue of moving beyond a producer/consumer model.

“Doors,” cc licensed ( BY ) flickr photo shared by JMacPherson

Why should teachers and faculty care about OER? Why do I?

I was less than impressed with the suggestion made in Smith and Casserly (2006) that faculty are motivated to make teaching and learning materials open simply because of “the idea that their content will have a much larger audience” (7). This doesn’t really say much at all unless accompanied by an explanation of why faculty might want a larger audience for their teaching and learning work. I suppose it’s possible that some want their resources to be viewable by a lot of people simply for the sake of ego, but I expect there are other reasons that would work better to motivate faculty to contribute OER.

One place to start is to ask why I think it’s important to open up my teaching materials, my reflections on teaching (e.g., my blog) and even my courses up to anyone who wants to view, reuse, remix them, etc. I’ve already done some thinking about the value of doing so, in an earlier blog post (in which I also consider some potential downsides). A few reasons, from that earlier post as well as some additions:

  • Making teaching more “open” can motivate one to ensure it is as good as possible (when it’s closed, it’s a bit easier to let some things slide, even with the best of intentions, due to time or other constraints). (As also noted in Albright, 2005 (p. 8)).
  • Seeing what others do with one’s work can lead to ideas to improve it.
  • Allowing others to use and adapt one’s work could potentially lead to collaboration with others, which can mean opportunities for thinking about teaching and learning in different (and potentially better) ways–e.g., it could reveal ways in which one’s thinking and teaching are unconsciously bounded by time, place or privilege. It might also lead to collaborative teaching, which could be beneficial for students (if different perspectives and methods are provided in the same course).
  • Students in a course that is “open” to many participants may benefit from the various ways of thinking, speaking and writing that could be more likely than if the course was made up of students who are similar in the sense of all being admitted to the same educational institution.
  • If we are really passionate about teaching, and think that what we do is valuable, then why should it be limited only to those who can pay, and to those who have had the experiences needed to be able to demonstrate their worthiness to be admitted to our particular institutions of learning? It is, of course, not necessarily the case that those without such credentials are incapable of learning the way students who are officially registered in our courses do.

The thought here is that perhaps one or more of these might be motivating for other faculty as well.

However, it is one thing to recognize that sharing OER is a good thing to do, and quite another to overcome the stresses of time constraints and heavy workloads in teaching situations in order to devote time to doing so.

Albright (2005) suggests some more pragmatic incentives (based on a forum discussion amongst many participants), including:

  • considering the development and dissemination of quality OER as evidence in support of tenure, promotion, and merit processes for teachers and faculty
  • giving awards for outstanding OER (along the lines, I suppose, of teaching awards)
  • “adoption of [other] institutional policies that encourage opening educational content and valuing the creation of such materials” (9), e.g., supporting faculty who wish to make their courses available to learners beyond the institution, by removing bureaucratic barriers to doing so and even providing IT and other support as needed

For UBC, especially for people in the “teaching” stream of Instructor I, Sr. Instructor, and Professor of Teaching, the first point here seems an obvious move. Insofar as those of us in this stream are evaluated on the basis of teaching, curriculum development, educational leadership (and related), then creation and dissemination of good quality OER seems to fit right into that already-established framework. The second suggestion seems fairly easy to implement as well (though of course, it requires time and effort to set up and sustain).

It’s crucial to point out here, though, that all of these suggestions can merely add to faculty workload, which in many cases is already too heavy. It makes sense to recognize the time commitment involved in developing OER, in opening up courses to wider audiences, etc., and to provide some kind of time compensation in return. How much depends, of course, on the degree of work needed for the OER. A course reduction is one possibility, as is reduction of service commitments.

Institutional support

But this raises another question, of course: what motivation do institutions have for providing such support for faculty to develop OER?

I have been impressed with the emphasis that many universities have on promoting open access research, at least through the provision of institutional repositories. At UBC, I have had extensive help putting my publications and conference papers into the UBC institutional repository, including having someone else find publishers’ policies on what version of publications can be posted online as open access, and when, and contacting publishers directly when needed.

But no one has encouraged me to put any of my teaching work online. I’m curious as to why there is emphasis and support for open access research at my and other universities, but not for OER. One might say that showcasing the research of the university can help with PR, including showing the public that the research they are helping to fund is useful. But showcasing the teaching that goes on at the university can work similarly, I would think, and could also be a way to attract students as well as faculty who are passionate about teaching.

Is the discrepancy between support for open access and OER yet another instance of the focus on research over teaching? More likely it’s that encouraging and supporting open access for research doesn’t require as many resources as doing so for OER (though really, this is just a guess on my part). Making publications open access means getting permission, ensuring they are tagged with the appropriate metadata, and posting them in an institutional or other repository. For OER, on the other hand, there are also other things to consider, including quality control, issues with technological interoperability, and questions about where to store them, at least (not every university does or should have an OER repository).

Thus, back to the question: what could motivate an institution like a university to encourage and support OER development amongst faculty? The possibility of attracting students and good teaching faculty through showcasing teaching and learning practice at the university may or may not be enough. Smith and Casserly (2008) point to the possibility of governmental support, but that is not likely to be long-lasting.

Of course, it may be that institutions are not the ones who are providing the financial and other support for faculty to develop OER. It may be other organizations that do so instead. Smith and Casserly (2008) suggest that scholarly societies might play a key role in such efforts, but they also link this to work by “volunteers” (13), which is not ideal. Downes (2007) lists numerous possible funding models that might work for organizations or even commercial ventures (34-35).

Still, even if the bulk of the work on OER is done outside of educational institutions, it would help if educational institutions provide some kind of encouragement for teachers and faculty to be involved in their development.

Collaborative development

Another potential issue that stands out for me in regard to OER is that it’s too easy to fall into a sense that as a teacher/faculty member one is “providing” OER for others to “use.” As both Albright (2005) and Downes (2007), it’s important to move from a “provider/user model” to “a community model of collaborative development” (Downes, 2007, p. 38). This is not simply to avoid the situation where a few countries and cultures are providing knowledge for the rest of the world, but also to reflect the reality of how OERs work most effectively. The idea is not for something to be created and remain static, used as is, but reworked and repurposed as necessary for new contexts. Thus, as Downes (2007) notes, pointing to a comment made by a participant in a UNESCO forum on OER, we should move from thinking of OER on the model of “‘knowledge for all’ to ‘construction of knowledge by all’” (38).

Downes takes this point further at the end of his article, when he discusses decentralizing OER production and dissemination. Noting the way Wikipedia and bitorrent work, Downes argues that a community of volunteers might be better for sustaining OER than a centralized organization. The distinction between producers and consumers could be collapsed in the sense that “The use of a learning resource, through adaptation and repurposing, becomes the production of another resource” (41), which could then be re-uploaded into a repository. This collapsing of roles could work even if OER are controlled more centrally, but perhaps would be even more likely if not. The idea of such a decentralized system is intriguing, though again, relying on volunteer effort can simply add to already-heavy workloads for teachers and faculty.

But importantly, Downes’ article points to the fact that users of OER are not simply other teachers, but also learners, and collapsing the distinction between producers and users of OER means also doing so for teachers and learners. Thus, learners can and should also be involved in producing and disseminating OER. Many teachers and faculty recognize the value of involving learners in the development of curriculum and materials in traditional teaching situations; the same could be said for OER. Working on altering and re-submitting OER could be an effective part of the learning process, and thus not “extra” work on top of a curriculum.

More?

I am sure there is more to be said about possible motivations for teachers/faculty and institutions to support OER, despite the fact that this is one of my longest blogs posts ever. Ideas?

P.S.

Successful completion of this activity can earn participants in the Open Education course an “understanding OER” badge. But the instructions say the blog post should be around 500 words or so. I did not follow the rules. Wonder if I can still get a badge, or if I get docked for my post being too long, as I sometimes tell my students might happen. Or hey…I’ve written nearly enough for FOUR badges!

Your Turn (#h817open, Activity 3)

I’m trying to participate in another MOOC (massive, open, online course), though being away on holiday for a couple of weeks is making it difficult. It’s the “Open Education” course from the Oen University.

This is the sort of MOOC where it makes sense to say I’m “behind,” because there is a clear weekly schedule with weekly assignments to complete. I’m about a week behind. This structure is not appealing to me at the moment, after ETMOOC, which was much more open to participants doing whatever they felt most meaningful in response to the presentations, Twitter chats, shared links, and more. But I’m giving it a go because I really want to learn more about open education. I’ll probably do a few of the assigned activities, and then for the rest just write blog posts about what strikes me as most interesting.

Here’s one of the activities for the first week. We were asked to read Martin Weller’s article entitled “The Openness-Creativity Cycle in Education” and Terry Anderson’s slides from a Keynote for Alt-C in 2009. (Alt-C is the conference for the Association for Learning Technology). The activity was to then create a visual representation of open education, based on what we had read. “The key is to provide a representation that draws together the key concepts of openness as you perceive them,” according to the activity instructions.

I didn’t actually do that. Instead, I focused on one aspect of open education, or rather, one aspect of “openness”: that it’s more than just being available for reading/viewing for free, but is also available for remixing, reworking, taking pieces and doing something entirely different with them.

Part of the reason I did this was because I wanted to take on the challenge of creating a visual representation that didn’t just replicate a textual one. And if I tried to pull together all the strands of open education from the two readings (and elsewhere), I’d just end up putting into visual form the same things I would have otherwise written in text. That’s pretty much all I do in my work: write and speak narrative text. And I wanted to try to get beyond that a bit by using a picture that speaks. As you’ll see with the image, though, I couldn’t get away from text entirely.

This is not my thing, usually, so I won’t be surprised if it doesn’t entirely work, and if what seems good today seems simplistic tomorrow.

Since I seem incapable of doing without textual explanations, I will just say this. Multiple pieces made by others can appear as useless scatter, or as raw materials.

P.S. The other reason I did this is because one of my Twitter peeps created Ruschagram and this was a great excuse for me to play around with it.

 

 

Goodbye, #etmooc

ETMOOC is finishing up next week, and I’m about to leave town and be very sleep deprived for the next 3-4 days or so, so this is kind of my last hurrah for ETMOOC. I was trying to think about how/why my experience in it has been so important, so much so that I’m very sad it’s nearly over.

Instead of writing my response to that, I decided to try my first vlog–as a final project for ETMOOC, something else I’ve never done before (though I did do a “true story of openness” for Alan Levine).

Not surprisingly, it’s probably too long–just like my blog posts, my articles, and my plans for class meetings.

And I had only a couple of hours today to get it done, so it’s not terribly polished. I have a lot to learn about video editing, such as how to not only edit out portions of the video but also the audio that went with them (I tried to get rid of some “ummm’s” and “so’s,” and the video got deleted but not the audio. Hmph.). And that handmade heart at the beginning and end? Yeah, that should have been a nice title for the video…didn’t have time to do anything but cut one out and write on it.

So this is my love letter to ETMOOC. So sad to see you go.

[P.S. Is it possible to change the “still” shots from videos (such as the one below) so they aren’t those horribly strange facial expressions that come up when you’re in the middle of saying something?]

Why do I care if I’m attributed?

During one of the Twitter chats for the ETMOOC topic on “The Open Movement – Open Access, OERs & Future of Ed,” Pat Lockley Tweeted this:

 

We were talking about sharing our educational or other work, why some people find this difficult, the difference between “open access” and things being open in a wider sense, and more.

During the chat Pat’s Tweet kind of just went past me, but as I went back to the #etmchat Tweets for that day to add some to my Storify board on my ETMOOC experience, I came across it again and became curious as to what he meant. Thus started a fairly long conversation about copyright, licenses, public domain, and more. You can see it all here.

There’s a lot I’d like to think about further in this conversation, but what is really standing out for me at the moment is this:

 

 

Why am I using a CC-BY license on my work? Why do I care if I’m attributed when someone uses something from my blog, or some “open educational resource” I create? Pat brought up an important point:

 

 

Why not make one’s work public domain instead of using something like CC-BY? In the current legal climate, apparently it’s rather complicated: some places, like Canada and the U.S. (and probably other places too–I haven’t done enough research to list them), grant copyright simply through creating a work, and this may not actually be easy (or possible?) to give up (see, e.g., re: the U.S., Wikipedia on granting work into the public domain, and this post from the Public Domain Sherpa, and the last section of this page from Copyfree). One can, though, try to state as clearly as possible that one gives up all copyright and related rights to whatever extent allowed by law, and if not allowed, to give a license to anyone to use the work however they wish, without requirement of attribution. That’s what Creative Commons CC0 is meant to do. Copyfree has a list of various licenses that conform to their standard of “free use,” “free distribution,” free modification and derivation,” “free combination” and “universal application,” and CC0 is one of them (as is the Nietzsche public license, which is rather a personal favourite).

So, getting back to the original question and modifying it a bit: why not just use CC0 or something similar, thus releasing one’s work for any use by anyone, without attribution? Why care about attribution?

As Pat Lockley noted, it would be good to know that others find my work useful and that they reuse, repurpose and/or rework it. This would be helpful, if for no other reason than to validate for yourself what you’re doing. It could help you do more of it, perhaps. Knowing this would probably also be a way to improve one’s work through finding out what others have done with it. Not to mention it could be a way to potentially connect with others, which might even lead to collaborations.

In my own situation, on a pragmatic level, if I could discover and document how others have used my work, this could provide evidence that what I am doing has influence in the wider educational community, which might be one of several ways to support a claim of “educational leadership” or “distinction in the field of teaching and learning” for the new Professor of Teaching rank at UBC.

So yes, there are plenty of good reasons to be able to know what others are doing with your work.

But all of this requires what is NOT happening with CC-BY (and possibly not with other licenses…I haven’t done enough research to specify): notifying the attributed person that their work is being reused. If another blog links to your blog, you may get a pingback (maybe not; depends on the settings of your blog and the other blog, I think). And it’s a good practice to let other people know when you’ve used their work, if there’s an easy way to do it (such as leaving a comment on a photo posted on Flickr). I try to do that, but too often I forget (I’m working on this).

As noted towards the end of the Storified conversation with Pat, what’s missing, in order to get the benefits noted above, is some systematic way to notify people as to how you’ve used their work. I don’t even know how such a thing could work–the technological hurdles seem huge–but theoretically, it seems a good idea. Now, like any such things, one wouldn’t have to choose such a license (an attribution + notification license?), but for some it would provide a useful way to not just be attributed, but to know what uses their work is being put to. Perhaps it is too difficult/too much of a hassle to bother with. But it’s an intriguing idea.

“Attribution,” by fotogail (see below)

Of course, there are good arguments for making work as free as possible, without restrictions on what you have to do once you’ve accessed it–like attributing the author/creator, or telling him/her what you’re doing with it. So I’m undecided whether I, personally, would want to require more of the people using my work than just attribution. I might not even recommend this to others. But some might want to do it, and it could be useful.

But until and unless something like this happens, I’m back to my original question: Why do I care about attribution? If, for the most part, I won’t get the above benefits, what am I getting out of knowing that perhaps, somewhere out there, is a piece of work with my name attached?

One might think that it’s kind of like citation in academia; except again, citations are tracked whereas use of my CC-BY work (unless it’s a publication) is not. So really, it’s just a sense that other people know I created something. Why should I care about this?

Add to this the point that much of my work is not, perhaps, really “mine” in a deep sense because it is a culmination of so many other influences, work by so many other people that I have read or otherwise interacted with, and the question becomes even more pressing.

Okay, maybe it will come back to me at some point; maybe I’ll discover my work being used somewhere with my name, and then I can realize some of the good things noted previously. But maybe not (and perhaps most likely not). Or perhaps someone will find something with my name on it and decide to connect with me–thus leading to a connection through effort on someone else’s part rather than mine. These things might happen, but is that enough to require attribution for my work? I’m not yet sure.

I don’t have an answer, and you can’t answer for me of course, but maybe you have some ideas on why asking others to attribute one’s work might be a good idea, rather than just letting it go free into the wild. I’m thinking not so much for people who have to rely on their work to make a living, to make money off of it, but for people like me who are getting a salary from a university and could just share their blog writings, their photos, their OERs for free and without restrictions.

Help me out here?

Image credit: “Attribution,”  flickr photo (CC-BY) shared by fotogail

Introduction for #h817open

Even though ETMOOC still has another 1.5 weeks to go, and though I’m about to leave town for two weeks, I still decided to enrol in another open course: Open Education from the Open University. I will not be able to participate to the level I have in ETMOOC, given that I have so many things to do before my sabbatical ends and I leave Australia at the end of June. But I think it will be useful, given that it’s focused on various aspects of open education, including discussing just what “open” means, talking about OERs, MOOCs, and more. So some of my blog posts in the next seven weeks will be focused on activities related to that course, but I expect they’ll be of interest to at least some current readers as well.

For fellow participants in the open education course (h817open): my name is Christina Hendricks, and I’m a Sr. Instructor in Philosophy at the University of British Columbia, which is a tenured position focused on teaching (rather than research). My About.me page has a good deal of info, as does my Google+ profile. This is me on Twitter.

I’m currently on sabbatical in Melbourne, Australia, trying to jump start a new research focus in the Scholarship of Teaching and Learning (SoTL). I’ve done quite a bit of reading on learning communities (as I teach in one at UBC, and Chaired it for two years: Arts One) as well as peer assessment–if you’re interested, you could click on the “Scholarship of Teaching and Learning” category on the right, as I think I’ve put all (or at least most) of my posts related to these issues into that category. I’ve also got a set of posts called “Research Reviews,” in which I summarize the main points of some of the SoTL articles I’m reading and respond to them–those can be found at the menu on the top.

Even though I really don’t have time to participate in h817open as I’d like, I decided to enrol because through my experience in another MOOC I’m in the process of finishing, ETMOOC (Educational Technology and Media MOOC), I’ve become very interested in open education and really want to learn more. I had an idea for opening up my own courses in Philosophy or Arts One: see my earlier posts here and here for explanations.

I may not post every week, but I’ll try to do so at least every two weeks. And I’m an avid Twitter user, so I’ll definitely be following and contributing to the #h817open hashtag. I look forward to connecting!

 

(etmooc) On openness and panopticism

“Panopticon,” cc licensed ( BY ) flickr photo shared by chad_k

A year or two ago a student came into my office and told me about some podcasts he had been listening to, which consisted of some lectures by a well-known philosopher as part of one of his university courses. The student then asked me why I didn’t put my lectures out on podcasts, or make them public in some other way.

I don’t remember what I said. But I do remember what I felt: apprehension. And some fear. I couldn’t imagine, at the time, doing such a thing.

Now I can, and largely through my experience in ETMOOC I’ve become very interested in the idea of “openness” in education and want to start doing some of this myself. Of course, “open” means different things in different contexts (here’s a nice post explaining some of them, and here’s an even larger list of various “opens”) , but I’m considering things such as posting and licensing many of my course materials for re-use, as well as possibly opening up a course to outside participants the way Bryan Jackson did with his high school Philosophy course.

The value of open education

There are plenty of good things about opening up your teaching and learning materials, space, interactions, etc. Bryan Jackson explains something good that happened as a result of having an open Philosophy course, in this video. Barbara Ganley had an interesting experience from a writing assignment in her class posted publicly on a blog (see “A Writing Assignment Gets Personal,” on this site).

David Wiley, in a presentation on open education called “Openness, Disaggregation, and the Future of Education” (the keynote for the 2009 Penn State Symposium for Teaching and Learning) gave several examples of things he had done recently in his courses to make them more open. Among them:

  • He required that all students’ written work must be made public on the course blog. One result of this was that Stephen Downes—a prominent Canadian researcher, blogger, cMOOC facilitator, and editor the popular newsletter OLDaily (online learning daily)–had read some of the work and highlighted a few posts, sending them out to thousands of his followers in the OLDaily newsletter. Wiley noted that the following week, much of the students’ writing got longer, better, and more thoughtful. Such improvement came much better this way than just encouraging students to write more carefully and address issues more deeply through the instructor’s comments.
  • He wrote up a script for a fake sitcom (situation comedy) tv show, to show differing viewpoints on opening up “learning objects” (what are now called open educational resources, I think). He put this up on a course wiki, and some of the graduate students in the course started writing in new characters in order to give even more perspectives. They hadn’t asked or said they were going to do it, but just did. This was, he stated in the talk, a great way to get students involved in creating learning materials for the course itself.

My experiences in ETMOOC are good evidence as well: I now have a much wider network of people to talk to about teaching and learning, and educational technology, because this course is open to anyone who wants to join and participate. I have more comments on my blog, many more twitter interactions, more people to help answer questions (I just ask the Twittersphere and answers come quickly), more links to helpful resources for my own thinking and teaching and learning, and more.

These are just a few examples of good things that can come from opening up education. I’m certain there are many more. 

In addition, ETMOOC-ers said some good things about the value of openness in a recent Twitter chat:

I can see many benefits to opening up my teaching and learning more than I’m already doing, and I expect there are more that I can’t even currently imagine.

So was I reticent before, when my student asked about podcasting my classes, only because I didn’t see these benefits then? I don’t think so.

Fear and Openness

There are many ways of making one’s courses more “open,” including just posting one’s course materials for others to see (e.g., written materials, digital presentations, video or audio of lectures); giving the materials a Creative Commons license that allows others to reuse, repurpose, and build on them; live streaming your class meetings publicly; all the way to opening out the course to any participants who want to join (see Alec Couros‘ Social Media & Open Education course as an example, as well as Bryan Jackson’s high school philosophy course noted above). The concerns I bring up below apply to all of these, but mostly to the last two.

The apprehension I felt at the idea of podcasting my lectures wasn’t just the usual fear of being in front of a camera or having one’s voice go out into the wider world; I was a college radio DJ in university and grad school, and am don’t mind speaking into the void with the knowledge that many people (or none) might be listening. Video is still a little tough for me, but I’m quickly getting over that.

It wasn’t just a lack of confidence, a sense that no one would want to listen to my lectures when they have access to those of people who are much more expert than me on the topics they’re discussing (though there was some of that too).

There was something about potentially being watched, being observed, at any time, by anyone; but mostly, by those who could have significant influence over my future. It’s not that I worry my teaching isn’t very good, or that I think bad things would happen if those who can affect my employment see most or all of what I do in class. I actually have (and have had) fantastic colleagues, and every time I’ve had a peer visit a class it has ended up being a very positive experience, complete with helpful advice–much of which I still vividly remember and use.

I think it was partly that in having my courses be “open” it’s as if I could be undergoing a peer review of teaching at any time, all the time. 

Which means, of course, (being a Foucault scholar) that I thought of Foucault.

Panopticism

Panopticon

Panopticon, Jeremy Bentham [Public domain], via Wikimedia Commons


In Discipline and Punish (1975), Foucault wrote a great deal about the “disciplinary society” being a “panoptic” one, referring to Jeremy Bentham’s idea for a panoptic design for a prison. Section 3.3 (“History of the Prison”) of the Stanford Encyclopedia of Philosophy’s entry on Foucault is a nice, concise explanation of Foucault’s discussion of panopticism and discipline. And here is a post that connects panopticism to social media, and starts to get to the concern I’m working towards here.

It’s not just a concern about possibly being observed at any given moment. Nor is it only that there could be a potential danger to this vis-à-vis power relations in one’s place of employment. It’s also that this situation of potentially being observed at any given moment can pressure one to change one’s own behaviour in order to bring it more in line with dominant norms. We police ourselves, rather than having to be policed. There doesn’t even have to be anyone watching for this to happen.

Now, this isn’t always necessarily bad. I agree with Alec Couros’s tweet, above, that knowing others might see my work would spur me to make it as good as possible. Plus, of course, if others saw it and commented, this could help me improve it even more.

But the potential downside is that one might be less likely to try radically new things, to experiment, to risk doing things that don’t fit with dominant views of how education is “done.” Clearly this isn’t true for everyone; there are people doing innovative things openly (e.g., many of the conspirators in ETMOOC)–though even then one usually has a community with its own norms that one is part of.

The issue would be prominent especially for those who don’t have tenured or otherwise semi-permanent positions–it’s often (though not always) in their best pragmatic interest to police themselves not to take too many risks if their work is open, though some risk-taking might be seen as positive.

So one reason some people might not be willing to be more open in their teaching and learning might be because of vulnerability. They could be vulnerable in the sense of not having a stable position, or in the sense of having a particular department or school climate that makes it such that opening their teaching could be dangerous to their position (because their colleagues may not agree with what they’re doing, e.g.).

I am fortunate in that neither of these situations applies to me, but that’s a bit of a luxury, and there are many people who don’t have it.

One more thing

I wonder if making my courses more open, in the sense of recording the sessions, would change how I conduct some of my class meetings. A fair number of them are unscripted, experimental forays into topics through (sometimes haphazard) discussion that may or may not come to a clear end point (usually not). I think of these as part of a work-in-progress, a long-term work in which I and the students are moving towards better understanding of certain issues, questions, arguments, texts. Or at least, different understanding that brings up fruitful, new ways of thinking about and approaching these things, showing further dimensions that were hidden before. This work-in-progress may last for a few weeks or months, a few years, or a lifetime. The courses, for me, are just a very small part of this process. In some ways I like that the class meetings are evanescent, short-lived; they aren’t final products in any sense and aren’t meant to be. I wouldn’t want anyone to watch one or two such meetings and get the sense that what I or anyone else says there represents anything more than a provisional test of a thought or argument. It will always change later.

Somehow, recording one’s course sessions seems to me to be making them more permanent, which goes against the way I think of the meetings. I want them to be memories only, things that change when you revisit them, just as the ideas do.

Of course, these issues exist with writing and publishing too–writing is never permanent, and one’s arguments can change radically over the course of a few years. But writing already seems more stable than a class discussion that takes place orally.

Conclusion?

I don’t have one. I just wanted to explore why I might have been reticent to be open, and why others might be. These thoughts on panopticism and sharing things publicly are anything but new, but they may be factors for some.

As with anything, there are benefits and drawbacks to being open in teaching and learning. I think the benefits, in my own personal situation, outweigh the risks of being open (as well as the concern about “permanency” noted above). But that may not be true for everyone, and it may for reasons other than a desire to keep one’s work to oneself, or out of a lack of confidence.

 

Summary of research on modes of peer assessment

I have been doing quite a few “research reviews” of articles on peer assessment–where I summarize the articles and offer comments about them. Lately I’ve been reading articles on different modes of peer assessment: written, oral, online, face to face, etc. And here, I am going to try to put together what that research has said to see if anything can really be concluded about these issues from it.

In what follows, I link to the blog posts discussing each article. Links to the articles themselves can be found at the bottom of this post.

I created PDF tables to compare/contrast the articles under each heading. They end up being pretty small here on the blog, so I also have links to each one of them, below.

Peer feedback via asynchronous, written methods or synchronous, oral, face to face methods

This is the dichotomy I am most interested in: is there a difference when feedback is given asynchronously, in a written form, or when given synchronously, as spoken word face to face? Does the feedback itself differ? Might one form of feedback be more effective than another in terms of being taken up in later revisions of essays?

Do the comments differ in the two modes of peer feedback, and are they used differently in later drafts?

The PDF version of the table below can be downloaded here.

van den Berg, Admiraal and Pilot (2006) looked at differences in what was said in peer feedback on writing assignments when it was written (on standardized peer feedback forms, used for the whole class) and when it was given in oral, face to face discussions. They found that written feedback tended to be more focused on evaluating the essays, saying what was good or bad about them, and less on giving explanations for those evaluative comments or on providing suggestions for revision (though this result differed between the courses they analyzed). In the oral discussions, there was more of a balance between evaluating content, explaining that evaluation, and offering revisions. They also found that both written and oral feedback focused more on content and style than on structure, though there were more comments on structure in the written feedback than in the oral. The authors note, though, that in the courses in which peer feedback took place on early drafts or outlines, there was more feedback on structure than when it took place on later drafts. They conclude: “A combination of written and oral feedback is more profitable than written or oral feedback only” (146).

Hewett (2000) looked at differences in peer feedback between an oral, face to face environment and an electronic, text-based environment. She found that the talk in the oral communication was much more interactive, with students responding to each others’ comments, giving verbal cues that they were following along, and also working together to generate new ideas. The text-based, online feedback was much less like a conversation, with students commenting on the papers at hand but not interacting very much with each other. Perhaps unsurprisingly, then, while the feedback in the written environment was mostly focused on the content of the essay being evaluated, the discussion in the oral environment ranged more widely. Hewett also analyzed essay drafts and peer comments from both environments to see if the peer discussion and comments influenced later drafts of essays. She found that in the oral environment, there was more use in students’ work of ideas that came up in the peer discussion about others’ essays, or that one had oneself said. Hewett concludes that a combination of oral discussion and asynchronous, written comments would be good, using the former for earlier stages of writing–since in oral discussion there can be more talk in which students speculate about wider issues and work together to come up with new ideas–and the latter for revisions focused more on content.

What are students’ views of each mode?

A PDF version of the following table can be downloaded here.

Figl et al. (2006) surveyed students in a computer science course who had engaged in peer assessment of a software project in both the face to face mode as well as through an online, asynchronous system that allows for recording of criticisms as well as adding comments as in a discussion board. There wasn’t a clear preference for one mode over another overall, except in one sense: about half of the students preferred using the face to face mode for discussion within their own teams, and with their partner teams (those they are giving feedback to and receiving feedback from). There was not as much discussion of the feedback, whether within the team or with the partner teams, in the online format, students reported, and they valued the opportunity for that discussion. Figl et al. conclude that it would be best to combine online, asynchronous text reviews with face to face activities, perhaps even with synchronous chat or voice options.

The study reported in Guardardo & Shi 2007 focused on asynchronous, written feedback for the most part; the authors recorded online, discussion-board feedback on essays and compared that with a later draft of each essay. They wanted to know if students used or ignored these peer comments, and what they thought of the experience of receiving the asynchronous, written feedback (they interviewed each student as well). All of the students had engaged in face to face peer feedback before the online mode, but the face to face sessions were not recorded so the nature of the comments in each mode was not compared. Thus, the results from this study that are most relevant to the present concern are those that come from interviews, in which the students compared their experiences of face to face peer feedback with the online, written, asynchronous exchange of feedback. Results were mixed, as noted in the table, but quite a few students said they felt more comfortable giving feedback without their names attached, while a significant number of students preferred the face-to-face mode because it made interacting with the reviewer/reviewee easier. The authors conclude that “online peer feedback is not a simple alternative to face-to-face feedback and needs to be organized carefully to maximize its positive effect” (458).

Cartney 2010 held a focus group of ten first-year students in a social work course who had engaged in a peer feedback exercise in which essays and comments on essays, as well as follow up discussion, was to take place over email. Relevant to the present concern is that the focus group discussion revealed that several groups did not exchange feedback forms via email but decided to meet up in person instead in order to have a more interactive discussion. Some groups did exchange written, asynchronous, online feedback, citing discomfort with giving feedback to others to their “faces.” The author concludes that there may be a need to use more e-learning in curricula in order for students to become more accustomed to using it for dialogue rather than one-way communication. But I also see this as an indication that some students recognized a value in face to face, oral, synchronous communication.

Peer feedback via electronic, synchronous text-based chat vs. oral, face to face methods

This dichotomy contrasts two sorts of synchronous methods for peer feedback and assessment: those taking place online, through text-based systems such as “chats,” and those taking place face to face, orally.

Do comments given synchronously through text-based chats differ from those given orally, face to face? And do these two modes of commenting affect students’ revisions of work differently?

A PDF version of both of the tables below can be downloaded here.

Sullivan & Pratt 1996  looked at two writing classes: in one class all discussions and peer feedback took place through a synchronous, electronic, text-based chat system and in the other discussions and peer feedback took place face to face, orally. They found that writing ability increased slightly more for the computer-assisted class over the traditional class, and that there were differences in how the students spoke to each other in the electronic, text-based chat vs. face to face, orally. The authors stated that the face to face discussion was less focused on the essay being reviewed than in the online chats (but see my criticisms of this interpretation here). They also found that the electronic chats were more egalitarian, in that the author did not dominate the conversation in them in the same way as happened with the face to face chats. The authors conclude (among other things) that discussions through online chats may be beneficial for peer assessments, since their study “showed that students in the computer-assisted class gave more suggestions for revision than students in the oral class” (500), and since there was at least some evidence for greater writing improvement in the “chat” class.

Braine 2001 (I haven’t done an earlier summary of this article in my blog) looked at students in two different types of writing classes in Hong Kong (in English), similar to those discussed in Sullivan & Pratt (1996), in which one class has all discussions and peer assessment taking place orally, and the other has these taking place on a “Local Area Network” that allows for synchronous, electronic, text-based chats. He looked at improvement in writing between a draft of an essay and a revision of that essay (final version) after peer assessment. Braine was testing students’ ability to write in English only, through the “Test of Written English.” He found that students’ English writing ability improved a bit more for the face-to-face class than the computer-mediated class, and that there were significant differences in the nature of discussions in the two modes. He concluded that oral, face-to-face discussions are more effective for peer assessment.

Liu & Sadler 2003  contrasted two modes of peer feedback in two composition classes, one of which wrote comments on essays by hand and engaged in peer feedback orally, face to face, and the other wrote comments on essays digitally, through MS Word, and then engaged in peer discussion through an electronic, synchronous, text-based chat during class time. The authors asked about differences in these  modes of commenting, and whether they had a differential impact on later essay revisions. Liu & Sadler were not focused on comparing the asynchronous commenting modes with the synchronous ones, but their results show that there was a higher percentage of “global” comments in both of the synchronous modes, and a higher percentage of “local” comments in the asynchronous ones. They also found that there was a significantly higher percentage of “revision-oriented” comments in the oral discussion than in the electronic chat. Finally, students acted more often on the revision-oriented comments given in the “traditional” mode (handwritten, asynchronous comments plus oral discussion) than in the computer-mediated mode (digital, asynchronous comments plus electronic, text-based chat). They conclude that for asynchronous modes of commenting, using digital tools is more effective than handwriting (for reasons not discussed here), and for synchronous modes of commenting, face to face discussions are more effective than text-based, electronic chats (219-221). They suggest combining these two methods for peer assessment.

Jones et al 2006  studied interactions between peer tutors in an English writing centre in Hong Kong and their clients, both in face to face meetings and in online, text-based chats. This is different from the other studies, which were looking more directly at peer assessment in courses, but the results here may be relevant to what we usually think of as peer assessment. The authors were looking at interactional dynamics between tutors and clients, and found that in the face-to-face mode, the relationship between tutors and clients tended to be more hierarchical than in the electronic, online chat mode. They also found that the subjects of discussion were different between the two modes: the face-to-face mode was used most often for “text-based” issues, such as grammar and word choice, while in the electronic chats the tutors and clients spoke more about wider issues such as content of essays and process of writing. They conclude that since the two modes differ and both serve important purposes, it would be best to use both modes.

Implications/discussion

This set of studies is not the result of a systematic review of the literature; I did not follow up on all the other studies that cited these, for example. A systematic review of the literature might add more studies to the mix. In addition, there are more variables that should be considered (e.g., whether the students in the study underwent peer assessment training, how much/what kind; whether peer assessment was done using a standardized sheet or not in each study, and more).

Nevertheless, I would like to consider briefly if these studies provide any clarity for direction regarding written peer assessment vs. oral, face-to-face.

For written, asynchronous modes of peer assessment (e.g., writing on essays themselves, writing on peer assessment forms) vs. oral, face-to-face modes, the studies noted here (van den Berg, Admiraal and Pilot (2006) and Hewett (2000)) suggest that in these two modes students give different sorts of comments, and for a fuller picture peer assessment should probably be conducted in both modes. Regarding student views of both modes (Figl et al. (2006), Guardardo & Shi (2007), Cartney (2010)), evidence is mixed, but there are at least a significant number of students who prefer face-to-face, oral discussions if they have to choose between those and asynchronous, written peer assessment.

For written, synchronous modes of peer assessment (e.g., electronic, text-based chats) vs. oral, face-to-face, the evidence here is all from students for whom English is a foreign language, but some of the results might still be applicable to other students (to determine this would require further discussion than I can engage in now). All that can be said here is that the results are mixed. Sullivan & Pratt (1996) found some, but not a lot of evidence that students using e-chats improved their writing more than those using oral peer assessment, but Braine (2001) found the opposite. However, they were using different measures of writing quality. Sullivan & Pratt also concluded that the face-to-face discussions were less focused and effective than the e-chat discussions, while Braine concluded the opposite. This probably comes down in part to interpretation of what “focused” and “effective” mean.

Liu & Sadler (2003) argued that face-to-face modes of synchronous discussion are better than text-based, electronic, synchronous chats–opposing Sullivan & Pratt–because there was a higher percentage of “revision-oriented” conversational turns (as a % of total turns) in the face-to-face mode, and because students acted on the revision-oriented comments more in the traditional class (both writing comments on paper and oral, face-to-face peer discussion) than in the computer-mediated class (digital comments in MS Word and e-chat discussions). Jones et al. (2006) found that students and peer tutors talked about different types of things, generally, in the two modes and thus concluded that both should be used. But that study was about peer tutors and clients, which is a different situation than peer assessment in courses.

So really, little can be concluded, I think, from looking at all these studies, except that it does seem that students tend to say different types of things in different modes of communication (written/asynchronous, written/synchronous, oral/face-to-face/synchronous), and that those things are all valuable; so perhaps what we can say is that using a combination of modes is probably best.

Gaps in the literature

Besides more studies to see if better patterns can emerge (and perhaps they are out there–as noted above, my literature search has not been systematic), one gap is that no one, so far, has considered video chats, such as Google Hangouts, for peer assessment. Perhaps the differences between those and face-to-face meetings might not be as great as between face-to-face meetings and text-based modes (whether synchronous chats or asynchronous, written comments). And this sort of evidence might be useful for courses that are distributed geographically, so students could have a kind of face-to-face peer assessment interaction rather than just giving each other written comments and carrying on a discussion over email or an online discussion board. Of course, the problem there would be that face-to-face interactions are best if supervised, even indirectly, so as to reduce the risk of people treating each other disrespectfully, or offering criticisms that are not constructive.

So, after all this work, I’ve found what I had guessed before starting: it’s probably best to use both written, asynchronous comments and oral, face-to-face comments for peer assessment.

 

Works Cited

Braine, G. (2001) A study of English as a foreign language (EFL) writers on a local-area network (LAN) and in traditional classes, Computers and Composition 18,  275–292. DOI: http://dx.doi.org/10.1016/S8755-4615(01)00056-1

Cartney, P. (2010) Exploring the use of peer assessment as a vehicle for closing the gap between feedback given and feedback used, Assessment & Evaluation in Higher Education, 35:5, 551-564. DOI: http://dx.doi.org/10.1080/02602931003632381

Figl, K., Bauer, C., Mangler, J., Motschnig, R. (2006) Online versus Face-to-Face Peer Team Reviews, Proceedings of Frontiers in Education Conference (FIE). San Diego: IEEE. See here for online version (behind a paywall).

Guardado, M., Shi, L. (2007) ESL students’ experiences of online peer feedback, Computers and Composition 24, 443–461. Doi: http://dx.doi.org/10.1016/j.compcom.2007.03.002

Hewett, B. (2000) Characteristics of Interactive Oral and Computer-Mediated Peer Group Talk and Its Influence on Revision, Computers and Composition 17, 265-288. DOI: http://dx.doi.org/10.1016/S8755-4615(00)00035-9

Jones, R.H., Garralda, A., Li, D.C.S. & Lock, G. (2006) Interactional dynamics in on-line and face-to-face peer-tutoring sessions for second language writers, Journal of Second Language Writing 15,  1–23. DOI: http://dx.doi.org/10.1016/j.jslw.2005.12.001

Liu, J. & Sadler, R.W. (2003) The effect and affect of peer review in electronic versus traditional modes on L2 writing, Journal of English for Academic Purposes 2, 193–227. DOI: http://dx.doi.org/10.1016/S1475-1585(03)00025-0

Sullivan, S. & Pratt, E. (1996) A comparative study of two ESL writing environments: A computer-assisted classroom and a traditional oral classroom, System 29, 491-501. DOI: http://dx.doi.org/10.1016/S0346-251X(96)00044-9

Van den Berg, I., Admiraal, W.,  & Pilot, A. (2006) Designing student peer assessment in higher education: analysis of written and oral peer feedback, Teaching in Higher Education, 11:2, 135-147.  DOI: http://dx.doi.org/10.1080/13562510500527685

Peer Assessment: Face to face vs. online, synchronous (Part 2)

Here I look at one last study I’ve found that focuses on the nature of student peer feedback discussions when they take place in a synchronous, online environment (a text-based chat). Part 1 corresponding to this post can be found here.

Jones, R.H., Garralda, A., Li, D.C.S. & Lock, G. (2006) Interactional dynamics in on-line and face-to-face peer-tutoring sessions for second language writers, Journal of Second Language Writing 15,  1–23. DOI: http://dx.doi.org/10.1016/j.jslw.2005.12.001

This study is rather different than the ones I looked at in Part 1 of face to face vs. online, synchronous peer assessment, because here the subjects of the study are students and peer tutors in a writing centre rather than peers in the same course. Still, at least some of their results regarding the nature of peer talk in the tutor situation may still be relevant for peer assessment in courses.

The Tutoring Centre, by Tulane Public Relations, on Flickr. CC-By licensed.

Participants and data

The participants in this study were five peer tutors in a writing centre in Hong Kong, dedicated to helping non-native English speakers write in English. For both tutors and clients, English was an additional language, but the tutors were further along in their English studies and had more proficiency in writing in English than the clients. Data was collected from transcripts of face to face consultations of the tutors with clients, as well as transcripts of online, text-based chat sessions of the same tutors, with many of the same clients.

Face to face tutoring was only available in the daytime on weekdays, so if students wanted help after hours, they could turn to the online chat. Face to face sessions lasted between 15 and 30 minutes, and students “usually” emailed a draft of their work to the tutor before the session. Chat sessions could be anywhere from a few minutes to an hour, and though tutors and clients could send files to each other through a file exchange system, this was only done “sometimes” (6). These details will become important later.

Model for analyzing speech

To analyze the interactions between tutors and clients, the authors used a model based on “Halliday’s functional-semantic view of dialogue (Eggins & Slade, 1997; Halliday, 1994)” (4). In this model, one analyzes conversational “moves,” which are different than “turns”–a “turn” can have more than one “move.” The authors explain a move as “a discourse unit that represents the realization of a speech function” (4).

In their model, the authors use a fundamental distinction given by Halliday into “initiating moves” and “responding moves”:

Initiating moves (statements, offers, questions, and commands) are those taken independently of an initiating move by the other party; responding moves (such as acts of acknowledgement, agreement, compliance, acceptance, and answering) are those taken in response to an initiating move by the other party. (4-5)

They then subdivide these two categories further, some of which is discussed briefly below.

Results

Conversational control

In the face to face meetings, the tutors exerted the most control over the discussions. Tutors had many more initiating moves (around 40% of their total moves, vs. around 10% of those for clients), whereas clients had more responding moves (around 33% of clients’ total moves, vs. about 14% for tutors). In the chat conversations, on the other hand, initiating and responding moves were about equal for both tutors and clients (7).

Looking more closely at the initiating moves made by both tutors and clients, the authors report:

In face-to-face meetings, tutors controlled conversations primarily by asking questions, making statements, and issuing directives. In this mode tutors asked four times more questions than clients. In the on-line mode, clients asked more questions than tutors, made significantly more statements than in the face-to-face mode, and issued just as many directives as tutors. (10)

Types of questions

However, the authors also point out that even though the clients asserted more conversational control in the online chats, it was “typical” of the chats to consist of questions by students asking whether phrases, words, or sentences were “correct” (11). They did not often ask for explanations, just a kind of check of their work from an expert and a quick answer as to whether something was right or wrong. On the other hand, when tutors controlled the conversations with their questions, it was often the case that they were using strategies to try to get clients to understand something themselves, to understand why something is right or wrong and to be able to apply that later. So “control” over the conversation, and who asks the most questions or issues the most directives, are not the only important considerations here.

The authors also divided the questions into three different types. Closed questions: “those eliciting yes/no responses or giving the answerer a finite number of choices; open questions: “those eliciting more extended replies”; rhetorical questions: “those which are not meant to elicit a response at all” (12)

In the face to face sessions, tutors used more closed questions (about 50% of their initiating questions) than open questions (about 33%); the opposite was true in the online chats: tutors used more open questions (about 50% of their initiating questions) than closed (about 41%).

Continue reading →

MOOCs and humanities, revisited

In the last post I discussed how I have come to learn about the different kinds of MOOCs through my participation in etmooc. I also said that through learning about a new kind of MOOC, the cMOOC or “network-based” MOOC, I was reconsidering my earlier concerns with MOOCs. Might the cMOOC do better for humanities than the xMOOC?

A humanities cMOOC

“Roman Ondák”, cc licensed ( BY ) flickr photo shared by Marc Wathieu

I haven’t yet decided whether or not one could do a full humanities course, such as a philosophy course, through a cMOOC structure. Brainstorming a little, though, I suppose that one could have a philosophy course in which:

  • Common readings are assigned
  • Presentations are given by course facilitators and/or guests, just as in etmooc
  • Participants are encouraged to blog about the readings and presentations and comment on each others’ blogs (through a course blog hub, like etmooc and ds106 have)
  • Dedicated Twitter hashtag, plus a group on a social network like Google+, and a group on a social bookmarking site like Diigo (see etmooc’s group site on Diigo)
  • Possibly a YouTube channel, for people to do vlogs instead of blogs if they want, or share other videos relevant to the course

Would this sort of structure be more likely to allow for teaching and practice of critical thinking, reading and writing skills, as I discussed in my earlier criticism of MOOCs (which was pretty much a criticism of xMOOCs)? I suppose it depends on what is discussed in the presentations, in part. The instructors/facilitators could model critical reading and thinking, through explaining how they are interpreting texts and pointing out potential criticisms with the arguments. They could talk about recognizing, criticizing, and creating arguments so that participants could be encouraged to present their own arguments in blogs as clearly and strongly as possible, as well as offering constructive criticisms of works being read–as well as each others’ arguments (though the latter has to be undertaken carefully, just as it is in a face to face course).

This would involve, effectively, peer feedback on participants’ written work. Rough guidelines for blog posts (at least some of them) could be given, so that in addition to reflective pieces (which are very important!) there could also be some blog posts that are focused on criticizing arguments in the texts, some on creating one’s own arguments about what’s being discussed, etc.

What you wouldn’t be able to do well with this structure are writing assignments in the form of argumentative essays. These take a long time to learn how to do well, and ideally should have more direct instructor/facilitator feedback rather than only peer feedback, in my view. Peer feedback is important too, but could lead to problems being perpetuated if the participants in a peer group share misconceptions.

Another thing you can’t do well with a cMOOC is require that everyone learn and be assessed on a particular set of facts, or content. A cMOOC is better for creating connections between people so that they can pursue their own interests, what they want to focus on. Each person’s path through a cMOOC can be very different. Thus, as noted in my previous post, there is not a common set of learning objectives; rather, participants decide what they want to get out of the course and focus on that.

One would need to have a certain critical mass of dedicated and engaged participants for this to work. If it’s a free and open course, then people will participate when they can, and can flit in and out of the topics as their time and interest allows. That’s fantastic, I think, though if there are few participants that might mean that for some sections of the course little is happening. So having a decent sized participant base is important. (How many? No idea.)

I envision this sort of possibility as a non-credit course for people who want to learn something about philosophy and discuss it with others. Why not give credit? There would have to be more focus on content and/or more formal assessments, I think (at least in the current climate of higher education).

A cMOOC as supplement to an on-campus course

Even if a full cMOOC course in philosophy or another humanities subject may not work, I can see a kind of cMOOC component to philosophy courses, or Arts One. In addition to the campus-based, in-person course, one could have an open course going alongside it. This is what ds106 is like. One could have readings and lectures posted online (or at least, links to buy the books if the readings aren’t readily available online), and then have a platform for students who are off campus to engage in a cMOOC kind of way.

Then, those off campus can participate in the course through their blog posts and discussions/resource sharing on the other platforms, like we do in etmooc. Discussion questions used in class could be posted for all online participants.  Students who are on campus could be blogging and tweeting and discussing with others outside the course as well as inside the course.

Frankenstein engraved

Frontispiece to Mary Shelley’s Frankenstein (1831),by Theodor von Holst [Public domain], via Wikimedia Commons. One of the texts on Arts One Digital.

Discussions would expand to include many more people with many more backgrounds and things to contribute, which is likely to enrich the learning experience. There might get to be too much for each individual to follow, but then one just has to learn to pick and choose what to read and comment on (more on this, below). All participants could make connections and continue discussions beyond the course itself.

Arts One has already started to move in this direction, with a new initiative called Arts One Digital. So far, there are some lectures posted, links to some online versions of texts, twitter feed, and blog posts. This is a work in progress, and we’re still figuring out where it should go. I think extending the Arts One course in the way described above might be a good idea.

Again, the main problem with this idea (beyond the fact that yes, it will require more personnel to design and run the off-campus version of the course) is getting a high number of participants. It won’t work well if there aren’t very many people involved–a critical mass is needed to allow people to find others they want to connect with in smaller groups, to engage in deeper discussions, to help build their own personal learning network.

Looking back at previous concerns with (x)MOOCs

Besides general worried about their ability to help students develop critical skills, I was also concerned in my earlier post with the following:

  • In the Coursera Course on reasoning and argumentation (“Think Again”) that I sat in on briefly, I found myself getting utterly overwhelmed by the number of things posted in the discussion board. I complained that I could scroll and scroll just to get through the comments on one post, to get down to the next post, and repeat for each of the thousands of posts. Even for one topic there were just too many posts.
  • I felt that the asynchronous discussion opportunities weren’t as good as synchronous ones, which allow for groups to be in the same mind space at the same time, feeding off each others’ ideas and coming up with new ideas. With asynchronous discussions, one might not get a response to one’s idea or comment until long after one has been actively thinking about it, and then at that point one may not be as interested in discussing it anymore (or at the very least, the enthusiasm level may be different).
  • The synchronous option of Google Hangouts seems to be a promising way to address the previous point, but I noted in my earlier post that there had been some reports of disrespectful behaviour in one or two of those in the “Think Again” course. I said I thought a moderator would be needed for such discussions, just as we have in face to face courses to ensure students treat each other respectfully.

Can a cMOOC address these concerns?

  1. From my experience with etmooc, the discussion does not have to get overwhelming. The thing is, each person focuses on what they want to focus on from the presentations, or from what others have said in their blogs, or from resources shared by others. There is no single “curriculum” that we all have to follow, so it’s not the case that everything posted by each person is relevant to everyone else’s interests and purposes for the course. This could be true of a philosophy or Arts One cMOOC as well, so it could be easier to pick and choose what, amongst the huge stream of things to read and think about, one wants to focus on.
  2. Synchronous discussions are difficult in a large group. In etmooc we have some opportunities for them in the presentations, which allow for people to write on the whiteboard, engage in a backchannel “chat,” and also take the mic and ask questions/offer comments. One could have the presentations have more time for discussion, perhaps, which could take place in part on the chat and in part via audio. It’s not as good as face to face discussions, though–much more fragmented.
  3. Google Hangouts are an alternative, though I haven’t tried doing one in etmooc. Some have, though, and reported success. However, the people taking etmooc are mostly professionals, both teachers and businesspeople, and they are both highly motivated and responsible/respectful. Having Google Hangouts where anyone in the world can show up could be inviting trouble. I don’t see a cMOOC addressing this problem.

cMOOCs in humanities–what’s not to love?

What other problems might there be with trying to do a cMOOC in humanities, whether on its own or as a supplement to another course? Or, do you love the idea? Let us know in the comments.

UPDATE: I just found, in that wonderfully synergistic way that etmooc seems to work, this blog post by Joe Dillon, which explains how well a cMOOC like etmooc stacks up to a face to face course. It’s just one example, but it can provoke some further thought on whether a cMOOC for humanities might be a good thing.