As noted in an earlier post, I have submitted some proposals for conference presentations on researching the effectiveness of connectivist MOOCs, or cMOOCs (see another one of my earlier posts for what a cMOOC is). I am using this post (and one or two later ones) to try to work through how one might go about doing so, and the problems I’ve considered only in a somewhat general way previously. I need to think things through by writing, so why not do that in the open?
I had wanted to think more carefully about connectivism before moving to some research questions about connectivist MOOCs, but for various reasons I need to get something worked out about possible research questions as soon as I can, so I’ll return to looking at connectivism in later posts.
Photo Credit: Detlef Schobert via Compfight CC-BY-ND
The general topic I’m interested in (at the moment)
And I mean general. I want to know whether we can determine whether a cMOOC has been “effective” or “successful.” That’s so general as to mean almost nothing.
What might help is some specification of the purposes or goals of offering a particular cMOOC, so one could see if it has been effective in achieving those. This could be taken from any of a number of perspectives, such as:
- If an institution is offering a cMOOC, what is the institution’s purpose in doing so? This is not something I’m terribly interested in at the moment.
- What do those who are designing/planning/facilitating the cMOOC hope to get out if doing so, for themselves? This is also not what I’m particularly interested in for a research project.
- What do those who are designing/planning/facilitating the cMOOC hope participants will get out if it? There are likely some reasons, articulated or not, why the designers thought a cMOOC would be effective for participants in some way, thus they decided to offer a cMOOC at all. This is closer to what I’m interested in, but there’s a complication.
The connectivist MOOC model as implemented so far by people such as Dave Cormier, Alec Couros, Stephen Downes and George Siemens encourages participants to set their own goals and purposes for participation, rather than determining what these are to be for all participants (see, e.g., McAuley, Stewart, Siemens, & Cormier, 2010 (pp. 4-5, 40); see The MOOC Guide for a history of cMOOC-type courses, and lists of more recent connectivist MOOCs here and here). As Stephen Downes puts it:
In the MOOCs we’ve offered, we have said very clearly that you (as a student) define what counts as success. There is no single metric, because people go into the course for many different purposes. That’s why we see many different levels of activity ….
Further, just what a cMOOC will be like, where it goes, what people talk about, depends largely on the participants–even though there are often pre-set topics and speakers in advance, the rest of what happens is mostly up to what is written, discussed, shared amongst the participants. The ETMOOC guide for participants emphasizes this:
What #etmooc eventually becomes, and what it will mean to you, will depend upon the ways in which you participate and the participation and activities of all of its members.
Thus, it’s hard to say in advance what participants might get out of a particular cMOOC, in part because it’s impossible to say in advance what the course will actually be like (beyond the scheduled presentations, which are only one of many parts of a cMOOC).
Some possible directions for research questions
Developing connections with other people
I at first thought that perhaps one could say cMOOCs should allow participants to, at the very least, develop a set of connections with other people that are used for sharing advice, information, comments on each others’ work, for collaborating, and more. As discussed in my blog post on George Siemens’ writings on connectivism, what may be most important to a course that is run on connectivist principles is not the content that is provided, but the fostering of connections and skills for developing new ones and maintaining those one has, for the sake of being able to learn continually into the future.
And even though I understand what Downes and others say about participants in cMOOCs determining their own goals and deciding for themselves whether the course has been a success, cMOOCs have been and continue to be designed in certain ways for certain reasons, at least some of which most likely has to do with what participants may get out of the courses. Some of those who have been involved in designing cMOOCs have emphasized the importance of forming connections between people, ideas and information.
Stephen Downes talks about this in “Creating the Connectivist Course” when he says that he and George Siemens tried to make the “Connectivism and Connective Knowledge” course in 2008 “as much like a network as possible.” In this video on how to succeed in a MOOC, Dave Cormier emphasizes the value of connecting with others in the course through commenting on their blog posts, participating in discussion fora, and other ways. The connections made in this way are, Cormier says, “what the course is all about.” Now, of course, Cormier states at the beginning and end of the video that MOOCs are open to different ways of success and this is just “his” way, but the tone of the video suggests that it would be useful for others as well. Cormier says something similar in this video on knowledge in a MOOC: participants in a MOOC “are [ideally?] going to come out with a knowledge network, a network of people and ideas that’s going to carry long past the end of [the] course date.”
So it made sense to me at first to consider asking about the effectiveness or success of a cMOOC through looking at whether and how participants made connections with each other, and especially whether those continue beyond the end of the course. But again, there are some complications, besides the important questions of just how to define “connections” so as to decide what data to gather, and then the technical issues regarding how to get that data.
Would we want to say that the course succeeded more if more people made connections to others, rather than less? Or how about the question of how many people each participant should ideally connect with–I don’t think more is necessarily better, but where do we draw the line to say that x number of people made y number of connections with others, so the course has been a success?
This is getting pedantic, but I’m trying to express the point that when you really dig into this kind of question and try to design a research project, you would have to address this kind of question, and it’s kind of ridiculous. It’s ridiculous because there are so many different ways that connecting with other people could be valuable for a person, and for one person, having made one connection ends up being much more valuable than for another who has made 50. So much depends on the nature and context of those connections, and those are going to be highly individual and likely impossible to specify in any general way.
Further, what if some participants are happy to watch a few presentations and read blogs and lurk in twitter chats but don’t participate and therefore don’t “connect” in a deeper sense (than just reading and listening to others’ work and words). Should we say that if there are a lot of such persons in a cMOOC, the course has not been successful? I don’t think so, if we’re really sticking to the idea that participants can be engaged in the course to the degree and for the reasons they wish.
One possibility would be to ask participants to reflect on the connections they’ve made and whether/why/how they are valuable. One might be able to get some kind of useful qualitative data out of this, and maybe even find some patterns to what allows for valuable connections. In other words, rather than decide in advance what sorts of connections, and how many, are required for a successful cMOOC, one could just gather data about what connections were made and why/how people found them valuable. If done over lots of cMOOCs, one might be able to devise some sort of general idea of what makes for valuable connections in cMOOCs.
But would it be possible to say, on the basis of such data, whether a particular cMOOC has been successful? If many people made some connections they found valuable, would that be more successful than if only a few did? Again, this leads to the problems noted above–it runs up against the point that in cMOOCs participants are free to act and participate how they wish, and if they wish not to make connections, that doesn’t necessarily have to mean the course hasn’t been “successful” for them.
Looking at participation rates
photo credit: danielmoyle via photopin CC-BY
One might consider looking at participation rates in a cMOOC, given that much of such a course involves discussions and sharing of resources amongst participants (rather than transferral of knowledge mainly from one or a few experts to participants). As this video by Dave Cormier demonstrates so well, cMOOCs are distributed on the web rather than taking place in one central “space” (though there may be a central hub where people go for easy access to such distributed information and discussions, such as a blog hub), and this means that a large part of the course is happening on people’s blogs, on Twitter, on lists of shared links, and elsewhere. So it would seem reasonable to consider the degree to which participants engage in discussions through these means. How many people are active in the sense of writing blog posts, commenting on others’ blog posts, participating in Twitter chats and posting things to the course Twitter hashtag, participating in discussion forums (if there are any; there were none in ETMOOC) or in social media spaces like Google+, etc?
This makes sense given the nature of cMOOCs, since if there were no participation in these ways then there would be little left of the course but a set of presentations by experts that could be downloaded and watched. Perhaps one could say that even if we can’t decide exactly how much participation (or connection, for that matter) is needed for “success,” an increase in participation (or connection) over time might indicate some degree of success.
But again, we run up against the emphasis on participants being encouraged to participate only when, where and how they wish, meaning that it’s hard to justify saying that a cMOOC with greater participation amongst a larger number of people was somehow more effective than one in which fewer people participated. Or that a cMOOC in which participation and connections increased over time was more successful than one in which these stayed the same or decreased (especially since the evidence I’ve seen so far suggests that a drop off in participation over time may be common).
Determining your own purposes for participating in a cMOOC and judging whether you’ve reached them
Another option could be to ask participants who agree to be part of the research project to state early on what their goals for participating in the cMOOC are, and then towards end, and even in the middle, perhaps, ask them to reflect on whether they’re meeting/have met them.
Sounds reasonable, but then there are those people–like me taking ETMOOC–who don’t have a clear set of goals for taking an open online course. I honestly didn’t know exactly what I was getting into, nor what I wanted to get out of it because I didn’t understand what would happen in it. And as noted above, even though there may be some predetermined topics and presentations, what you end up focusing on/writing about/commenting on in discussion forums or others’ blogs/Tweeting about develops over time, as the course progresses. So some people may recognize this and be open to whatever transpires, not having any clear goals in advance or even partway through.
For those who do set out some goals for themselves at the beginning, it could easily be the case that many don’t end up fulfilling those particular goals by the end, but going in a different direction than what they could have envisioned at the beginning. In fact, one might even argue that that would be ideal–that people end up going into very different directions than they could have imagined to begin with might mean that the course was transformative for them in some way.
Thus, again, it’s difficult to see just how to make an argument about the effectiveness of a cMOOC by asking participants to set their goals out in advance and reflect on whether or not they’ve met them. Perhaps we could leave this open to people not having any goals but being able to reflect later on what they’ve gotten out of the course, and open to those who end up not meeting their original goals but go off in other valuable directions.
This would mean gathering qualitative data from things such as surveys, interviews or focus groups. I think it would be good to ask people to reflect on this partway through the course, at the end of the course, and again a few months or even a year later. Sometimes what people “get out of” a course doesn’t really crystallize for them until long after it’s finished.
Conclusions so far
It seems to me that there is a tension between the desire to have a course built in large part on the participation of individuals involved, and the desire to let them choose their level and type of participation. In some senses, cMOOCs appear to promote greater participation and connections amongst those involved, while also backing away from this at the same time. I understand the latter, and I appreciate it myself–that was one of the things that made ETMOOC so valuable for me. I was encouraged to choose what to focus on, what to write about, which conversations to participate in, based on what I found most important for my purposes (and based on how much time I had!). There are potential downsides to this, though, in that participants may not move far beyond their current beliefs, values and interests if they just look at what they find important based on those. But overall, I see the point and the value. I expect there are some good arguments in the educational literature for this sort of strategy that I’m not aware of.
Still, this is in tension, to some degree, with the emphasis on connecting and participating in cMOOCs. Perhaps the idea is that it would be good for people to do some connecting and participating, but in their own ways and on their own time, and if they choose not to we shouldn’t say they are not doing the course “correctly.” It might nevertheless be possible/permissible to suggest that, given the other side of this “tension,” looking at participation or connection rates could be considered as part of looking at the success of a cMOOC? Honestly, I’m torn here.
[Update June 7, 2013] I just came across this post by George Siemens, in which he doubts the value of lurking, at least in a personal learning network (PLN). There are likely differences of opinion amongst cMOOC proponents and those who offer them, on the value of letting learners decide exactly how much to participate.
It is, of course, possible that the whole approach I’m taking is misguided, namely trying to determine how one measure whether a cMOOC has been successful or not. I’m open to that possibility, but haven’t given up yet–not until I explore other avenues.
I had one other section to this post, but as it is already quite long, I moved that section to a new post, in which I discuss a suggestion by Stephen Downes as to how to evaluate the “success” of MOOCs. In that and/or perhaps another post I will also discuss some of the published literature so far on cMOOCs, and what the research questions and methods were in those studies.
Please comment/question/criticize as you see fit. As you can tell, I’m in early stages here and am happy for any help I can get.
Hi!
So cool that you have been awarded badges! And on the good side of ‘Talky Tina’ too! You’ve been busy.
Just a note about research areas from my perspective at KPU. Admin. is decidedly not in favor of a cMooc idea for faculty PD, and it seems they have found other more centralized solutions in an institutional push for more online offerings. My own motivation for involvement in #OOE has been to be a mentor to KPU colleagues as I have been in the area of edtech applications for the past 8 years or so. I really want to share the excitement I have found in the open space communities we have formed since #etmooc and I know that if I could get colleagues involved in #OOE, the exposure could change the logjam we seem to be in within our department. I would like to be able to document and analyze the effectiveness of #OOE on a cohort of KPU faculty members, but I don’t know whether I will get the chance to. A few of us who teach writing at KPU would love to see a ‘culture’ change in terms of how writing is taught in higher ed, and I think this could be a worthy investigation as well-studying the communication and writing patterns of educators in programs like etmooc or ooe. Personally, I am not on a research /publication track as faculty, but I have a strong desire to add value to my department through my professional activities. The REB requirements and FOIPPA laws seem to be stumbling blocks particular to BC and may inhibit my research activities in OOE. I am also wondering now about the ‘presence’ of so many researchers in OOE-do you think participants will feel as though ‘Talky Tina’ might be lurking somewhere?
Hi Janet:
Sorry to hear that KPU is not in favour of cMOOC idea for Faculty PD; was hoping we could get a higher ed focus in it. But of course, I’m not trying at my institution either–just too hard from afar. Let’s see how it goes more informally and then maybe things will shift if all goes well. I share your concerns about there possibly being too many researchers in OOE, stalking participants. I’m not sure what can be done about that, short of some of us backing off of researching it, which I might be willing to do (especially since I’m having so much trouble coming up with a good research question!).
I am doctoral student in Instructional Technology who is looking to research factors that affect MOOC retention and persistence. This is simpler with xMOOCs, where there is a defined completion goal, but even they are looking at intention as a primary factor in completion (like the recent research published from Coursera). I am examining many of the concepts you mentioned, like making connections with other learners and personal goals for starting the MOOC. I think there are important lessons from social network analysis for determining who was most engaged in the network, but that’s another issue.
In the meantime, I think all MOOCs, even cMOOCs, should have some sort of a stated goal for offering it. Otherwise, why put the effort into it? Maybe the goal is to raise awareness about a topic, start a discussion, or build a network of interested individuals – these are primarily affective, but definitely measurable. Success would mean reaching that goal.
I also think that cMOOCs that have set timelines have an inherent goal of keeping their network engaged for the duration. That “engagement” could be at multiple levels, but I think that is acceptable, as well. I really like a study conducted through MIT on how engagement patterns changed over time: http://eddinit.wordpress.com/2013/05/01/deconstructing-disengagement/ (okay – shameless self-promotion since that is my review of the article, sorry!)
I will definitely be following your research as it develops. It is thrilling to see how MOOC research will develop over time, and I would love to see a solid methodology for researching cMOOCs in particular. Thanks!
Hi Stephanie:
Thanks for the comment! I agree that there must be some goal for the cMOOC that drives why it’s being offered. But that goal could be (and just looking at goals in terms of students’ experience, rather than the goals of an institution or the individuals offering it for what it could do for them) just providing an opportunity for participants to connect, blog, discuss if and how much they want to. So the hope might be that many people will want to, and clearly there will have to be some critical mass of participants who do or else the course won’t go very well, still, the people offering it might want to leave participants free to choose not to connect or write very much in blogs, or simply “lurk” if they choose. I get the sense that there is that sort of combined goal with many cMOOCs: we’d like you to participate and connect a fair bit, but we’re leaving it up to you. In which case, it’s hard to determine what to measure. The freedom that people had to choose how/when to participate? If they had a lot of freedom, has it met its goal?
If it were just a matter of raising awareness of a topic, then this could be done without a “connectivist” aspect to a MOOC, I’m thinking. So while that may be part of the goal(s) of some cMOOCs, I don’t think it’s sufficient to justify why one would choose the cMOOC structure. There does seem to be an emphasis on connecting, of course, but the emphasis on freedom to participate how you want mitigates that to some degree.
Thank you for the summary of the article by Kizilcec, Piech, & Schneider. I recall hearing about this, but hadn’t thought about it very carefully. There are certainly those kinds of levels of engagement in cMOOCs as well, I think (judging from my experience & my reading so far–I’ll be discussing some of the cMOOC research in a later post!). The only problem is that there seems to be a value judgment in those four categories, in the sense that the first two are better than the last two. That makes sense for a traditional type of course, and many xMOOCs are like that. But for many cMOOCs, like the ETMOOC course I took, there was absolutely nothing wrong with “disengaging”: with doing a lot in one section and then not much in later ones. Nor with starting in the middle, doing a lot for a few weeks, and then dropping off. The course was divided into two-week topics in part precisely so that people could come in for one or more of them if they wanted, and they wouldn’t have to feel they should have done the earlier topics to do so. Some of us stayed the whole time, and yes, we made stronger connections to each other that way, but there was no push for people to do so, no sense that they were somehow not doing the course right if they didn’t stay for all the topics.
So again, I’m still struggling with just what one might use to say a cMOOC had been effective! I still think there’s something to participation, engagement, connection over time because again, if at least some people aren’t doing that, there really isn’t a course at all. But I’m not sure what to do with that, exactly.
Thanks for helping me to think through some of this a bit more!
Hi Christina,
Interesting topic! How to measure a cmooc’s success will be a challenging task. My first question is “Do we have to measure it? Why measure?” since all measurement is a human construction: a value judgement. The strength of the cmooc from the point of view of the learner is the ability to take what you want from it as a learner and leave. The commitment to the course is determined by the learner, the activities to accomplish are chosen by the participant. It truly is learning decided by the learner- rather like a grown up virtual democratic school. The value judgment or measure of success comes from the participant themselves, rather than the organizers of the learning experience. So from my perspective the questions should be: “Did you play in this space? And did you find joy/fun in the learning? How has this changed you/your practice? Will you pass this knowledge on?”
How do you measure individual internal change? We saw that change expressed in a number of reflections from ETMOOC for example. Is that the measure of success?
Karen
Good points, Karen. In part my motivation is pragmatic. If the cMOOC format (or something similar) can be shown to be “successful” in some way that a university would accept, then I could possibly get more support for offering a cMOOC extension of a course. Otherwise I’m doing it all on my own (assuming no one says I can’t do it at all, which I doubt…would that even be possible?), which is okay, but more time consuming if I have to learn how to do all the technical stuff by myself, etc. Partly it’s also to get other faculty to the point where they might take one as Prof Development, and the universities to see that such a move should could as valid PD.
But partly, I also want to be able to see if certain forms work better than others, so as to know what form to use when I consider offering my own. Do I go with the ETMOOC format, or is there something about that sort of format that doesn’t quite work and something needs to be changed, for example? So I do think there are valid reasons for asking the kinds of questions I’m asking, even though I am completely open to the idea that I may be asking them the wrong way–that’s why I’m putting all this out into the open so I can get feedback!
I do agree that the decision of success comes from the learners themselves (insofar as one is looking at “success” from the perspective of what it does for learners rather than facilitators, which I am). And asking for participants to reflect on the experience and what they’ve gotten from it, how they’ve changed (if at all) is an important part of this sort of project. Perhaps it should be the whole of this sort of project, in that maybe one should only ask for learners’ subjective evaluations and reflections.
But I also wonder: if part of the whole point of cMOOCs is to promote connections amongst people for the sake of continuing to learn and collaborate beyond the course, why not look at how much this is happening or not? Participants themselves may not even recognize the extent to which they’re making such connections during or even right after the course, though maybe they would if you asked them 6 months later or something.
I started to realize last night, though, that I may be going about this backwards. What I really need to do is think more carefully about the reasons why I would offer an open online course, or an open online extension of an on-campus course, and then see if the cMOOC model seems to fit those reasons or not. Is there some way I might want to change the current cMOOC model? Then perhaps this would help provide some guidance on what the model is supposed to do for learners and then I could see whether it worked in that sense or not.
But still, the thing is, I could try to decide all this in advance and then people would get all kinds of different things out of it, and I don’t want to say the course only worked if it fit what I decided in advance it should be doing. I’d rather be open to what actually happened, what people actually got out of it, whatever that is.
So maybe, in the end, it really is mostly about asking people for their own reflections and evaluations of what the course has done for them, because I can’t know in advance what that will be. That’s the point.
Obviously I am still struggling here. Thank you for the thought-provoking comment!
Would it make sense to compare a cMOOC to an actual class? Amount learned / enjoyment and so on?
Hi Pat:
Yes, this could make sense. I was thinking something along these lines if a group of people did a cMOOC for professional development, for example. Maybe they could compare their experiences there with “traditional” PD courses at a school or Uni. But why not do it with a course? Maybe if I offered one version of an on-campus course with a cMOOC component and another without (same course, different terms) that would provide an interesting comparison. I’d have to figure out just what data to gather, though, depending on what reasons I might have for making a cMOOC extension of one of them. Not sure they would learn more, exactly, though maybe. And not sure how I’d measure that in a philosophy course. I’d have to think more about what to look at, though perhaps just gather their reactions generally on what the course did for them, as Karen suggests in her comment, for both courses, and see what the difference is (if any). Hmmmm…good idea.
Hi Christina – I am just catching up with your posts. This post reminded me of a couple of things that you might be interested in:
Re goals for learning in MOOCs. You might want to follow the research that Allison Littlejohn and her team are doing at Glasgow Caledonian Academy – http://www.gcu.ac.uk/academy/people/allison-littlejohn/ . Following the Change 11 MOOC, her team interviewed participants (myself included) with the specific question of whether or not we had goals for the MOOC and whether we achieved them. I’m not sure where they’re up to with this research, but thought I would mention it.
Re George Siemens post about lurking – I did respond at the time in this blog post – http://jennymackness.wordpress.com/2010/12/04/in-defense-of-lurking/
I don’t think my views have changed. Openness is something that some of us need time to grow into!
Jenny
Hi Jenny:
So glad you found and commented on my posts! I appreciate the link here to Alison Littlejohn’s stuff…this sounds very useful. And thanks for your post about lurking, which really resonates with me. I lean much more on the side of wanting to give people the freedom to participate if/when/how they want to. That’s the experience that was so powerful for me in ETMOOC a few months ago (which got me started on all this cMOOC stuff in the first place!). I chose to be an active participant, but I certainly respect that there are many legitimate reasons why people might not do so, and that not providing them the freedom to do so may discourage them from connecting in any way at all.
Christina, I’m so pleased that you are doing this, and I think the issues that you raise cut to the heart of the discussion about cMOOCs. A couple of ideas came to mind as I read your post:
First, are questions about the effectiveness of cMOOCs somewhat premature? Do we need more research on just what cMOOCs actually do before we can assess if they do it well, or even if what they do is worth doing at all? In his post What a MOOC Does, Stephen Downes says quite bluntly that “MOOCs don’t change the nature of the game; they’re playing a different game entirely.” In this same post, Downes cautions against trying to “regard the MOOC through the perspective of the traditional course,” and he concludes that “If we can get past the idea that the purpose of a MOOC is to ‘teach people stuff’ then we can begin to talk about what benefits they bring.” Okay, then what is the MOOC game? I’m not confident that has been well defined, not by Downes, Siemens, Cormier or others, though I think they are trying. I’m willing to accept that, as an emerging phenomenon, cMOOCs are not yet stable enough to be defined, but I think we can start sketching the broad outlines. So research to determine just what cMOOCs actually do may be a first step.
Next, if we take seriously Downes’ comment that we must get past the idea that the purpose of a MOOC is to ‘teach people stuff’, then we undermine much of educational research and its methods. I think this is what you are struggling with. We have quite good methods for measuring things in the simple and complicated domains (to use terms from Dave Snowden’s Cynefin Framework), but cMOOCs work in the complex, sometimes the chaotic, domains, “in which the relationship between cause and effect can only be perceived in retrospect, but not in advance, the approach is to Probe – Sense – Respond and we can sense emergent practice” (Wikipedia). You may need to borrow assessment methods from complexity studies and transdisciplinarians.
Also, I think Downes’ comment mostly attacks the conventional notion that the purpose of education is to transfer skills and knowledge from an expert/teacher to a novice/student, as if knowledge and skills were some kind of thing that can be passed from one person to another. As you know, connectivist epistemology denies that knowledge is some nugget to be passed along. And this brings me back to my first question: if cMOOCs are not transferring knowledge, then what are they doing, can we measure what they do, and how?
I’m becoming circular here, but in complexity theory that is expected. This is worth thinking about. Thanks.
Hi Keith:
So glad you dropped by again…I have fallen so behind in my RSS feeds and haven’t read much of your blog lately. I must change that!
You’ve brought up some very good points. It is definitely the case that cMOOCs are still evolving, and amorphous, and it’s still not clear just what they do/are like, exactly. And of course, different ones have different characteristics. I have gone a bit back and forth in my mind and in what I’ve been writing here between asking whether a particular cMOOC has been effective, and asking about how to evaluate cMOOCs as a type of course in general. The former might be easier to do, because then one doesn’t have to think about what sorts of methods might work to evaluate any and all cMOOCs, and then one doesn’t run into the problem of trying to define what a cMOOC is, what counts as one and what doesn’t (which can start to get into thorny territory, if a cMOOC is thought to be a good thing and some people start to police its boundaries, cutting out courses that others think should be in the category, etc.). But at the same time, it does seem there could be some common characteristics, enough to say there is such a thing as a “cMOOC,” and that if we can determine that they are effective then courses that have those characteristics may be likely to be effective as well (recognizing that they could be done well or badly). So I’m still wavering back and forth between talking about cMOOCs generally and talking about how to evaluate a particular course with particular characteristics.
If the former, then yes, perhaps just doing some case studies to see what they even do, what happens in them, might be in order first. Still, that could mean we end up asking: what are cMOOCs doing, and then later asking, do they do well what they are doing. Which seems an odd thing to ask. Perhaps one could find out what they tend to do, and then ask whether they all do that thing equally well or not.
I would also like to think that those people who are offering cMOOCs are doing so with some sense of what they can and should be doing, which would drive the design of the course rather than offering them and just seeing what happens. But perhaps the latter is not so bad; I guess it often happens that courses work out in ways we didn’t plan in advance (and that can be a good thing). Still, I do think Downes, for example, has a pretty good sense of why the MOOCs he’s been involved in have been designed as they are, a theory underlying them, which could be part of measuring whether they’re effective–are they doing what they are designed to do? But I have to read more of Downes’ work before I get a good sense of what he thinks they should do.
There is, for him, at least the point that somehow the network should generate new knowledge. Not by teaching people things, providing content or teaching skills, but in some other way. That would be one thing one might measure, but I still haven’t figured out how one would determine if it has happened.
And, of course, not all cMOOCs are designed according to Downes’ theoretical arguments and foundations, so they may have different designs and goals, and do different things.
Which leads me back to the point that even trying to define and generalize about cMOOCs may be problematic.
Good point about usual research methods perhaps being better suited to traditional courses that are designed to teach people content or skills. I haven’t really looked into complexity theory very much, and probably should do so, so thank you for that!
And that’s about all I can reply with intelligently at the moment…I’m on holiday and my brain isn’t really in this sphere right now! I’m moving back to Canada in a couple of weeks and will get back into all of it then!
Hi Christina,
How to measure the effectiveness of a cMOOC? There are 4 semantic conditions of networks that Stephen Downes has proposed. As Stephen has commented, those properties – openness, diversity, autonomy and connectedness & interactivity is not perfect in cMOOCs. Besides Connectivism as applied in cMOOCs could likely best be based on an informal learning, rather than a traditional institutional model. I have reiterated that the constraints typically imposed with a institutional model would be huge challenge for administrators and educators to adapt, as is witnessed even in xMOOCs, where a totally new approach (such as flipping the class or flipped learning) as perceived by professors would be at odds with the mass lecture approach typical in mass-education, with a broadcasting model. How to overcome those challenges, and ensure learning is more effective, when cMOOCs are embedded in an institutional model? Here is my response http://suifaijohnmak.wordpress.com/2012/11/30/cfhe12-oped12-the-emergence-of-moocs-part-4-assessment-certification-and-accreditation/ that I perceive as a way to measure the effectiveness of cMOOCs – in its 1. awareness of Networked Learning and Connectivism as an “informal learning paradigm”, 2. an adoption and leveraging of the 4 properties openness, diversity, autonomy and connectedness & interactivity when networking, 3. an achievement of personal goals with immersion in the network and community (and community of practice) on personal basis, 4. adoption of Personal Learning Environment and Network PLE/PLN in pursuit of life-long learning, and 5. a shift of frame of reference and paradigm from knowledge transmission to knowledge sharing and creation model. Thanks again for your sharing. John Mak
Thank you, John! In a rush at the moment, but will look at your post soon and comment there. I’m very happy you have written a response, and I look forward to reading it!