Category Archives: Assignment 1: Rubric

     Our group (Group 3: YESNet: J. Chrona, M. Gallant, S. Turner, T. Walsh) designed a rubric that would be used to select a learning management system (LMS) for the Yukon Ministry of Education. The hope is to develop blended secondary courses that are relevant from an Aboriginal perspective.

     We decided to use Bates’ (2014) SECTIONS model since according to him, there is very little literature out there on how to choose the appropriate technologies for teaching. His SECTIONS model clearly breaks down 8 important criteria for people to use for this matter. Each of our group members chose two criterion to work on so the distribution of work was fair. The two criterion that I was tasked with was, “Teaching and Media Selection” and “Interaction”. It was really helpful to discuss as a group on Skype so we could each put our two cents in. As we were discussing Teaching and Media Selection, I didn’t think to include synchronous and asynchronous communication. However my group members pointed this out and I agreed that this was important to have. Working collaboratively in a synchronous environment, such as our group did, helped me to look at things differently in a broader context.

     One main point that I was overlooking at first was to include the Aboriginal perspective. I was more concerned at how our group should break down the selection of the LMS. I should have been much more mindful to include this topic and let it be the basis for our rubric since I took last term ETEC 521- Indigeneity, Technology and Education. Nevertheless, I was quick to point out that the Aboriginal perspective is just as important as the selection of the LMS.

     Throughout the MET program, I’ve encountered many online projects, in which all of them were communicated through Google Docs. This is the first time for me having multiple group meetings over Skype and I liked how we all worked in a team effort. Previously, I felt it challenging to read over everyone’s comments on Google Docs to see who changed what and when; with Skype we could all talk about our thoughts at that moment and work together.

Designing a rubric can sometimes be a daunting task. Trying to determine all the elements you want to evaluate, especially of a complex project, can lead to frustration and gaps. In this case, using the SECTIONS framework significantly helped to narrow down and focus the development of categories. It was interesting to see how the framework is general enough to evaluate a specific technology application and also effective to evaluate an entire technology learning platform. Isolating each part of the framework allowed us to target the specific needs of the University. We could essentially ask, “What questions would the University need to answer?” and then incorporate these questions into topics under each category. Most topics were easy to provide reasoning for numerical data, i.e. for IT support, 24 hour assistance vs. limited time of availability. Others were much more challenging as there was no exact way to distinguish a 3 from a 4, i.e. Usage Strategies, where some or all available components would constitute a change in levels, rather than an obvious linear evaluation. It was valuable to view other online rubrics as a reference and to further flesh out topics.   

Applying the concepts of the case study allowed the application of the framework in an authentic way, bringing the concepts to life. Considering the needs of the University in context of a globalized learning system and the complexity of both the market and the delivery systems, allowed us to understand the value of the framework. Factoring in where the University stands now, where they want to be in the future and some of the strategies they plan on using to get there, allowed for a longitudinal analysis. Ensuring that they chose a platform that would mesh with their current situation and be sustainable for the future is of key importance for an implementation of their size. Committing to a platform that will affect all teaching staff and be flexible to the addition of students from multiple countries is a significant decision that should be well researched.

In terms of group work, it was useful to have multiple forms of communication. Using the asynchronous discussion message board allowed us to share initial brainstorming ideas. Using the Google doc allowed all members to contribute and edit each others work. On multiple occasions more than one group member was adding to the doc at the same time and we used it as a temporary instant messaging system, providing instant feedback. Working collaboratively offered us a chance to deepen our understanding and create a more complete and reflective rubric.

Assignment #1: Reflection

For the Online Delivery Platform Evaluation Rubric Assignment our group had to develop a rubric to guide the leadership team at BCcampus to decide between two learning management systems. Looking back at the process involved in creating a rubric has been a very valuable experience. It allowed me to apply the Bates (2014) SECTIONS framework as well as the ‘Seven Principles of Good Practice’ discussed by Chickering and Ehrmann (1996) to a specific case scenario. Understanding the readings was one thing but being able to apply them is a different challenge. Each criteria had to be considered and evaluated based on its relevance to the case at hand. Prioritizing the criteria that would be included in the rubric took time but working in a group with diverse educational backgrounds and work experiences enriched the discussion. Listening to other members rationale for their criteria choices broadened my own perspectives on what is considered to be most important. Our entire group had solidified the importance of keeping the pedagogical foundations a top priority and not letting the technical aspects of the technology distract us from focusing on the learners needs.

The one criteria that stood out to me to be most in line with the trends discussed by Spiro (2014) is customization. This feature enables learning to be more adaptive to individual learners, digital information curation to occur and learners to take more control in constructing knowledge that is meaningful to them (Spiro, 2014). This feature will align BC Campus to be more responsive to the needs of current and future learners.

The other five criteria chosen also play an important role in the decision making process. Support, access and functionality serves to make the software application or web-based technology usable without significant stress for the learner such they there distracted from spending time on the actual learning objectives that need to developed. The cost and organizational requirements serve a greater purpose to the leadership team in order for them to be financially responsible in their decision and consistently aligned with existing mandates.

All in all, when making decisions regarding learning management systems many criteria need to be carefully considered. Certain criteria may have greater importance than others but decisions should be made with the flexibility to serve the needs of future learners as well as current ones.

References

Bates, T. (2014). Teaching in digital age http://opentextbc.ca/teachinginadigitalage/ (Chapter 8 on SECTIONS framework)

Chickering, A. W., & Ehrmann, S., C. (1996). Implementing the seven principles: Technology as lever. American Association for Higher Education Bulletin, 49(2), 3-6. Retrieved from http://www.aahea.org/articles/sevenprinciples.htm

Spiro, K. (2014). 5 elearning trends leading to the end of the Learning Management Systems. Retrieved from http://elearningindustry.com/5-elearning-trends-leading-to-the-end-of-the-learning-management-system

Designing our online delivery platform rubric was a good experience for me. After our slightly staggered start people worked together quite professionally. We completed the whole assignment asynchronously as we quickly discovered all of our schedules were very full and time zone differences complicated matters. Honestly I very much appreciated that. I find that when meeting synchronously there is a lot of time wasted with waiting for late arrivals and technical problems. People also tend to have a hard time agreeing to things when in a group setting. In our group we discussed a plan on the boards and decided very quickly what we wanted to do. Interestingly again, we didn’t assign roles. Jason started a doc and I remember starting the rubric. The next day work had been done on it. I worked awhile the next night and signed off to find the section completed. I’m not sure who finished it but I like that. I like that our assignment has been created together in such a way that no one claims ownership. In our scenario this what would have served our university best. That sort of collaboration brings out the best ideas and the cohesion without ego results in a more efficient process. I can only hope that future group projects will run so smoothly.

Situation:

Yukon Education Student Network (YESNet) and Yukon First Nations Programs & Partnerships Unit of the Yukon’s Ministry of Education want to develop blended secondary courses that are relevant from an Aboriginal perspective.

The Na Cho Nyak Dun First Nation has agreed to pilot the program, but there are concerns about bandwidth and reliability of internet access in their traditional, unceded territories: many of its members rely on 3G technology, for example. If the pilot goes well the program will be rolled out across the Yukon’s other 12 First Nations.

Task:

As members of YESNet’s Learning Technologies Advisory Committee (LTAC) you have been tasked with developing an evaluation rubric to decide which LMS would best suit YESNet’s overall needs, given both current usage and possible upcoming expansion.

Process:

Our group decided to use Bates’ (2014) SECTIONS model as a framework for the rubric due to its inclusion of a broad rage of factors that would need to be considered in choosing appropriate technology for learning environments. Although some of the consideration within the SECTIONS model seemed to more directly apply to post-secondary education contexts, the framework was flexible enough to adapt to a K-12 9and in the case of our scenario 8-12) context. In deciding on which questions to include we wanted to look closely at the information provided in the precis and ensure that the rubric was going to include criteria that would help address both the explicitly stated needs (i.e. bandwidth, reliability of internet access) and the implicit needs (i.e. what factors might need to be taken into consideration when implementing in unceded First Nations communities).

Our LMS evaluation rubric and paragraph summary was posted by Meghan here.

Reflection:

The process of determining what would be included in the rubric was collaborative in nature; we each provide initial draft content for specific sections of the rubric and then worked together to adapt and refine as necessary. In my estimation, what was built was more complete than what would have been developed if we had each worked individually. During the process I also came to focus on two ideas:

1. It is important to fully understand the context an LMS will be used in, especially if that context differs from our own. What might be relevant or suitable for some contexts, may be less useful in others. In the case of this scenario, I wanted to help ensure that we were paying attention to the specific context of YESNet’s mandate, where the LMS would be implemented, and who (both teachers and students) would be using it. Understanding this (which in the case of this assignment was helped by doing some additional simple research) helps ensure that a rubric for evaluating a potential LMS for this context would be as useful as possible.
2. Related to the previous idea is the understanding that some parts of an evaluation rubric for an LMS might have more weight than other aspects for evaluators. When I look at the factors (or considerations) we decided were necessary to include, I wonder if some of them might be more important than others during an actual evaluation. For example, what (if any) might be the primary drivers? Costs? Ease of Use? Adaptability? Perhaps in a real situation these (and other) factors might be given equal weight. But there is also the potential that the people making the final decisions might place more weight on some factors than on others. It is helpful to know about, and discuss these potential factors.

The process of collaboratively developing the rubric underscored the following for me:

Trying to choose the best LMS for a specific situation can be a challenging process, but it is helped by:

1. As much as possible, comprehensive knowledge of the learners/teachers and learning contexts
2. A group process of evaluation, as each member is able to bring different perspectives and help the other think about and question our own assumptions.

Reference:

Bates, T. (2014). Teaching in a digital age. Retrieved from http://opentextbc.ca/teachinginadigitalage

Our group’s evaluation rubric can be consulted here.

Group work

Carrying out this first assignment allowed me to pose a critical judgment on my school board’s LMS, or which I am the administrator.  All team members collaborated well in order to complete this task.  At first, we had an asynchronous brainstorming session in our group’s discussion Forum section of Connect.  It was a challenging task for us because of the different time zones.  We were never able to have a synchronous work session.  I was impressed at how, even though we were working asynchronously, all team members contributed ideas for our evaluation rubric.  After doing some research to see what already existed, we opted for an evaluation rubric similar to the ones teachers co-construct with students to better guide them in assignments and evaluation.

Our Rubric

We opted to construct our rubric around the criteria from SECTIONS (Bates, 2014).  We felt like Bates addressed most of the imports points that needed to be considered to thoroughly evaluate which  LMS would best suit Athabaska University’s needs.  We then took selected components from SECTIONS that applied to our scenario.  We included a 4 points scale to allow the ad hoc committee evaluation the different LMS to pin point how effectively it met each criterion (4= exceeds expectations, 1= does not meet expectations).   When selecting evaluation criteria for our rubric, we considered the upcoming expansion of Athabasca University’s programs to the South Asian Market. The main concern was accessibility, which is addressed with the Student component of the SECTIONS model.

References

Bates, T. (2014). Teaching in a Digital Age/ (Chapter 8 on SECTIONS framework). Retrieved January 29, 2016 fromhttp://opentextbc.ca/teachinginadigitalage

Photo credit: Entering the Yukon Territory of Canada by R. Martin (WikimediaCommons)

Our group (Group 3: YESNet: J. Chrona, M. Gallant, S. Turner, T. Walsh) created a rubric which would be used to select a learning management system (LMS) for the Yukon Ministry of Education which hopes to create blended-learning secondary courses that would serve the needs of First Nations’ communities.

We chose to use Bates’ (2014) SECTIONS framework as the basis for our rubric. I was familiar with Bates’ work having read Bates and Sangra (2011) “Managing Technology in Higher Education” for a summer course. Bates’ expertise really is on choosing technology for post-secondary institutions; however, his framework is broad enough to encompass the issues we thought were important, and it is flexible enough to be applied to any major investment in educational technology, regardless of the institution or educational level at which it is employed.

It was helpful to complete this project as a group because we could benefit from each other’s experience. We each drafted a couple of sections on our own, but then came together to review and fine-tune our rubric. Working it out together took longer than we expected, but still less time than it would have, had we reviewed our own projects individually.

We were lucky to have someone knowledgeable in First Nations culture who could ensure that we did not miss any important elements of Aboriginal educational values. However, every member of our team was committed to ensuring that we were developing a rubric that would be inclusive of diverse student needs and strengths. In other words, we all followed Chickering and Ehrmann’s (1996) description of the seventh principle of good practice in education, which is: respecting diverse talents and ways of learning.

Like Nel, Dreyer, and Carstens (2010) encourage, we focused on their ‘primary criteria’ (enhancing learner-centered educational principles) when deciding on our standards for our LMS. We hoped that the medium chosen would enable instructors to create an excellent learning experience for their students. Although, the Nel, Dreyer and Carstens (2010) study focused on the teaching of language, they point out that the ability of technology to provide “additional possibilities to receive input and produce output…can establish a rich context” for learning to occur (p. 242). Consequently, we wanted our LMS to be supportive of multi-media content both instructor- and student-generated.

It was also crucial that the LMS had networking capability because connection to community is an essential component of education for First Nations’ people (Chrona, 2015). This is, of course, a value that is not exclusive to Aboriginal cultures. Indeed, Porto (2013), describes the importance of social media in education for supporting communication and collaboration. When online instructors work to help their students form learning communities, they often turn to social media such as blogs, wikis and other networking sites (Porto, 2013).

The greatest obstacle that we faced was one of access. As Coates, James, and Baldwin (2005) point out, increasing access to education is a significant reason why educational institutions consider adopting an LMS. Presumably, providing access to quality education is a key reason why the Yukon Ministry of Education was interested in piloting its courses on an LMS for use on the Na Cho Nyak Dun First Nation. However, the nation’s citizens rely primarily on 3G technology, having otherwise only limited bandwidth and reliability when it comes to internet access. So, relying on an LMS may be a double-edged sword. Technology could truly improve access to quality education for First Nations’ people but for those living in remote areas technological disruptions may also hamper their success.

References:

Bates, T. (2014). Teaching in a digital age. (Chapter 8). Retrieved from http://opentextbc.ca/teachinginadigitalage/

Bates, A. W., & Sangra, A. (2011). Managing Technology in Higher Education. San Francisco, CA: Jossey-Bass.

Chickering, A. W., & Ehrmann, S. C. (1996). Implementing the seven principles: Technology as lever. American Association for Higher Education Bulletin, 49(2), 3-6. Retrieved from http://www.aahea.org/articles/sevenprinciples.htm

Chrona, J. (2015). First people’s principles of learning. Retrieved from https://firstpeoplesprinciplesoflearning.wordpress.com/

Coates, H., James, R., & Baldwin, G. (2005). A critical examination of the effects of Learning Management Systems on university teaching and learning. Tertiary Education and Management, 11(1), 19-36. http://link.springer.com/article/10.1007/s11233-004-3567-9

Nel, C., Dreyer, C., & Carstens, W. A. M. (2010). Educational technologies: A classification and evaluation. Tydskrif vir letterkunde, 35(4), 238-258. Retrieved from http://www.ajol.info/index.php/tvl/article/download/53794/42346

Porto, S. (2013). The uncertain future of learning management systems. The Evolllution: Illuminating the Lifelong Learning Movement. Retrieved from http://evolllution.com/opinions/uncertain-future-learning-management-systems/

Here is the Evaluation Rubric for our Group.

Véronique, Jason, Amber and Kimberley

Our group was tasked with creating a rubric to evaluate LMSs for BCcampus. Our group spanned 2 provinces and one territory. We completed our task through a shared document in Google. A link to our completed work is here.

The specifics of the scenario we were given were these:

BCcampus is a publicly-funded agency that offers “teaching, learning, and educational technology support“ to the 25 post-secondary institutions in British Columbia and one in the Yukon.  As part of its shared services, BCcampus has been running two LMS platforms; one open-sourced (Moodle) and the other vendor based (D2L) whose contract will expire shortly.  With current restructuring, BCcampus will be losing half of its tech support team in three months, and therefore the possibility of running two separate LMS platforms is no longer an option.  BCcampus will need to decide which LMS they wish to proceed with across the board.  However, as BCcampus is known for its leadership in innovation, they are open to the idea of selecting an entirely new platform. As employees at BCcampus, we have been asked to create an evaluation rubric to help the BCcampus leadership team in the decision making process.  We have designed the rubric to help select a LMS that will fit with the needs, the vision and the mandate of BCcampus: “to connect, collaborate and innovate“. The selection process must also recognize the LMS’ ability to conform and adapt to the geographically and culturally diverse needs of BCcampus’ partner institutions and the thousands of post-secondary students across the country.

The detailed rubric we developed looks like this:

The reasoned articulation of our choices are in this researched-supported rationale:

Our LMS evaluation rubric is based on elements of the Bates SECTIONS model as well as Chickering and Ehrmann’s seven principles of good practice. As Coates points out, “decisions about university teaching and learning should not be restricted to checklist evaluations of technical and organizational factors” (2005). With that in mind, we did not limit our rubric to standard evaluation criteria, rather we designed the rubric around the specific needs of BCcampus; developing six high-level criteria encompassing the primary concerns of the agency: Access, Support, Functionality, Cost, Customization, and Organizational Requirements.  

With 25 affiliates, the importance of versatility without restrictions is paramount. A system that can be accessed by every institution, device and end user, without compromise to functionality, will keep BCcampus at the forefront of innovation. Further, the BCcampus IT support will be reduced in three months making system compatibility an important requirement.

Secondly, it is important that the selected LMS provides support to all users. IT support must be available for BCcampus, the institutions, and the individual users, as these stakeholders might require different assistance at various times. If teachers and learners are not well supported then there is a high risk that the LMS may not be used at all (Bates, 2014).

The functionality of the LMS is an integral criteria for BCcampus to maintain their leadership in innovation by empowering the current and future learners through intuitive user interfaces.  Furthermore, the functionality should not compromise the security and privacy of its users in order for open discussions to occur without consequences (Bates, 2014).

BCcampus uses an innovative collaborative model that allows various post-secondary institutions to share resources and costs (BCcampus, 2013). Participating institutions can access the resources and technologies they would not be able to individually. BCcampus must have information about the initial implementation, and future LMS maintenance costs in order to budget effectively. This in turn, will affect the participating institutions and their budgeting.

As Spiro mentioned, students are increasingly taking charge of their own learning; gone are the days of one-size-fits-all courses (2014). This mentality calls for a LMS platform that is customizable – allowing students and teachers alike, the opportunity to curate their own learning path. Further, BCcampus is looking for a long term LMS, meaning it has to be adaptable and able to integrate with new technologies in order to meet the needs of its diverse student population.

Lastly, BCcampus provides a leading-edge collaborative interface.  As such, the LMS it uses must have strong cognitive design in its structural components, align with the mandates of its partners, and provide exceptional modalities for communication. We understand this will not be the only tool an evaluation team uses, but believe that the six areas covered will provide a well-rounded summary to the committee.

We used this relevant literature to support our decisions:

Bates, T. (2014). Teaching in digital age http://opentextbc.ca/teachinginadigitalage/ (Chapter 8 on SECTIONS framework)

BCcampus. (September 2013). Annual Report 2012-2013, Strategic Plan 2013-2016,  Retrieved from: http://bccampus.ca/files/2013/10/2013-ar-stratplan.pdf

Chickering, A. W., & Ehrmann, S., C. (1996). Implementing the seven principles: Technology as lever. American Association for Higher Education Bulletin, 49(2), 3-6. Retrieved from http://www.aahea.org/articles/sevenprinciples.htm

Coates, H., James, R., & Baldwin, G. (2005). A critical examination of the effects of Learning Management Systems on university teaching and learning. Tertiary Education and Management, 11,(1), 19-36. http://link.springer.com/article/10.1007/s11233-004-3567-9

Spiro, K. (2014). 5 elearning trends leading to the end of the Learning Management Systems. Retrieved from http://elearningindustry.com/5-elearning-trends-leading-to-the-end-of-the-learning-management-system

My reflection upon your experience completing this assignment

The learning objectives for this module were centred on understanding the role of delivery platforms, and to think about the challenges and opportunities that come with those platforms. Our group work, toward these ends, afforded me a unique learning experience.

My first realization was that evaluative work, such as creating a rubric to select an LMS interface for BCcampus, cannot be a one-person task. The sheer enormity of facets to be aware of needs to be tackled by a team. That team needs be one that have members who know their facts, the implication of those facts, and be thorough.

My second realization was two-part. I did not know/understand what LMS are/were. And further to that, it was a stretch to comprehend the ubiquitous role they play. To add to what I wrote in the first point, many and varied are the things that the LMS interface must provide and perform, and many more are the things that we still wished they could do. The best ones seem to be “invisibly there.’

My third and final realization for this purpose, is that the work that goes on “behind the scenes” often goes under-appreciated by those that use these systems. We know when we don’t like something (how it looks, how it functions, getting help), but sadly it is rare to acknowledge all of the thinking and planning and coding and envisioning and testing and modifying and . . . that several layers of hard work that have resulted in what we see before us.

Keri

• Describe the role delivery platforms play in creating rich educational environments;