Assignment #1: Online Delivery Platform Evaluation Rubric
Group Members: Patrick Conlan, Victoria Olson, Allen Wideman, Heather Woodland
Link to Assignment in Google Document (preferred viewing for rubric portion)
Scenario Précis
Our group was given the responsibility of developing an evaluation rubric to determine which LMS would successfully meet the needs of a new online course program being developed to support students enrolled at Le Conseil scolaire francophone de la Colombie Britannique. Currently, Le Conseil runs a number of face to face schools across BC, as well an online portal, Ecole Virtuelle, that supports enrolled students.
In order to provide opportunities for adult francophone students to access courses, including those required for high school graduation, Le Conseil aims to work with the cooperation of LearnNowBC to develop an online program accessible to over four thousand potential students living throughout BC. As the current demand outside of greater Vancouver does not justify the offering of face-to-face high school completion programs for adult students, Le Conseil seeks to ensure that they select the most suitable LMS to support these students, many of whom perceive their lack of English literacy skills to be a challenge in further pursuit of their studies.
Online Delivery Platform Evaluation Rubric for Le Conseil:
This portion of the assignment is more easily reviewed via Google Doc. Please navigate to our original assignment document or download the attached file: Assignment1Rubric
| Component | Exceeds Expectations | Meets Expectations | Minimally Meets Expectations | Does Not Meet Expectations |
| Logistics, Support, & Management | ||||
| Cost | Budgetary allowances completely cover costs associated with licensing of product, infrastructure (i.e. servers/hosting solutions, network access), training and development personnel, and technical support resources (i.e. IT staff, web developers, support tickets). Some opportunity for savings. | Budgetary allowances completely cover costs associated with licensing of product, infrastructure (i.e. servers/hosting solutions, network access), training and development personnel, and technical support resources (i.e. IT staff, web developers, support tickets). No opportunity for savings. | Budgetary allowances do not quite cover costs associated with licensing of product, infrastructure (i.e. servers/hosting solutions, network access), training and development personnel, and technical support resources (i.e. IT staff, web developers, support tickets). No opportunity for savings. | Budgetary allowances are unable to cover costs associated with licensing of product, infrastructure (i.e. servers/hosting solutions, network access), training and development personnel, and technical support resources (i.e. IT staff, web developers, support tickets). No opportunity for savings. |
| Open-source vs Proprietary | The software platform (i.e. customizable code, privacy, internal development, data retention, future migration, flexibility) surpasses the institutional requirements. | The software platform (i.e. customizable code, privacy, internal development, data retention, future migration, flexibility) meets the institutional requirements. | The software platform (i.e. customizable code, privacy, internal development, data retention, future migration, flexibility) addresses some of the institutional requirements. | The software platform (i.e. customizable code, privacy, internal development, data retention, future migration, flexibility) addresses none of the institutional requirements. |
| Required infrastructure | Specifications for LMS include complete flexibility on platform location (i.e. remote hosting, local servers, etc), server hardware/software, user hardware/software (i.e. desktop, laptop, mobile, Mac, PC, Linux, etc.), and network speed/connection type (i.e. LAN, WiFi, cellular, dial-up, etc.). | Specifications for LMS include flexibility on platform location (i.e. remote hosting, local servers, etc), server hardware/software, user hardware/software (i.e. desktop, laptop, mobile, Mac, PC, Linux, etc.), and network speed/connection type (i.e. LAN, WiFi, cellular, dial-up, etc.). | Specifications for LMS include limited flexibility on platform location (i.e. remote hosting, local servers, etc), server hardware/software, user hardware/software (i.e. desktop, laptop, mobile, Mac, PC, Linux, etc.), and network speed/connection type (i.e. LAN, WiFi, cellular, dial-up, etc.). | Specifications for LMS include no flexibility on platform location (i.e. remote hosting, local servers, etc), server hardware/software, user hardware/software (i.e. desktop, laptop, mobile, Mac, PC, Linux, etc.), and network speed/connection type (i.e. LAN, WiFi, cellular, dial-up, etc.). |
| IT Support | Fully offers logical system for submitting support tickets, support communities/discussion boards, administrator conferences/training, various methods of communicating support, and integrates into current institutional IT systems. | Offers logical system for submitting support tickets, support communities/discussion boards, administrator conferences/training, various methods of communicating support, and integrates into current institutional IT systems. | Partially offers logical system for submitting support tickets, support communities/discussion boards, administrator conferences/training, various methods of communicating support, and integrates into current institutional IT systems. | Unable to offer logical system for submitting support tickets, support communities/discussion boards, administrator conferences/training, various methods of communicating support, and integrates into current institutional IT systems. |
| Management | Intuitive and simple management features of account creation, variable access permissions, course enrollment, term migration, data integration with other student information systems, and student-to-student and student-to-teacher interaction (i.e. discussion boards, chat systems, email, etc). | Provides management features of account creation, variable access permissions, course enrollment, term migration, data integration with other student information systems, and student-to-student and student-to-teacher interaction (i.e. discussion boards, chat systems, email, etc). | Difficult management features of account creation, variable access permissions, course enrollment, term migration, data integration with other student information systems, and student-to-student and student-to-teacher interaction (i.e. discussion boards, chat systems, email, etc). | Lacking management features of account creation, variable access permissions, course enrollment, term migration, data integration with other student information systems, and student-to-student and student-to-teacher interaction (i.e. discussion boards, chat systems, email, etc). |
| Communication | ||||
| Effective Communications Channels
(internal email and/or links to external email; forum capabilities; voice chats; video chats) |
Availability of both public and private asynchronous and synchronous communications options, including all of: student-student communication, student-instructor communication, and instructor-instructor communication across Le Conseil’s virtual offerings. | Availability of both public and private asynchronous and synchronous communications options, including most of: student-student communication, student-instructor communication, and instructor-instructor communication across Le Conseil’s virtual offerings. | Availability of public and/or private asynchronous and synchronous communications options, including only some of: student-student communication, student-instructor communication, and instructor-instructor communication across Le Conseil’s virtual offerings. | Highly limited availability of communications options for student-student communication, student-instructor communication, and/or instructor-instructor communication across Le Conseil’s virtual offerings. |
| Flexible Communications | LMS provides ability to run courses in both singular student correspondence and cohort-based models. Movement between these models is highly flexible. | LMS provides ability to run courses in both singular student correspondence and cohort-based models. Movement between these models is moderately flexible. | LMS provides ability to run courses in both singular student correspondence and cohort-based models. Movement between these models is not flexible. | LMS provides ability to run courses in either singular student correspondence or cohort-based models, but not both. |
| Assessment Opportunities & Features
(assignment dropboxes, comment forms, grade reporting, discussion fora, etc.) |
Offers capabilities to integrate formative and summative assessments for both individuals and groups into course design. LMS offers an abundance of options to the instructor. | Offers capabilities to integrate formative and summative assessments for both individuals and groups into course design. LMS offers a moderate amount of options to the instructor. | Offers capabilities to integrate formative and summative assessments for both individuals and groups into course design. LMS offers a limited amount of options to the instructor. | Highly limited capabilities to integrate formative and summative assessments for both individuals and groups into course design. Not satisfactory to general instructor needs. |
| Integrated Services
(third party applications, collaborative features, etc.) |
LMS provides robust integration of third-party applications or features that provide collaborative opportunities to both instructors and students. | LMS provides satisfactory integration of third-party applications or features that provide collaborative opportunities to both instructors and students. | LMS provides limited integration of third-party applications or features that provide collaborative opportunities to both instructors and students. | LMS provides no integration of third-party applications or features that provide collaborative opportunities to both instructors and students. |
| Design | ||||
| Layout | Technology provides exceptional opportunities for personalized pedagogical design and course layout, including links to external digital sources, multimedia, or readings. | Technology provides personalized pedagogical design and course layout, including links to external digital sources, multimedia, or readings. | Technology demonstrates limitations in personalized pedagogical design and course layout, including some opportunities for links to external digital sources, multimedia, or readings. | Technology lacks personalized pedagogical design and course layout, including functionality for links to external digital sources, multimedia, or readings. |
| Flexibility | Subject-specific & interdisciplinary needs are addressed through diverse, flexible design and application components. Provides opportunities to meet differentiated student learning goals within the format of a variety of course offerings, including core high school courses and electives. | Interdisciplinary needs are addressed through flexible design and application components. Provides ample opportunities to meet differentiated student learning goals within the format of a variety of different course offerings, including core high school courses and electives. | Interdisciplinary needs are addressed in a limited capacity through flexible design. Limited opportunities to meet differentiated student learning goals within the format of a variety of course offerings. | Overall lack of flexible design and application components. Fails to address the interdisciplinary needs of courses to be offered. |
| Customizable | Abundance of templates available for course design that may be customized to meet a variety of instructor specific needs. | Multiple templates available for course design, but may be customized to meet instructor specific needs. | Limited templates available for course design, but may be customized to meet some instructor specific needs in a basic capacity. | Available templates do not allow for adequate course design and fail to meet customizability requirements for instructor specific needs. |
| Usability | ||||
| Ease of Use | All navigation of the user interface provides intuitive pathways for the user. | Most navigation of the user interface provides intuitive pathways for the user. | Some navigation of the user interface provides intuitive pathways for the user. | The navigation of the user interface does not provide intuitive pathways for the user. |
| Orientation for Use
(media offered by LMS or similar media to be created and embedded by Le Conseil) |
An integrated, appropriate, and language accessible orientation is made available to all users of the LMS or there is potential to integrate customizable media for this purpose. | A language accessible orientation is made available to all users of the LMS or there is potential to integrate customizable media for this purpose. | A language accessible orientation can potentially be made available to users of the LMS. | There is no potential for a language accessible orientation. |
| Inclusion of in-app tutorials or app “tours” | Integrated, appropriate and language accessible in-app tutorials are made available as the user navigates through the system. | Language accessible in-app tutorials are made available as the user navigates through the system. | There is potential to have language accessible in-app tutorials made available to users as they navigate through the system. | There is no potential for in-app tutorials. |
| LMS offers multiple language capabilities in user interface | LMS offers multiple language capabilities in user interface. | LMS offers multiple language capabilities in user interface but there are some issues with its performance. | LMS offers multiple language capabilities in user interface but there are many issues with its performance. | LMS does not offer multiple language capabilities in user interface. |
Rationale for Inclusions in the Rubric
Our scenario specifically calls to assess Learning Management Systems (LMS) that cater to adult students who lack confidence in English language proficiency skills and may not come to the program with previous online learning experience. There were a number of logistical facts that we were uncertain about with our given scenario, including whether or not courses would be offered via correspondence to each individual student enrolled or whether a cohort-based model would be followed. We also didn’t know whether or not instruction was explicitly in English or in French, or a combination of the two, though we deduced that multilingual capabilities within the chosen LMS was probably a requirement. As such, we have broken our rubric into four major categories:
- Logistics, Support, & Management,
- Communication,
- Design, and
- Usability
Our group chose these components to address a number of positive impacts on teaching and learning listed in Coates, James, & Baldwin’s LMS article from 2005, while taking considerations of some cautions surrounding the potential for future LMS obsoletion from Spiro (2014) and Porto (2015), as well as Bates’ (2014) updated SECTIONS model. First, we aimed to assess whether or not technologies featured sufficient logistical components from a managerial stance, including cost effectiveness, infrastructure compatibility with the institution, and intuitive management features for both IT and instructors. Secondly, there was a focus on the availability of both public and private communications and assessments between the various stakeholders that would utilize the LMS, including student-to-student and student-to-instructor communication within courses, and instructor-to-instructor communication across the Le Conseil institution. This category also included third-party collaborative or social applications included within the LMS framework, that would directly address Porto’s (2015) growing concerns of lack of learner personalization within these technologies. Thirdly, we focused on the design and layout components of the LMS, as these may affect the experiences of instructors and students alike. Creation of customizable course offerings provides instructors and designers with the ability to be adaptable to the needs of diverse academic cultures and communities (Coates et al., 2005, p.31; Spiro, 2014). Lastly, we focused on the general usability of the platform for all stakeholders (Bates, 2014), including ease of use, multilingual capabilities for the user interface, and tutorial options for students who may not have experience in online learning environments.
My Individual Reflection
This assignment challenged me to consider a scenario beyond the scope of my own context. Oftentimes technologies are chosen because they fit a highly specific need in an institution and those factors may vary greatly from one site to another. The needs presented in our group scenario (language differences, adult learners, etc.) obviously required attention while, at the same time, we opted to maintain balance in assessing for quality in the overall technologies chosen. This triggered some personal thought about the different school sites in my own district, which is decentralized (I.e. we are not required to use the same software and apps from school to school), and how each school goes about choosing an LMS or similar technologies. I also considered the pros and cons of such a model: what are the benefits of being decentralized when compared to centralized districts or vice versa? Overall I enjoyed this assignment as it got me thinking about which overall features that I value in a technology-infused learning environment (as per Coates, James, & Baldwin, 2005) and which features might be missing or require improvement (as per Porto, 2015). I highly value ease of use for all stakeholders involved as well as communication and personalization features, including third-party app integration.
References
Bates, J. (2014). Teaching in digital age, Chapter 8. Retrieved from http://opentextbc.ca/teachinginadigitalage/
Coates, H., James, R., & Baldwin, G. (2005). A critical examination of the effects of Learning Management Systems on university teaching and learning. Tertiary Education and Management, 11,(1), 19-36. http://link.springer.com/article/10.1007/s11233-004-3567-9
Porto, S. (2015). The uncertain future of Learning Management Systems. The Evolllution: Illuminating the Lifelong Learning Movement. Retrieved from http://www.evolllution.com/opinions/uncertain-future-learning-management-systems/
Spiro, K. (2014). 5 elearning trends leading to the end of the Learning Management Systems. Retrieved from http://elearningindustry.com/5-elearning-trends-leading-to-the-end-of-the-learning-management-system