Please access the Moodle Quiz at this link.
The level of fidelity in training describes the feedback between the training context and real-world situations (Keller-Glaze, Horey, Nicely, Miller-Nihill & Cobb, 2013, p. 166). Research shows that learners frame their engagement with education around assessment (Gibbs and Simpson, 2005; Jenkins, 2004), and therefore to provide meaningful engagement the level of fidelity in assessment must be high.
To support the fidelity of this quiz, questions relate to
- Functional requirements of interns, and
- Fundamental concepts of the CAF Campus project.
Computer-assisted assessment options in Moodle are diverse and powerful. In addition to thirteen question types in Moodle 2.5+, there are seven question behaviours that allow diverse and engaging design choices. To comply with the requirements of ETEC565A Assignment 3, four question types and assessment strategies have been included. Jenkins (2004) provides multiple examples of the ways in which CAA can support student learning, and the mix of question design and feedback that I have included in this quiz attempts to deploy the diagnostic, formative and summative functions. Diagnostic assessment identifies to an instructor the state of the learner group, formative assessment refers to the ways in which students test their own understanding, and summative assessment refers to an end-point testing of the knowledge gleaned or gained (Mason & Rennie, 2006). These may be alternatively described as assessment for, as, or of learning, and in the context of this assignment, assessment functions appear as follows:
Assessment for learning: Learners are quizzed about basic components of their internship and important components of the course and study strategies are highlighted. The instructor has the ability to review learner attempts to diagnose which learners may require additional support or intervention or where the course may benefit from changed content.
Assessment as learning: The overarching question behaviour is set to “interactive with multiple tries,” allowing the opportunity for immediate feedback. After numerous incorrect responses, the answer is locked, which discourages guessing. Because learners receive multiple tries and cues about where to find the information in their texts, the quiz serves the function of a learning and review tool for students to check their progress.
Assessment of learning: The quiz contains two short essay questions which, unlike the other questions, does not allow for immediate feedback and is summative in nature. Following the introductory module icebreaker activity where students were asked to reflect back on the events that brought them to NDIP101, they are now asked to look forward to how they see themselves applying their skills, and to argue the overall necessity of the CAF Campus. Because the entire project scope is outlined in the opening chapters of the CAF Campus Operational Framework (2013), students are not required to have read the entire text before submitting their response. An assessment rubric is provided to assist in the manual assessment of essays and to frame individualize feedback, which has been shown by Gibbs and Simpson (2005) to be highly valued by learners.
Pedagogic Choices Relating to Question Type
Multiple Choice
Multiple choice questions in this quiz assess the interpretation of acronyms that are frequently used throughout the CAF Campus Operational Framework (2013). Although testing acronyms may seem to serve a simple, low-order memorization function; highlighting their importance serves a practical purpose in NDIP101.
Acronyms are prevalent throughout defence literature and used frequently even in spoken conversation. Assessing acronyms focuses their importance to learners, as learners tend to frame item importance by what appears on assessments (Gibbs and Simpson, 2005; Jenkins, 2004). Familiarity of frequently used acronyms will assist interns in fluency and comprehension of oral and written communication consistent to their context. Further support for assessing frequent acronyms comes from Butcher, Davies and Highton (2006), who assert that meaningful assessment enables learners “…to improve their learning and study skills” (pg. 95). Questions have been written to adhere to best practices as outlined by Malamed (2010a).
Matching
Matching questions represent factual and objective assessment, providing more flexibility for content than multiple choice questions (Malamed, 2010b). Following the matching format conventions and best practices covered by Malamed (2010b), three matching questions that assess knowledge of essential organizational support components of the CAF Campus vision and projects appear on this quiz.
Short Answer
In Moodle, short-answer response input must include an exact match with what an instructional designer has entered into the answer field. It is therefore important for the instructional designer to provide clear instructions and brief answers. The use of wildcards ( * ) in the Moodle short-answer assessment allows for some flexibility in the student’s form and format of text provided (Short-Answer question type, 2011). However, even with the use of wildcards, it can be difficult to predict all possible responses. In this quiz, short answer questions are supported by graphics and relevant text is highlighted in bold characters.
Short Essay
Moodle documentation states that short essay questions in the quiz format are intended for one or two paragraph answers and this question type must be manually graded (Essay question type, 2012). Because they are manually graded, a rubric has been included in the instructor notes area to assist with assessment. This rubric is also provided to students prior to, and during the quiz.
Stated goals of the CAF Campus (2013) include the support of civilian-military partnerships and recognition of the CAF as a learning organization; and as such, the short-essay questions in this quiz focus on addressing these goals. The purpose behind these essay questions are to equip learners with self-awareness and self-efficacy of their role in the NDIP. Because of their self-reflective nature, both short essay questions fit within Gibbs and Simpson’s (2005) condition of a productive learning activity. The process of self-reflection fits into the framework of self-directed leadership development (Nesbit, 2012), and both questions allow high-fidelity opportunity for learners to become further equipped for a successful internship experience.
REFERENCES
Butcher, C., Davies, C., & Highton, M (2006). Designing Learning: From Module Outline to Effective Teaching. In Chapter 6: Matching your Assessment to Outcomes. Taylor & Francis. Retrieved 19 February 2014, from <http://www.myilibrary.com?ID=64837>
Essay question type. (January 28, 2012). Moodle Docs. Retrieved 20 February, 2014, from http://docs.moodle.org/22/en/Essay_question_type
Gibbs, G. & Simpson, C. (2005). Conditions under which assessment supports students’ learning. Learning and Teaching in Higher Education. Retrieved 18 February, 2014, from http://www.open.ac.uk/fast/pdfs/Gibbs%20and%20Simpson%202004-05.pdf
Jenkins, M. (2004). Unfulfilled Promise: formative assessment using computer-aided assessment. Learning and Teaching in Higher Education , i, 67-80. Retrieved 20 February, 2014, from http://insight.glos.ac.uk/tli/resources/lathe/documents/issue%201/articles/jenkins.pdf
Keller-Glaze, H., Horey, J., Nicely, K., Miller-Nihill, M., & Cobb, M.N. (2013). A practical decision guide for integrating digital applications and handheld devices into advanced individual training. (Research Report 1967). U.S. Army Research Institute for the Behavioural and Social Sciences.
Malamed, C. (March 18, 2010a). 10 Rules for writing multiple choice questions. The eLearning Coach. Retrieved February 28, 2014, from http://theelearningcoach.com/elearning_design/rules-for-multiple-choice-questions/
Malamed, C. (July 20, 2010b). Tips for writing matching format test items. The eLearning Coach. Retrieved February 28, 2014, from http://theelearningcoach.com/elearning_design/writing-matching-test-items/
Mason, Robin; Rennie, Frank. (2006). Elearning: The key concepts. Taylor & Francis. Retrieved 21 February 2014, from <http://www.myilibrary.com.ezproxy.library.ubc.ca?ID=54298>
National Defence (2013). Learning without boundaries: CAF campus operational framework. Individual Training and Education Modernization Initiative, Canadian Defence Academy [Kingston].
Nesbit, P. L. (2012). The role of self-reflection, emotional management of feedback, and self-regulation processes in self-directed leadership development. Human Resource Development Review,11(2), 203-226. Retrieved February 26, 2014, from http://dx.doi.org/10.1177/15344843124391
Short-Answer question type. (July 6, 2011). Moodle Docs. Retrieved February 29, 2014, from http://http://docs.moodle.org/22/en/Short-Answer_question_type
