A method of assessment
The Moodle exam was designed to employ objective tests that assess aspects of student learning about computer programming. Objective tests (Pass IT, 2004) require examinees to provide responses to questions the correct answers of which are pre-determined. Multiple-Choice, Short-Answer and Matching question types are good choices to use in objective tests with CAA (Computer Aided Assessment). Questions based on these three types are designed to test Bloom’s taxonomy (Center for Teaching, 2001): specifically knowledge, comprehension, application, and analysis. Essay questions are not easily assessed through CAA because pre-determined answers cannot be provided. Therefore, essay questions should be used for evaluating leaners’ analysis, application and synthesis skills.
Multiple Choice Questions
Multiple-choice questions are suitable for objective tests that measure learners’ level of understanding in a particular knowledge area. Multiple-choice questions can help teachers identify learners’ conceptual knowledge and their ability to give examples related to the subject (Anderson, 2001) instead of simply recalling knowledge. I created three multiple-choice questions to evaluate learners’ understanding of the specific areas through examples.
Matching Questions
Matching questions are a good choice to evaluate learners’ understanding of the relationship between different pieces of knowledge: between definitions and terms, between examples and terms, between classifications and structures, and so on. Matching questions are a variation of the multiple-choice format and can be employed for the same purpose but they can also be used to evaluate a learner’s ability to apply knowledge . For example, three of the matching questions created in Moodle evaluate learners’ understanding of the relationship between the Scratch program blocks and corresponding programming concepts, as well as the application of programming knowledge in using Scratch blocks.
It appears that partial marking is missing in the Moodle matching question type. However, I think that partial marks should be given to students successfully identify any correct answer.
Short Answers
A possible advantage of the short-answer question type is that the student must supply the correct answer rather than identify or choose it. The likelihood that the learner will guess the correct answer is lower than that of either a multiple-choice question or a matching question.
Setting up answers for short answers questions was a bit frustrating because answers could be in many different formats. I overcame the issue by giving specific instructions on how to write answers with the acceptable format. For example, one of the short-answer questions required that the answer be “conditional statement, loop, statements”. I asked examinees to provide the answer in a specific order, separating its parts with commas. This would avoid issues like using different separators and writing the answers in a different order. Luckily, there was a feature in Moodle to allow short answer questions to have more than one answer to be evaluated. I successfully evaluated answers based on the number of matching words using wild cards (*).
Essay Questions
I chose two different topics. One of them asks students to write about “the ethical implications of adapting someone else’s software programming”. The topic aims to promote real-life learning opportunity. The question can be part of a real-life interview. Analyzing the topic is a good way to foster programming ethics. The other topic asks students to compare differences between two programming syntax styles. The goal is to evaluate students’ analysis skills.
Assessment Feedback
The feedback has been carefully designed to promote extended learning based on Gibbs & Simpson’s (2004) assessment framework.
I provided feedback that contained either details or further review links so that the process can promote active learning. The feedback did not state the responses were either correct or not correct. The assessment was provided as a part of a learning process. The exam provided students with constructive learning opportunities – further reading suggestions and course content URLs associated with the questions – to foster the assessments to be an extent of learning. (Gibbs & Simpson, p. 17)
The feedback was given immediately after the exam and received by students while it still matters to them, in time for them to pay attention to further learning or receive further assistance (Gibbs & Simpson, p. 18). This immediate feedback was also emphasized in the seven principles of good practice (Chickering & Gamson, 1987; 1991).
The Moodle feedback setting was missing one feature. It did not provide individual feedback when choices were not selected by examinees. Moodle only displayed feedback when answers were selected by examinees. The feature is important because non-selected answers can also provide students with opportunities to learn. Currently the opportunities are somewhat limited to showing feedback only related to the selected answers.
References:
Airasian, Peter W.; Cruikshank, Kathleen A.; Mayer, Richard E.; Pintrich, Paul R.; Raths, James; Wittrock, Merlin C. (2000). Anderson, Lorin W.; Krathwohl, David R., eds. A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. Allyn and Bacon
Chickering, A.W. and Gamson, Z.F. (1987). Seven Principles for Good Practice in Undergraduate Education. American Association for Higher Education Bulletin, 39 (7), p. 3-7. http://www.aahea.org/articles/sevenprinciples1987.htm
Good Practice Guide in Question and Test Design (2004), Pass IT, Retrieved Feb 28, 2014, from http://www.calm.hw.ac.uk/GeneralAuthoring/031112-goodpracticeguide-hw.pdf
Gibbs, G. & Simpson, C. (2005). “Conditions under which assessment supports students’ learning.” Learning and Teaching in Higher Education Accessed online 11 March 2009. http://www.open.ac.uk/fast/pdfs/Gibbs%20and%20Simpson%202004-05.pdf
Patricia Armstrong, Bloom’s Taxonomy, Center for Teaching, Retrieved Feb 28, 2014, from: http://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/#2001