Reflection on Assessment task

I have written my first online assessment for a Mentoring in Moodle course titled – Introduction – Learning More about Mentoring.

PURPOSE OF ASSESSMENT
“Formative assessment in computer-supported collaborative learning is intended to give students agency to assess their own and community knowledge advances. For this to happen, assessments need to be designed that both measure and foster deeper inquiry and collaboration.” (Lee, Chan & van Aalst, 2005)  This statement was a starting point for my personal investigation and experiences in setting up assessment tasks within the Mentoring in Moodle course. The assessments I was to integrate into the course needed to fulfill this key purpose, since this is also an underlying purpose to a mentoring program.

Jenkins stated that “formative assessment requires students to put forward their initial ideas and understanding for ‘critique’ and sharing’.” This was my idea of how mentors, persons being mentored and course facilitators would effectively assess the responses to course content being investigated. Jenkins also stated that “peer feedback can help focus learning and be positive for a group of learners” which also resonated with my sense of how course participants would engage in the learning being offered in the Mentoring course. With that in mind, I set up an online journal and discussion forums.

Jenkins elaborated that assessment could be used “for a variety of reasons, including: providing feedback, grading, enabling students to correct errors, motivating students, consolidating learning, applying abstract ideas to practical examples, estimating students’ potential, guiding selection or option choice, giving staff feedback on how effective their teaching has been and providing statistics for internal and external uses.” This extended my concept of assessment beyond the notion of providing marks, which really is an evaluative component. It reemphasized for me the sense that assessment is about the learning, not the person. This is what I needed to remember as I created the assessment questions in the Moodle course.

It was important, as I worked through the question design in relation to the content or purpose of the learning task, that assessment, whether it is formative or summative, involved some form of judgment by someone as to the quality in comparison to some criterion. (van Aalst, Chan, Chan, Wan, Chan, & Teplovs, 2005)  The judgments, where feedback to the question responses was concerned, were mine to establish and difficult to create.

EXPLORING ASSESSMENT
As I reflect on developing the course assessments for Mentoring in Moodle, I tried to keep in mind the 7 principles (Chickering and Gamson, 1987 and Chickering and Ehrmann, 1996) as I designed the assessment tasks.

1. Encourages contacts between students and faculty. – I need to encourage contacts between mentors and the person being mentored, as well as between mentoring groups EA’s, classroom teachers, vice principals, principals, all those mentoring, all those being mentored.
2. Develops reciprocity and cooperation among students. – In the mentoring course, the assessments needed to create space for reflective, collaborative communication that is positive, open and respectful. Some essential groundwork needed to be done to develop these expectations.
3. Uses active learning techniques. – Quizzes, questionnaires and surveys all have their place within this course environment, but I was challenged to find the best use of these tools when I was moving toward using discussion forums, online journaling and wiki/blog portfolio experiences as primary ways of communicating. The quiz tools would quickly provide some feedback to the course facilitators and establish a sense that self-assessment and sharing of information was an essential component of the mentoring experience.
4. Gives prompt feedback. – One essential component to adult learning models is feedback and through the feedback built into the quizzes, questionnaires and surveys, it is able to give effective and timely feedback to course participants.
5. Emphasizes time on task. – Since the educators involved in this online course are very busy and have little time to spare, the elements have to be short, concise, clearly outlined and can be completed in 60 minutes or less. In that way, the course participants can get in, get one or two module elements completed and get back to work or personal matters. For this reason, each quiz was limited to a maximum of 10 questions, with each question being focused on quick response time or limited content.
6. Communicates high expectations. – As I began establishing the questions for the initial modules, I reflected on the need to set a tone or focus for the task being undertaken. For this reason, I began to integrate an inspirational quote to introduce the question, along with a short passage to outline the focus for the question.
7. Respects diverse talents and ways of learning. – Since my work in education hasexposed me to diverse learners (both students and adults) I am aware of the need to provide specific content, direction and approach for varying employee groups. This is not only determined by the various learning styles of individuals in each group, but the contractual requirements and working conditions for each group. The types of questions within this Moodle course were designed to not only respect these conditions, approaches and requirements, but also to break open the conversations between the employee groups within specific educational roles.

WEAKNESSES OF ASSESSMENT
After rereading Gibbs and Simpson (2005) my awareness of the importance placed on feedback was heightened and made me realize my own unique biases in this area (knee to knee, eye to eye, sitting at a table in a confidential setting, listening before speaking). I tried to integrate a conversational style of response to the feedback I used in the multiple-choice questions. Providing effective feedback is an area of personal weakness in assessment that I will investigate further. One question I have relates to how to integrate the effectiveness of rubrics into the online quiz feedback feature. The importance of awareness of personal biases when creating effective feedback will continue to challenge me when creating questions and feedback that benefit user groups participating in the Mentoring in Moodle course.

ASSESSMENT VS. EVALUATION
I am aware that assessment tools must be carefully matched and appropriately applied to the learning activities (Lombardi, 2008), even when there is no specific mark or outcome. It was challenging to apply the specific expectations for this particular assignment (eg. 3 multiple choice, 2 short answer, 2 essay questions etc.) to the content and process that I had envisioned for the Moodle course. In the end, I was able to make the questions work in specific ways to provide key information to the course facilitator, but not without considerable rethinking the question content and focus.

I was encouraged to persevere in my efforts to make the assessment quiz not about marks or evaluation when I read Jenkins’ (2004) comment that “over-reliance on assessment as an extrinsic motivator can lead to surface learning approaches”. Gibbs (2005) commented that in the “absence of marks it has been reported that students read feedback much more carefully and use it to guide their learning”. My notion to engage the course participants in reflective thinking and deeper learning were on the right track. The challenge was how to write a question that would create these learning conditions that did not have a number grade attached.
My comfort with the quiz format of assessment, for the Mentoring in Moodle course, continues to be problematic in terms of where and when to use this specific tool. My bias toward this form of assessment was clearly tied closely to the concept of evaluation.

My first module and assessment was based on Anderson’s (2008) statement that it was important to “measure learners’ internet efficacy – not just internet skills but the user’s strong sense of internet efficacy that enables them to adapt to requirements of working online.” From my own experience with the ETEC 565 quiz on HTML, I reflected on the need for course participants to understand the basic set of skills required for engagement in the online Mentoring course. The feedback that I provided for each response was intended to establish a core level of expertise with some mechanism for participants to share their expertise.
NEXT STEPS
Portfolios have not yet been integrated into the Mentoring in Moodle course and will be established to provide participants with a repository for a selection of best items (e.g., papers, diaries, drawings) accompanied by reflection statements.

Synchronous methods of assessment are not yet present within the Moodle course and will need to be further investigated before being applied to specific content areas in the Mentoring process.
The assessments within the Moodle course need to be “formative, process oriented, collaborative, and integrated with instruction”. (Lee, et.al, 2005) I will continue to embed frequent assessment type tasks within each learning module to engage the course participants in some form of reflective, collaborative and novel forms of self, peer and group assessment.

I will continue to investigate, explore and apply concepts in assessment based on the key areas outlined by Lee, Chan & van Aalst on the Virtual Design Center. They include assessment of learning AND assessment for learning, assessment of individual AND collective learning and assessment of content AND process. Assessments will “focus on both individual and collective growth; be formative, embedded, and concurrent so they can serve scaffolding purposes for learning and collaboration; support student agency. Turn over the responsibility of assessment to students so they can have increased agency as they examine their own and community progress; provide explicit criteria; and assess both processes and products.” (Lee, et.al, 2005)
References
Anderson, T. (2008). Towards a Theory of Online Learning. In: Anderson, T. & Elloumi, F. Theory and Practice of Online Learning. Athabasca University. Accessed online 3 March 2009 http://www.aupress.ca/books/120146/ebook/02_Anderson_2008_Anderson-Online_Learning.pdf

Chickering, A.W. & Gamson, Z.F. (1987). Seven Principles for Good Practice in Undergraduate Education. American Association for Higher Education Bulletin, 39 (7), 3-7. Accessed online 11 Mar 2009 http://www.aahea.org/bulletins/articles/sevenprinciples1987.htm

Chickering, A.W. & Ehrmann, S.C. (1996). Implementing the Seven Principles: Technology as Lever. American Association for Higher Education Bulletin, 49(2), 3-6. Accessed online 11 Mar 2009 http://www.aahea.org/bulletins/articles/sevenprinciples.htm

Gibbs, G. and Simpson, C. (2005). “Conditions under which assessment supports students’ learning.” Learning and Teaching in Higher Education Accessed online 11 March 2009 http://www.open.ac.uk/fast/pdfs/Gibbs%20and%20Simpson%202004-05.pdf

Giving Prompt Feedback. 7 Principles Collection of TLT Ideas Retrieved June 2, 2011 from http://www.tltgroup.org/Seven/4_Feedback.htm

Jenkins, M. (2004). “Unfulfilled Promise: formative assessment using computer-aided assessment.” Learning and Teaching in Higher Education. 1, 67-78.

Lee, Eddy Y.C., Chan, Carol K.K., & van Aalst, Jan. (2005) Students Assessing Their Own Knowledge Advances in a Knowledge-building Environment, Paper presented at the 2005 Computer-Supported Collaborative Learning Conference in Taipei, Taiwan. Retrieved on 11-01-29 from http://vdc.cet.edu/entries/lee.htm

Lombardi, M. (2008) Making the grade: The role of assessment in authentic learning. Educause, Jan. 2008 Retrieved June 4, 2011 from: http://net.educause.edu/ir/library/pdf/ELI3019.pdf

The TLT Group. “Seven Principles” Collection of Ideas for Teaching and Learning with Technology. Accessed online 10 April 2009. http://www.tltgroup.org/Seven/Library_TOC.htm

van Aalst, J., Chan, Yuen Yan., Chan, Carol K. K., Wan, Wing-San., Chan, Chi-Fung., Teplovs, Christopher. Development of formative assessment tools for knowledge building.