Making the Grade
Creating a traditional online test for the course “Social Media for Workplace Learning” was a challenging task. The traditional test is an excellent way to measure the knowledge retained from a learner on a particular subject. However, as Gibbs & Simpson point out, tests are actually ” very poor predictors of any subsequent performance, such as success at work” (2005). There are likely several reasons for why this is true. For today’s adult learner the reason might be that so little of the work they do is related to the knowledge they have in their head at a point in time. One longitudinal study posed the question ““What percentage of the knowledge you need to do your job is stored in your own mind?” There were remarkably varied results over a twenty year time period: 1986 ~ 75%, 1997 ~ 20%, 2006 ~ 10%. (Jarche, 2011).
This statistic was troublesome for me while designing a test for working adults. The very nature of the content of the course is based on the idea that knowledge is actually networked. The real skill needed by today’s knowledge worker is the ability to source credible information from their network. The most valid learning measure of this course would likely be the evaluation of a learner’s personal learning network (PLN). This sort of PLN assignment would also be consistent with observations from Gibbs & Simpson that find that the quality of learning is actually higher in assignment based courses than exam based courses (2005). The measurement of conceptual learning is not easily captured by traditional test questions and I found that the majority of questions I constructed did not adequately address the deeper level thinking of learners.
Still, there is a place of course for the assessment of declarative knowledge in structured e-learning environments. Many supervisors may just want to know if learners actually completed the course. A simple test can prove that. From there, managers can implement procedures for people to apply this knowledge at work. From a larger perspective, employers will asses the return on investment of structured learning based on employee performance. With this in mind, I designed a simple test that provides feedback to students on their learning but also would demonstrate to supervisors that they completed the course.
Multiple Choice
I created five multiple choice questions representing each of the five topics in the course: social bookmarking, video, RSS, microblogging, and blogging. For some of the questions there were more than answer that could be acceptable, depending on your informed opinion of the subject, but only one answer was correct based on the course content.
Matching
The four matching questions tested the learners knowledge of the main categories of the course and their associated web applications. For these questions I inserted the application icons as visual cues. In an earlier mis-step for this assignment I self hosted the icons for a matching question in Hot Potatoes. Out of curiosity, I decided to keep the icons on my own hosting service and in each question use the source attribute line:
<img src=”url” alt=”some_text”/> .
There is an easier way to insert images by using the Moodle “Files” folder but I learned an interesting workaround if there was ever a problem with this (ie. storage capacity).
Short Answer
I was the least satisfied with the short answer questions. Automating the marking for a short answer question has negative consequences on the quality of questions that can be designed, in my opinion. To assess deeper levels of understanding I would typically ask learners to apply knowledge learned in a course to their own personal work context. However, for this type of question you can only enter so many acceptable answers. As a result, I believe this question type can only be effectively used to test factual knowledge. For the purpose of this assessment however, which is to provide learners with feedback and to prove to an employer that the course has be completed, these fact type questions are appropriate.
Essay
The essay question was alway been an effective question type to assess a student learning on particular course outcomes. I posed the questions with instructions to provide examples in support of the answer so that students would apply their knowledge to to their personal work circumstances. It is this application that truly tests how well a learner understands a subject. The shortcoming of the essay question within Moodle is that the work is submitted only to an instructor. Sometimes, there are good reasons to do this. For this course however, it would make much more sense for participant to share their work with their classmates and the community at large. This pokes at the deeper wound of the LMS where content is restricted only to those registrants with the appropriate permissions. This authority style teaching undervalues the learning that is possible with community experts and and information.
The assessment of learning in the workplace is becoming more complex, especially as many companies begin to move towards social learning. As much as learning and development programs change, there will always be a place, albeit smaller, for the formal traditional test . It remains to be a familiar and widely accepted format to provide feedback to the learners and managers.
References
Gibbs, G. and Simpson, C. (2005). “Conditions under which assessment supports students’ learning.” Learning and Teaching in Higher Education. Retrieved from http://www.open.ac.uk/fast/pdfs/Gibbs%20and%20Simpson%202004-05.pdf
Jarche, H. (2011). “Social learning, complexity, and the enterprise.” Life in Perpetual Beta. [Weblog]. Retrieved from http://www.jarche.com/2011/04/social-learning-complexity-and-the-enterprise/
Image by U.S. Coast Guard
