Multiple choice often gets a bad rap, seen as only being useful for testing memorization and recall of facts. Carefully phrased multiple choice questions can actually be used in the assessment of learning from remembering to synthesis, as the examples below are intended to show. (For more information on these levels, see the page on Bloom’s Taxonomy.) Importantly, multiple choice is ideally suited to online assessment and automatic grading. In addition, many tools (such as quizzing features built into an LMS) offer the ability to provide immediate feedback, contextual to a response selected by a student. In addition, as noted on the Academic Integrity materials, it is possible to ask a large number of multiple choice questions, randomly ordered and/or drawn from pools of questions, as a way to reduce the opportunity for misconduct.
Expand the sections below to see multiple choice question examples at each learning level. At the bottom, you will also find a partial list of some multiple choice dos and don’ts.
Level 1: Remembering
Keywords: recall, list, name, locate, memorize, define,…
- How many degrees of freedom does a spherical bearing have between the inner and outer race?
a. 1
b. 2
c. 3
d. 4
e. 5
Notes: it is generally best practice to arrange answers in order, where applicable; however, to reduce misconduct, it is also recommended to randomize answer choices to give each student a unique-looking question. Unfortunately, some LMSs, such as Canvas, only have a single setting on a test for the answer randomization so it is not possible to turn randomization on for some questions but not others.
Level 2: Understanding
Keywords: Recognize, explain, summarize, paraphrase, discuss…
- A spherical bearing is often used in conjunction with a deep-groove bearing to support a shaft. This is because the spherical bearing...
a. requires less maintenance
b. is easier to install in confined spaces
c. permits angular misalignment of the shaft
d. permits thermal expansion of the shaft
e. has a higher load rating for the same size
Notes: as shown, it is generally recommended to place any repeating text in the question stem rather than the answer choices. In this case, "the spherical bearing..." could have been used to start each answer, but it has been placed in the question stem instead.
Level 3: Applying
Keywords: Compute, use, solve, implement, diagram, calculate…
- Air travels without loss in a converging 1-D nozzle as shown in the figure [figure omitted]. If p2 is 200 Pa lower than p1 and if v1 = 10.0m/s, what is v2?
a. 10.0 m/s
b. 15.3 m/s
c. 18.3 m/s
d. 20.8 m/s
e. 30.6 m/s
- For the bearing shown below [figure omitted], determine the L10 life if the radial load rating is 2150 N and the equivalent radial load is 1000N.
a. 10,000 revolutions
b. 100,000 revolutions
c. 1 million revolutions
d. 10 million revolutions
e. 100 million revolutions
Notes: a box for students to show their calculation work could be included with this question if you wanted. See the notes at the end on Accompanying Answer Boxes and Part Marks. A quick approach to generate multiple questions for a question pool is to revise the parameter values in the question and recalculate the answer choices, ideally making sure each incorrect answer choice represents a plausible error. If you do this, just be sure to set up the pools so each student gets at most one question from that group.
Level 4: Analyzing
Keywords: Determine, organize, compare, categorize,…
- For the machine shown below [figure omitted], rank the possible bearing locations A, B, and C from of most- to least-preferable in terms of bearing life.
a. A - B - C
b. A - C - B
c. B - A - C
d. B - C - A
e. C - B - A
Notes: a short textbox for students to explain their reasoning could be included with this question if you wanted. See the notes at the end on Accompanying Answer Boxes and Part Marks.
Level 5: Evaluating
Keywords: Judge, critique, appraise, test, defend, criticize,…
- In the machine shown below [figure omitted], Bearing A failed prematurely and was replaced 3 times before Bearing B failed. What is the most cost-effective design change that would eliminate the premature failures of Bearing A while still allowing the machine to operate as intended?
a. Replace Bearing A with a spherical bearing
b. Replace Bearing A with an angular contact bearing
c. Switch the locations of Bearing A and Bearing B
d. Relocate Bearing A to the far end of the shaft
e. Replace the U-Joint with a helical beam coupling
Notes: a short textbox for students to explain their reasoning could be included with this question if you wanted. See the notes at the end on Accompanying Answer Boxes and Part Marks.
Level 6: Creating
Keywords: Develop, design, formulate, plan, construct, devise…
- In the machine shown below [figure omitted], Bearing A failed prematurely and was replaced 3 times before Bearing B failed. A colleague recommends checking the following five elements, but is not sure what to do first.
I. Confirm the radial load rating for Bearing A is sufficient
II. Confirm the axial load for Bearing A is sufficient
III. Confirm Bearing A is properly mounted and constrained to the shaft
IV. Confirm there is no unwanted over-constraint on the bearings
V. Confirm there is no excessive shaft vibration prematurely damaging Bearing A
Develop a plan to check some or all of the above by indicating which of the above tests you would conduct and prioritized in the order you would recommend (i.e., choose an option below that contains only the tests you would recommend)
a. I - II
b. I - II - III
c. III - IV - V
d. V - IV
e. V - III
Notes: I included this example for completeness, but I think it is a stretch. This is the level least suitable for using multiple choice with. As above, a short textbox for students to explain their reasoning could be included with this question if you wanted. See the notes at the end on Accompanying Answer Boxes and Part Marks.
Accompanying Answer Boxes and Part Marks
If you wanted, you could add answer boxes with multiple choice questions where students describe their answer in some way in writing. You could leave the contents of the boxes ungraded, but available to check if misconduct is suspected on the exam. Alternatively, if you were so inclined, you could use the written elements to assign part marks for students who get the final answer incorrect; the value here is that you could focus your marking efforts on incorrect questions only. For example,
- Air travels without loss in a converging 1-D nozzle as shown in the figure [figure omitted]. If p2 is 200 Pa lower than p1 and if v1 = 10.0m/s, what is v2?
a. 10.0 m/s
b. 15.3 m/s
c. 18.3 m/s
d. 20.8 m/s
e. 30.6 m/s
- Please provide all supporting calculations for Question 3 in this box OR Please briefly describe the key assumptions and steps in your solution to Question 3 in this box.
Two examples of phrasing are given in Question 4. The first (showing calculations) might be appropriate if students are writing the exam by hand and scanning their paper, while the second (brief written description) would be more suitable for a fully online exam.
Some Multiple Choice Dos and Don’ts
First, Jim Sibley has a fantastic handout on Writing Good Multiple Choice Questions, with tips on constructing questions and interpreting the data that comes out. A list of some other multiple choice dos and don’ts are below.
Dos
- Try to minimize the use of negatives (e.g. “not,” “false,” etc.) in questions, unless doing so makes the question awkward or more difficult to understand
- Use emphasis to highlight key, easily-overlooked words in the question (e.g. the word “not” completely changes a question, so consider emphasizing it as “NOT” instead)
- Use complete but clear and concise writing. Try to minimize the amount of reading required, placing repeating text directly in the question stem (see Q2 from the “Understanding” section above)
- Use qualifiers such as “best example,” “most likely,” “least complete” in the question stem to phrase questions requiring judgement
- Arrange answer choices in sequence (see the Remembering and Applying examples above), unless you will be randomizing answer choices (a good practice to enhance exam integrity) on the platform you are using to deliver the questions
Don’ts (or at least things to avoid)
- Avoid giving away the answer in the question stem:
1. To determine the area beneath a function use an…
…a. Derivative
…b. Product
…c. Integral
- Avoid double negatives in questions (it becomes a logic puzzle, not a test of understanding):
2. Which of the following is NOT a reason for using multiple choice?
…a. Wanting to avoid manually grading questions
…b. Not being able to craft a high-level question
…c. Multiple choice is not an authentic assessment
- Avoid relating one answer to another (and note that the choice letters a., b., c. disappear with some online test platforms)
3. What colour(s) indicate(s) a tomato is ready to eat?
…a. Red
…b. Green
…c. Black
…d. a and b but not c
…e. b and c but not a
- Similar to the above, try to minimize the use of “None of the above” and “all of the above.” Furthermore, with randomized answer choices, all of these phrasings can become very confusing if not totally inaccurate (what happens when “None of the above” appears as the first choice).
- Again related to the above, make sure the question is not open to interpretation (i.e., red tomatoes are most common, but green and black varieties also exist); a better phrasing might be something like “What colour indicates most tomato varieties are ready to eat?”
- Avoid giving away the answer in an earlier question (i.e., you only need look at Q3 to figure this one out!)
4. Which of the following is a common method for determining if a tomato is ready to eat?
…a. Symmetry
…b. Colour
…c. Roundness
| Previous (Rubrics) | Next (High Level and Easy to Grade) |