Writing Multiple-Choice Questions That Teach as Well as Test

Writing a good multiple-choice question comes down to four decisions: what the question is for, how the stem asks it, which wrong options sit beside the right one, and what the learner reads after answering. Start from a learning objective. Write the stem as a complete question. Build the wrong options from mistakes learners really make, and stop at three options unless you have more good ones. Then write feedback that explains why each answer is right or wrong.

Most guides to writing multiple-choice questions treat them as a measuring instrument: rules for fair, reliable exams. Those rules matter, and we use them below. But in an online course, most quiz questions aren’t there to grade anyone. They’re practice. The research on how people learn from tests adds two things the exam guides leave out: the wrong options do some of the teaching, and the feedback decides whether learners keep the right answer or the wrong one.

Why a quiz question can teach

Answering a question changes what you remember. This is the testing effect, also called retrieval practice: recalling information from memory strengthens it more than reading it again. A meta-analysis by Olusola Adesope and colleagues pooled the studies and found that “practice tests are more beneficial for learning than restudying and all other comparison conditions”, with the size of the benefit depending on the features of the tests (Adesope et al., 2017).

Multiple-choice questions have a reputation as the weak form of practice, and there’s something in that. Recognising an answer is easier than recalling it, and Jeri Little and Elizabeth Bjork note that cued-recall questions (where learners produce the answer themselves) have usually led to better retention of the tested information (Little & Bjork, 2015). But their experiments found something multiple-choice can do that recall questions don’t. When the wrong options were competitive, meaning plausible enough that learners had to think about them, answering the question also improved learners’ later answers to related questions, especially questions about the wrong options themselves. When the wrong options were obviously wrong, that benefit disappeared.

A later study by Erin Sparck and colleagues explains why:

So the work a learner does to rule out a plausible option is part of the learning. A question with one sensible answer and three throwaways asks for none of that work.

These are lab studies with university students, short passages and short delays (a few minutes before the final test), so we’d treat them as moderate evidence for the mechanism rather than a promise about any particular course. The practical direction is clear enough to design for.

How to write a multiple-choice question, step by step

We’ll use one running example throughout: an invented self-paced course for small-business owners on managing cash flow. One of its objectives is: Given a month’s figures, identify the main cause of a cash shortfall.

1. Start from the objective, and decide what the question is for

Every question should test something an objective names. If you haven’t written the objectives yet, do that first: our guide on how to write learning objectives covers it. An objective that says “identify the cause” needs a question that asks learners to identify a cause, not to recall a definition of cash flow.

Then decide whether the question is practice or a graded check. Practice questions sit inside or straight after a lesson, can be retried, and carry detailed feedback. Graded questions decide a certificate or a pass, so you’ll want more of them, drawn from a pool, with feedback that doesn’t give away the next attempt. Most course quizzes are practice, and the rest of this guide assumes that.

Example: the cash-flow objective gets three practice questions at the end of the lesson and two graded questions in the module test.

2. Write the stem as a complete question

The stem is the question itself. A good one makes sense before the learner sees any options: cover the options with your hand, and a learner who knows the material should be able to answer. Put the shared wording in the stem rather than repeating it in every option, cut anything that isn’t needed to answer, and avoid negatives (“Which of these is NOT…”). If you must use one, make it impossible to miss.

To test application rather than recall, give the stem a small scenario: a set of figures, a customer’s email, a screenshot of a report. The learner then has to use what they know.

Example. Weak: “Cash flow is…” Better: “Mia’s café made a profit of US$4,000 in March, but her bank balance fell by US$2,500. Her supplier now wants payment in 7 days instead of 30. What is the most likely cause of the shortfall?”

3. Write the right answer, then wrong options from real mistakes

Write the correct answer first. Then write the wrong options, which test writers call distractors, from mistakes learners really make. The best sources are the wrong answers learners give to an open question, the misconceptions you hear in live sessions, and the errors you find yourself correcting in assignments. Each distractor should be something a learner who half-understands the lesson could believe.

This is where the lab evidence above earns its keep. Plausible distractors make learners retrieve why each option is wrong, which is what produced the benefit on related questions. Implausible ones turn the question into a reading test.

Example for Mia’s café:

• She is paying her supplier sooner than before, so cash leaves before sales replace it. (correct)
• Her profit figure must be wrong, because profit and cash should match. (the core misconception)
• Her prices are too low to cover her costs. (a real problem for many cafés, but her profit says it isn’t this one)

4. Stop at three options unless you have more good ones

Many quizzes default to four or five options. The extra ones are often filler. In an analysis of 737 items from high-stakes medical exams, about 70 per cent of five-option items had at least one distractor chosen by fewer than 1 per cent of candidates (Rogausch et al., 2010). An option nobody picks does no work: it doesn’t test anything, and it doesn’t prompt anyone to think.

Catherine Loudon and Aide Macias-Muñoz tested the number of options directly, in the normal exams of a human physiology course with 662 students. Each question appeared with three, four or five options in different versions of the exam. With more options, questions became only slightly harder (84 per cent answered correctly with three options, 80 per cent with five), and the measures of how well questions separated stronger from weaker students barely moved (Loudon & Macias-Muñoz, 2018).

In practice, we’d write three options as the default: the right answer and the two mistakes that matter most. Add a fourth only when there’s a fourth mistake learners genuinely make. Fewer options also means less reading, which matters on a phone.

5. Take out the clues

Learners who don’t know the answer can often find it anyway, from the way the question is written. Thomas Haladyna, Steven Downing and Michael Rodriguez reviewed 27 educational testing textbooks and 27 research studies and reviews to build a taxonomy of 31 multiple-choice item-writing guidelines (Haladyna et al., 2002). The guidelines universities hand their staff today repeat the same core rules. The ones that matter most for a course quiz:

• Keep options a similar length. The longest, most carefully qualified option is often the right one, and test-wise learners know it.
• Keep the grammar parallel. If the stem ends in “an”, an option starting with a consonant gives itself away.
• Avoid absolutes. “Always” and “never” in a distractor signal that it’s wrong.
• Avoid “all of the above” and “none of the above”. Learners can pick “all of the above” by spotting two true options, and choosing “none of the above” shows they rejected the options, not that they know the answer.
• Vary the position of the right answer, or let the platform shuffle the options, unless their order carries meaning (numbers, dates).
• Keep questions independent, so one question’s stem doesn’t answer another.

Example: in the first draft of the café question, the correct option was the only one that mentioned “timing”. Rewording the other options to talk about timing too removed the clue.

6. Write feedback that explains

What learners read after they answer matters as much as the question. Andrew Butler and Henry Roediger gave students a multiple-choice quiz with no feedback, immediate feedback or delayed feedback, then a recall test a week later (Butler & Roediger, 2008). Without feedback, the quiz helped, but learners also reproduced some of the wrong options they’d chosen. With feedback, they got more right and those errors fell back to the level of students who never took the quiz.

With feedback, a multiple-choice quiz lifted recall a week later from 14% to as much as 56%

Proportion of answers correct, and of wrong answers taken from the quiz's lures, on a cued-recall test one week after a multiple-choice quiz (forced report, averaged across study conditions) · % of final-test answers

CorrectLure intrusions

What the feedback says matters too. In two experiments by Butler, Namrata Godbole and Elizabeth Marsh, students answered short-answer questions and then saw either the correct answer or the correct answer with an explanation (Butler et al., 2013). Two days later, both groups did equally well on the same questions. On new questions that asked them to apply the idea, the explanation group did better: 45 per cent correct against 30 per cent in the first experiment. That study used short-answer rather than multiple-choice questions, but the lesson carries over: if you want learners to use an idea in a new situation, tell them why the answer is right.

For each option, write one or two sentences: why it’s right or wrong, and the rule that gets you there. Our library entry on the characteristics of effective feedback covers the wider evidence, including the timing question.

Example, for the learner who chose “her profit figure must be wrong”: “Not quite. Profit and cash can differ for weeks at a time, because profit counts sales and costs when they happen, and cash counts money when it moves. Mia’s supplier now takes cash 23 days earlier, before her sales replace it. See ‘Profit is not cash’ in Lesson 2.”

7. Spread the questions through the course, and mix in recall

A single quiz at the end of a module is the least useful place for practice. Put two or three questions at the end of each lesson, and bring back questions from earlier lessons in later quizzes, so learners retrieve the same idea more than once. Let learners retry practice questions, ideally with the options shuffled.

Because recall questions usually beat multiple-choice for remembering the tested fact itself, use both. Ask learners to write or say an answer first where your platform allows a short-answer or reflection question, and keep multiple-choice for the decisions where the wrong options are worth thinking about. In a long lesson, a short quiz is also a natural break: our entry on cognitive load explains why splitting content into segments helps.

Example: the cash-flow course asks learners to write one sentence on why profit and cash differ before the multiple-choice questions, and the café question returns, with new figures, in the module 3 quiz.

8. Check how each question performed

Once real learners have answered, look at the results question by question. Three numbers are enough for most course builders:

• The proportion who got it right. Almost everyone right can mean the question is too easy or gives itself away. Almost everyone wrong can mean a confusing stem, a wrong key or a lesson that didn’t teach it.
• How often each wrong option was chosen. An option nobody picks should be replaced with a better mistake, or cut.
• Which option the strugglers chose. A popular wrong answer is a map of the misconception, and the lesson should tackle it head on.

Course platforms differ a great deal in how much of this they show. Before you choose or change one, check in its current documentation, on your plan:

• whether you can write feedback for each answer option, not one message for right and one for wrong
• whether quizzes can draw questions from a pool and shuffle the options
• whether learners can retake a practice quiz, and whether you can limit retakes on graded ones
• whether reports show results per question and per option, not just a score per learner

Our online course design checklist includes these checks alongside the rest of a module review.

What to do next

Take one practice quiz from your course. For each question, cover the options and check the stem stands alone, replace any option almost nobody would choose with a mistake a real learner makes, cut to three options where the fourth is filler, and write one sentence of feedback for each wrong answer saying why. Then run it past two learners before you rewrite the rest.