Good Question Design

Learning Statement: In this article we cover our top tips to help you write good multiple choice questions.
Audience: This article is aimed at all OMBEA users.

Background

There is lots of research on what makes a multiple choice more or less effective. Before we run through our top five, here’s the anatomy of a multiple choice test question:

  • The question is known as the stem.
  • The incorrect answers are known as distractors.
  • The correct answer is known as the key.

With these definitions in mind, here are our top five tips:

1. Start your question with an ‘open question’ word.

Open question words include ‘who’, ‘what’, ‘why’, ‘where’, ‘when’ or ‘how’.

The reason this works isn’t obvious until you try it. Take, for example, a situation where you’re trying to discover your audience’s commuting habits. Ignore our advice and write the question up as:

Did you travel here by car?

Your subsequent answer choices are limited to ‘Yes’ and ‘No’. You’ve alienated everyone that walked, jogged, cycled, used public transport, or some combination of the above. At best people will speak up and point out the omission but in the worst case they’ll remain quietly distracted by the fact that you didn’t include them in the activity.

Now stick an open word in front and the question changes subtly but the effect on the answer options you will start to think of is marked.

How did you travel here today?

You’ve no choice but to come up with a range of options for your group to choose from and, in doing so, you’re more likely to cover all bases.

2. Avoid overly wordy questions or answers by cutting out unnecessary information.

Not only will your slides become harder to read, but your audience is more likely to become confused about the true meaning of your question or answer options. Consider the two versions of the same question below (we’ll let you decide which will work better for yourself):

Kelly Slater, the 11x ASP world surfing champion and ex-Baywatch star in the 1990s, was the face of which global surf brand for twenty-three years before leaving to join a different surf brand in 2014?
1. Billabong
2. Quiksilver
3. Rip Curl
4. Hurley

Which global surf brand did world champion Kelly Slater represent for 23 years?
1. Billabong
2. Quiksilver
3. Rip Curl
4. Hurley

3. Use plausible distractors

When writing a question, make sure your distractors are plausible. Better still, if your distractors represent common mistakes that your students might make, then you’ll get deeper insights into where they are going wrong and therefore how they can improve.

4. Avoid giving clues to the answer in the question

Consider the following question. If you know nothing about surfing you might still get the answer right simply because you know the rules of grammar.

In surfing a peak with lefts and rights coming off it is know as an
1. Point break
2. Reef break
3. A-frame
4. Close-out

5. Avoid negatives

Asking your participants to process a negative statement makes it harder for them to understand your question. Consider the two question variations below. We would recommend the first one for this reason:

  1. Which of the following is not a symptom of CSR?
  2. Which of the following are symptoms of CSR?