What does the research say about audience response systems in medical education?
A Best Evidence Medical Education (BEME) systematic review looked at comparative studies in health professions education. Of 1,013 titles screened, 21 studies were included, and 14 of the 21 reported a statistically significant improvement in knowledge scores with an audience response system. The review adds two caveats: the differences were larger in non-randomised studies, and larger when the comparison was a non-interactive lecture. In other words, part of the effect comes from making a lecture interactive at all.
A UK randomised controlled trial in undergraduate anatomy makes the same point from another angle. Seventy medical students received the same lecture, one group with an audience response system. Immediately afterwards, the response-system group scored 17.3 out of 20 against 15.6 in the control group, but 10 weeks later there was no difference between the groups.
So what lasts? US work on test-enhanced learning in medical education, from Washington University in St. Louis, points to three things: questions given often and spaced out in time, questions that require effortful recall, and feedback. That argues for short retrieval questions repeated across a module, with feedback each time, over a single quiz at the end of a lecture.
A review of the pros and cons of response systems in health professions teaching lists the upside: real-time feedback on knowledge gaps, peer debate about why one answer is better, and anonymity that removes the fear of looking foolish in front of peers. It also lists the costs: preparation time for well-formulated questions, less lecture time for content, and learners who object to grades being attached to their answers. For a wider look at the evidence across subjects, see what the research shows about clickers in the classroom.
How do you run case-based teaching with an audience response system?
Clinical reasoning is a chain of decisions, and each decision point can be a question. A format for a 50-minute session with three short cases:
- Present the vignette (2 minutes). Age, presenting complaint, two or three key findings. Stop before the decision.
- Ask for a commitment (1 minute). "What is your next step?" with four or five plausible options. Keep the results chart hidden.
- Look at the split before you speak. If 85% chose the right answer, confirm it briefly and move on. If the room is split between two options, that split is your teaching point.
- Peer discussion (2 to 3 minutes). Ask learners to convince the person next to them. In a study of undergraduate genetics students at the University of Colorado, peer discussion improved understanding even when no one in the discussion group had known the right answer at first.
- Re-vote and reveal. Show both charts side by side and ask what changed people's minds.
- Explain the distractors. Spend more time on why the popular wrong answer is wrong than on why the right one is right.
- Unfold the case. Give the result of the investigation and ask the next question.
Three cases with two or three decision points each gives you six to nine questions.
How do you write good clinical questions for a response system?
Write questions that make the learner decide, as they would on the ward, rather than recall a fact they could look up. Patterns that work:
- Next step: "A 67-year-old with sudden breathlessness and a normal chest X-ray. What do you do next?"
- Priority: "Which of these findings needs escalating first?"
- Interpretation: show an ECG strip, blood gas or image and ask what it shows.
- Error spotting: show a prescription or handover note with one problem and ask where it is.
- Confidence: after a hard question, ask "How sure were you?" A room that is confident and wrong needs a different conversation from one that is unsure and right.
Build distractors from real mistakes you have seen learners make. Four options is usually enough. Avoid "all of the above", and avoid options that are obviously absurd, because they turn a four-option question into a two-option one.
Tests that require effortful recall produce better retention than recognition tests such as multiple choice. Multiple choice is recognition, so after the vote, ask one or two learners to explain their reasoning out loud. That adds the recall step.
Should answers be anonymous or named?
Use both, for different jobs, and tell learners which mode is on.
Anonymous is for teaching. Clinical teaching has a strong hierarchy. When a consultant or registrar (in the US, an attending or senior resident) is in the room, juniors tend to wait for their answer. With anonymous voting, the first-year student and the consultant press a key at the same moment, and you see what the whole room actually thinks.
Named is for records. Competency checks, mandatory training and CPD often need proof that a specific person answered a specific question. In named mode each answer is saved per participant, and you can export a report afterwards.
Put "Anonymous" or "Named" on the question slide itself, so nobody has to guess whether their answer is recorded.
How can you use it in grand rounds, simulation debriefs and CPD?
Grand rounds. In a packed auditorium, few people will speak up in front of colleagues. Open with one question about the case before the presenter reveals the diagnosis, and close with the same question so the room sees how its view moved.
Simulation debriefs. After a scenario, ask anonymous questions before the spoken debrief: "When did you first think the patient was deteriorating?" or "Did you feel able to speak up?" Use the answers to decide what the spoken debrief covers first.
Compare professions. In interprofessional sessions, ask a background question first (nursing, medicine, pharmacy, year of training). You can then split any answer by group and see where professions read the same case differently.
CPD and CME. Continuing professional development in the UK and continuing medical education in the US both need a record of who took part. Two or three questions per topic add retrieval practice to an update talk, and a named report gives the training record.
Course evaluation. Ask your evaluation questions in the last five minutes while everyone is still in the room, so the answers come from nearly everyone in the room rather than from whoever opens a survey email a week later.
Clickers or phones in clinical teaching?
Phones are fine in a university lecture where learners already use them and the Wi-Fi is reliable. They also allow free-text answers and word clouds.
Dedicated clickers suit settings where phones are kept away, where hospital Wi-Fi is locked down for guests, or where you want a full auditorium ready in seconds without log-ins. OMBEA ResponsePads, for example, use their own radio link to a USB receiver, need no Wi-Fi or app, and show results live inside PowerPoint. They handle up to 2,000 participants in one session. Our guide to clickers or phones covers the trade-offs in more detail.
What are the most common mistakes?
- Too many questions. A question every three slides becomes a quiz. Aim for one decision point every five to ten minutes.
- Questions only at the end. The evidence favors spacing. Ask early, ask in the middle, ask again later in the module, term or semester.
- Grading every answer. The review above found students disapproved of grades being attached to participation. Keep teaching questions ungraded and anonymous.
- Revealing the chart too early. Showing results while voting is open can pull late voters toward the majority.
- Ignoring the result. If half the room got it wrong, change your plan.
- No feedback. The testing research names feedback as one of the three conditions. Always explain the answer.
For more quick ways to see what a room has understood, see these check for understanding techniques. You can see how question slides, grouping by profession and reports work on the features page, and how this looks in a teaching session on the medical education page.
