EMERGO BY UL SUMMARY OF KEY POINTS:

• Medical device usability test administrators are often challenged by no-show participants.
• Steps such as overrecruiting, rescheduling and partial participation allowances can help mitigate this challenge.

It’s 1:25PM, and you’ve just jotted down “P6” – the 1:30PM participant’s unique identifier – at the top of your moderator’s checklist in preparation for your next test session. Five minutes go by, and it’s time for the scheduled session – surely they’ll walk in the door any second now. Another 5 minutes goes by – 1:35PM –  maybe your participant got stuck in traffic or is having difficulty finding a parking spot. You call and ask your recruiters if they’ve heard from the participant recently. At 1:45PM, your recruiters call back and tell you that the participant is sick and won’t be able to participate as scheduled. Now what?

Despite meticulous planning, including real people in usability tests means that no-shows, last-minute participant cancellations, and late participants are unavoidable. But, there are several steps that you can take to prevent no-shows from becoming show-stoppers:

1. Overrecruit. We don’t like to be pessimists, but it is good to assume that you’ll have at least a couple no-shows or last-minute cancellations during a usability test. So, always recruit a couple more participants than you’ll need. For a 15-participant usability test, we typically recruit 18 participants. In this case, more is better; you can always cancel extra participants after you meet your quota. Just make sure that you provide them with sufficient notice and compensation, if appropriate.
2. Give the participant a second chance. Unfortunately, things happen. It’s very possible that your participant had every intention of attending the test session as scheduled but woke up feeling under the weather. As soon as you or your recruiter can get in touch with the participant, ask if he or she is able to reschedule for later in the day or week as the test schedule allows. It’s much easier to reschedule an already-screened and qualified participant than to try to find a replacement. However, keep in mind that if a rescheduled participant cancels or “no-shows” a second time, it might be wise to place the individual on a “do not recruit” list for future studies due to repeated unreliability.
3. Determine if partial participation is an option. In some cases, a participant will call 15 minutes after the scheduled session start time and inform you that they can be at the facility in another 15 minutes – a total of 30 minutes late. In this scenario, reassess your test goals and consider whether having the individual participate in a portion of the session still adds value. In the case of a 90-minute formative test session, you could still likely gain valuable insights during the remaining one hour. On the other hand, for a 60-minute HF validation test session, the remaining 30 minutes might not provide enough time for the participant to complete all test activities required during HF validation. This makes the decision easy: you’ll need to reschedule the participant for another time, or recruit a new participant altogether.
4. Recruit “floaters.” If your project budget allows for it, consider recruiting what we refer to as “floaters”– screened and qualified participants that we pay to remain on “standby” for the day in the event that we have a no-show or cancellation. These back-up participants typically remain on-site for the entire testing day – or stay within a 10-15-minute drive of the facility – so that we can call upon them at a moment’s notice to participate if needed.

Importantly, there are also several approaches you can take to help prevent no-shows from occurring in the first place. A couple key approaches are as follows:

1. Create an accommodating test schedule. We all know the stress associated with running late. Although we don’t have great (or any) insights into our participants’ personal schedules, strive to create a test schedule to accommodate your participants as much as possible. For example, create a schedule with plentiful evening sessions for adolescent or day-time healthcare professional participants. Such flexibility will help ensure that participant attendance is not dependent on an on-time school bus or a nurse’s last set of rounds.
2. Avoid the big holidays. Yes, a four-course Thanksgiving dinner might render individuals immobile for a couple of hours. But, holidays can actually have a greater impact on usability test participants. When preparing for or enjoying a holiday with family and friends, an individual’s upcoming test session is unlikely to be top of mind. Or, individuals might be traveling and – despite best intentions – miss the recruiter’s confirmation call reminding them of their scheduled session for the following week. Beginning your test sessions at least one week after a big holiday provides sufficient time for confirmation calls and for individuals to settle back into their normal routines.

Rachel Aronchick is Managing Human Factors Specialist at Emergo by UL Human Factors Research & Design.

Additional medical device human factors engineering, design and usability resources:

• Human factors engineering (HFE) user research for medical devices and IVDs
• HFE design and prototype development support
• Medical device evaluation
• Webinar: HFE for medical devices