Conducting a successful usability study, particularly for summative validation of medical products, can require a great deal of preparation and coordination of many moving parts. Whether it’s successfully simulating your target use environment or nailing down the exact scenarios and tasks to be presented, everything must come together. After all, nobody wants to waste the time and money it requires to complete a study if the result doesn’t align with what was intended.

One of the best ways to ensure success is to perform a pilot study with your protocol before starting the actual study. A pilot study is like a miniature version of the actual study conducted with far fewer participants. This approach helps confirm the study design will work as expected, the desired data can be obtained, the participants understand the task prompts, and more. For best results, the pilot study participants should be as close to actual participants as feasible; the same applies to the use environment. You’ll also want to conduct the pilot study early enough before the actual study to ensure there is sufficient time to update the protocol according to the findings.

Many people use the terms “dry run” and “pilot study” interchangeably, but there is technically a difference. A dry run is done to practice the protocol both before the pilot study and after the finalization of the protocol, ensuring the moderator and notetaker/analyst are comfortable with the product and script. This activity can be done with proxy participants or with no participants at all. What’s notable about a dry run is that it can be done in place of a pilot study if the participant population is expensive or difficult to recruit, if the protocol is very simple or essentially a repeat of a previous study, or if the timing does not permit a full pilot study.

While there can never be guarantees, these are proven ways to exponentially increase the odds of a study’s success. Taking the time upfront will ultimately save time and effort when it’s time to conduct the actual study. At Kaleidoscope, we've conducted hundreds of studies, so our process is as rigorous and dialed in as possible. If you're looking for a rock-solid research partner, we're ready to roll up our sleeves.