Reducing dropouts in online experiments
I believe we all have face the same problem when running synchronised online experiments so I would like to share what we do at Experimentics.
At the very beginning we thought about:
A. Using bots
B. Finishing the experiment if there was a dropout on any group
From a technical point of view that solution is feasible but I think that from a research perspective most of the times these are not good options.
For that reason, we explored new practices and found out that the key was to simulate, as much as possible, a physical lab environment.
Here is what we do:
1. Objective: Ensure our online experiment looks as serious as if it was a lab experiment.
Action: When recruiting we inform subjects about the characteristics of the experiments. If it is synchronised, we explain them how important for us their active participation (Which will be necessary if they have any problem or they just find boring to be waiting for others’ decisions).
2. Objective: Make sure subjects are really interested and available:
Action: Before the start of the experiment, we qualify subjects to be able to know how interested are they in participating (we do phone calls, online interviews, etc…). Only those who are really interested will receive the invitations for the experiment. Those who does not show a real interest/commitment will be disqualified.
3. Objective: Simulate participants’ questions and real time troubleshooting as if it were a physical lab.
Action: When we set date/time of the experiment we provide an alternative and anonymous way to have live communication with them. It allows us to have real time communication and provide direct support in case they have any technical/misunderstanding problem.
We believe that the best way to reduce dropout is to have a strategy to engage participants and make them easy to rejoin the experiment if they have any problem.
I hope it helps!