How to reduce dropouts in online experiments

I believe we all have faced the same problem when running synchronised / multiplayer online experiments so I would like to share what we do at Playstudies.

At the very beginning we thought about:

A. Using bots
B. Finishing the experiment if there was a dropout on any group

From a technical point of view, that solution is feasible but I think that from a research perspective almost every time these are not good options.

For that reason, we explored new practices and found out that the key was to simulate, as much as possible, a physical lab environment.

Here is what we do:

1. Objective: Ensure our online experiment looks as serious as if it was a lab experiment.
Action: When recruiting we inform subjects about the characteristics of the experiments. If it is synchronised, we explain them how important is for us their active participation (Which will be necessary if they have any problem or they just find boring to be waiting for others’ decisions).

2. Objective: Make sure subjects are really interested and available:
Action: Before the start of the experiment, we qualify subjects to be able to know how interested are they in participating (we do phone calls, online interviews, etc…). Only those who are really interested will receive the invitations for the experiment. Those who does not show a real interest/commitment will be disqualified.

3. Objective: Simulate participants’ questions and real time troubleshooting as if it were a physical lab.
Action: When we set date/time for the experiment we provide an alternative and anonymous way to have live communication with them. It allows us to have fluid communication and provide direct support in case they have any technical/misunderstanding problem.

We believe that the best way to reduce dropout is to have a strategy to engage participants and make it easy for them to rejoin the experiment if they have any problem.

I hope it helps!

Ignacio Alastrué