Should we stop doing evaluations and fieldwork for environmental and development projects during COVID-19? In many cases, yes. Everyone needs to follow the advice of health professionals, and completing a study is often not worth the risks it poses. Things can be tricky with physical distancing but some evaluations and fieldwork can go on safely if we think through the protocols.
We have come up with a few tricks and strategies with which we and clients are content. We share this with a hope they are useful. Note that a lot of our work is in the context of low and middle-income countries, so that is how we approach this topic.
Phone calls and video conferencing for individual and group interviews with professionals. Three weeks into physical distancing, almost all professionals including government officials in developing countries are getting used to meeting remotely. We use the same semi-structured interview tools we usually use, just over the phone. The key is to have the contact details of the respondents from the client or our networks.
There are a few drawbacks to remote interviews. There are some very old studies that noted the different ways that respondents may answer questions on the phone compared to in-person (see Groves and Kahn 1979, but times (and people) have changed, and video calls let us see one another's faces and build a bit of rapport. COVID-19 has pushed many people into remote work and they are quickly becoming accustomed to it.
A more recent study shows that telephone (not even video calls) interviews can have many advantages. Block and Erskine (2012) find that in a developed country context, telephone interviews:
provide a better sense of anonymity for the respondent,
are optimal for open-ended questions because respondents don't have to remember the choices and allow the respondent to think through answers with less pressure, and
allow for better sampling.
A lot of our work is on land-use change. We often ask respondents to comment on visuals like maps in interviews. Even this is manageable if a respondent has easy access to a screen on a computer or tablet and we e-mail the map in advance, but this is impractical for many respondents. Also, if remote interviews are longer than an hour, they can start to wane. Breaking it into two sessions is one way out, or just focus the interviews more, which is not always a bad thing. Our biggest concern with video or audio calls is a bad connection, so we always have a back-up internet connection at the ready, but the respondent's connection is out of our control.
Interviewing community members is a different story. We often do group interviews (often incorrectly referred to as focus groups) at the community level. Obviously, bringing people together is inappropriate in the context of physical distancing. We turn to clients to help us identify someone who already lives in the community to help with a brief closed-end survey that can be done with individuals.
We set the following protocols to make sure everyone is safe:
The village head should formally give consent to do data collection this way and ensure that there are no known cases of COVID-19 in the village. If there are any suspected cases, we stop immediately.
Select the enumerator to conduct the surveys. He or she can be a NGO staff person, teacher or village secretary who already lives in the village. We train them before data collection.
Enumerators must wear masks at all at times while doing data collection.
The enumerators and respondents have to maintain a 2m distance from one another
The interviews must be conducted outdoors.
The enumerators and respondents must not make physical contact and not exchange any materials.
The enumerators must wash her or his hands with soap and after every interview.
The enumerators’ clothes must be washed immediately after returning back to her or his house.
There must be a cellular signal that can be obtained without entering another village.
Regular contact between the evaluation team leaders and enumerators.
The enumerators collect responses either directly in the mobile app (which is available offline) or to paper and transferred to the app nightly. Data is uploaded at least every two days.
From a cost perspective, hiring locally-based enumerators and having no travel expenses or travel time for consultants, this method is likely cheaper than place-based data collection, even when the extra time of training and working with enumerators is factored in.
This is not ideal, but we face an extraordinary time during COVID-19 and we need to quickly adapt. Some potential biases may occur if the enumerator is a person who is familiar with the respondents or involved in the implementation of the project. This bias can be reduced through both training of the enumerators and making a clear opening statement that welcomes critique (e.g. there will be no repercussions in terms of future involvement in other projects), however, that does not solve all the problems. Obviously, sensitive or highly personal data could not be collected this way. We are also challenged with how to engage women in interviews without a man there. If two women are from the same household, they could be interviewed together, but otherwise, it can be difficult to ensure that women feel free to speak in an interview, a situation not helped by being outside and possibly with fewer people leaving the village during the day. This is perhaps a compromise that we need to accept in this challenging time.