By vicsolbert | June 9, 2014 | 1 Comment
So my first visit to the field revealed a need to re-evaluate my survey strategy. While I wanted to incorporate focus group discussions, I had hoped to also do a number of more in depth personal interviews. There were some serious flaws to this plan however.
First, I was accompanied to the field by three other staff members of the CREL project – my escort from the main office, a site manager, and a field officer. This made us a rather large overkill group. So it is understandable that people took notice of us. The real problem though, was the white girl.
Despite our large group, community members were largely familiar with the other people accompanying me and didn’t really pay them much attention. I however, was EXTREMELY interesting. Children especially, but all ages would start to follow us or wander over to join in our group once they saw me, unless they were really in the middle of something. So even if we started talking to two or three people, this quickly grew by a dozen or more. Only a couple would actually join the discussion, most just watched and listened. You would think many who weren’t actively participating would lose interest after a while, but they stuck around for surprisingly long times – half and hour to an hour.
These crowds of onlookers definitely changed the interview dynamic, but mostly just for me. The original interviewees still seemed perfectly comfortable speaking their minds, and had no problem dealing with additional voices in the discussion. I was the only one that got really flustered. There were also some translation issues – I have no idea how much information I missed or what questions they were even actually asked. As the groups grew lots of people talked at once so I’m sure I missed many peoples comments. All of this should of course not have been unexpected to me. It is completely understandable I would present an object of interest – I am obviously a visitor outside of the norm, which means something must be happening outside of the norm and they want to know what that is. I was extremely happy to find that people seemed very enthusiastic to talk to me and had very thoughtful feedback. Still, it all certainly begs the question – should I be the one doing this?
All of our classes on impact evaluation, monitoring and assessments stress the importance of capturing an unbiased sample of the truth. As an obvious outsider who causes such a disruption, how can any information I collect possibly be unbiased? Training the local staff to conduct the survey would likely result in an assessment more in line with traditional quality standards. Better yet, training a community member to interview their own people would likely achieve the greatest in depth access. So who am I?
But then both those routes create their own biases too. Beneficiaries may not feel comfortable bad mouthing program to the staff, or the staff might not probe as deeply in areas they know are weak. Even recruiting locals brings personal biases, relationships, and power structures into play. I just bring in a different bias. Maybe people are so excited to talk to a foreigner, for the novelty of it, they will actually be more vocal? Maybe they will share thoughts or opinions they wouldn’t have otherwise? Maybe that’s the point, of just bringing in someone from the outside for a fresh set of eyes and ears who will hear and see different things than you can inside your program? I’ll never know exactly what kind of bias I am creating. In the mean time I definitely have a lot of work to do to make sure I carry out this assessment in the most effective way I can. I am just a small piece in the over all monitoring and evaluation program of the project, but hopefully I can provide some fresh perspectives on the activities.