I’ve been gathering data in the field. How do I know when I’m done?

Professional program evaluators always start a study with a sampling plan. But program managers and other professionals often find themselves gathering data in a less structured way. We often need to gather feedback or input from the community without developing a research plan in advance. For example, you might want to gauge community interest in a new park by asking members of the community at every opportunity. Many times, the ending point will be clear – you’ll just run out of time. Other times, you may wonder when it’s ok to stop gathering new data and start drawing conclusions.

Here are some general guidelines that would let you know that you have collected enough data to start drawing conclusions.

 

The first thing to ask yourself is whether you’ve asked the right kind of people.  If your project is about people in a neighborhood, have you asked people who live in that neighborhood? Do you also want to ask people who work in the neighborhood but live elsewhere? If your project is about youth, have you asked youth? Think carefully about who you want to represent and how you’ll represent them when you’re talking about your findings. If you’re confident that you’ve asked the right people, and you know how you’ll represent them when you present your findings, you are on the way to being done.

The second question to ask yourself is have you asked all the right types of people? Who do you want to be able to comment about? All the youth in your community? Or just African-American youth in your community? If there is a demographic group that has not had a chance to respond to your survey – women, youth who are parenting, youth who are not in school – you can’t generalize your findings to speak about them.

 

Here, you might want to think about whether there is anything in the way that you are gathering information that keeps some voices out. Are you only asking people in English? Then you can only generalize about people who speak or read English. Are you asking in public places during the school day? You might be missing the voices of youth who are in school. If your work is specifically about underrepresented or marginalized groups, I highly recommend Studying Ethnic Minority and Economically Disadvantaged Populations by George Knight, Mark Roosa, and Adriana Umaña-Taylor. If you are confident that you have captured the diversity of the community you want to comment on, you might be done.

The third way to tell you might have enough information is that you’re not getting any new or unexpected answers. Qualitative researchers call this saturation. When your new respondents are consistently repeating answers you’ve heard before, you might be done. If your instrument is a survey and the new answers tend to be grouped the same way as the previous answers, you might be done.

What I have not told you, here, is a specific number. I’ve talked about sampling and sample size here, but the answer is a little more complicated than a single number and depends on the size of the community that you want to comment on and how diverse it is. If you are not planning to do any statistical tests, you don’t need to worry about sample size as much as representation.  Following the above guidelines should help you feel confident that you are representing your community fairly.

 

 {{cta(‘43979b85-ab72-49bc-a0b0-44a90d372251’)}}