Better survey results: Sampling
Whether its for a research project or a program evaluation, one of the hardest things about implementing a survey is understand who you should have respond to it. A survey is only a useful method of gathering data if you have planned carefully about your sample. The goal is to make sure that the way you have sampled is consistent with you you want to make statements about or generalize about. To accomplish that, you need to make sure that the people who have responded to your survey represent your target population well.
There are two ways to do this. One is ask every person who is in the group. This is called a census and it’s probably appropriate if you want to learn about a small group, for example to understand how all the participants in a course felt about the course. But if the population is very large, like a whole neighborhood, you’ll need to ask some of the people and generalize to all the people. To do this, you’ll make a plan (called a sampling frame) to ensure that you get some of all the types of people. If you do decide to sample, you’ll want to make sure that participating in the survey is as random as possible. Randomness helps ensure that you’ll get a more representative sample. For example, you might randomly select addresses and interview the residents.
Your sampling frame might mean that you’ll survey at different times of day or in different places to ensure representation. It’s ok to adjust your sampling frame to ensure representation. For example, you might decide that it’s important to have all four corners of your neighborhood equally represented even though they’re not equally populated, so you’ll divide the neighborhood into four sections and then select the same number of households randomly from those sections. You might also decide that having a particular group’s voice included is very important, so you’ll ask more of those people. For example, if your neighborhood is 10% African-American and you survey 100 people randomly, about 10 African-Americans would be included. You could decide that you’re going to make sure you interview 20 African-American residents to make sure their opinions are included.
It is important not to let the way that you implement the survey skew who is able to respond. For example, if your survey is only available in English, then only English-speakers will respond. If you only administer your survey at a coffee shop, then only coffee-drinkers will respond. This doesn’t mean it’s not a good survey, just that you are limiting who you can generalize to: English-speaking residents of the neighborhood instead of all residents of the neighborhood, for example.
{{cta(‘9181152a-00a9-45de-9dfd-3e3552d26951’)}}
It’s easy to get carried away with implementing surveys at times or places where there will be a lot of people present. Be cautious about this, because while it might give you a lot of answers to your survey, it might not help with making your survey representative. For example, if you want to generalize about everyone in the neighborhood, and you distribute the survey at a children’s festival, you’re probably only going to get responses from families with children, people who don’t work on Saturdays, and people who like festivals — and not other types of people who live in the community.
How much energy you put into designing your sampling frame will depend on how much time and how many resources you have as well as the importance of the survey. For many small projects, convenience samples are fine. A convenience sample means that you have the survey completed by people who are convenient to you — say people who walk by your office. Convenience sampling is used frequently because it’s so easy and it yields quick results. However, if the results of the survey are very important, and you want to ensure that you are able to generalize to a broader population, you may want to use a more complicated sampling method.
