One of the risks when developing an evaluation plan, is developing measures without being really clear about what kind of measure they are or how we’re going to use them to understand our program. In this post, I’m going to talk about process measures (AKA outputs) and outcome measures — how they are different from each other and how they are both useful in understanding a program.

When evaluating your programs, you may want to consider two kinds of measures: process measures and outcome measures. Process measures help you assess what you’re doing and how well you’re doing it, and they are evaluated while the program is in progress. Outcome measures evaluate the effect the program has/had on its target population (its impact), and they can be assessed during or after the program is complete. 

Two kinds of process measures: How much and how well

You may know process measures by another name, such as “activity” or “output” measures. These measures are mostly concerned with two questions: “how much?” and “how well?” For example, in an education program, you might track how many students are enrolled (“how much?”) as well as how many students graduated (“how well?”). Each measure offers vital data for measuring the success of your program.

How much?

The “how much?” process measure indicates productivity and penetration; in short, you are asking how many of the target population are getting the service. Though it may seem like a simple measure, the “how much” question is vitally important for maintaining fidelity to scale: to measure whether you are doing what you stated you would do in this program. If you said you would enroll 200 kids, are there 200 kids enrolled? If not, knowing that information early on can help you make adjustments to your recruitment processes, access to the program, or even a fundamental change to how your program is meeting a need in your community.

How well?

While the “how much” process measure indicates whether you are doing what you said you would do, the “how well” process measure indicates the extent to which the program is being implemented successfully. In other words, it helps you evaluate fidelity to plan. Yes, maybe you enrolled 200 kids, but are they the right kids? And are they receiving the services that were planned? “How well” measures are also a good indicator of participant satisfaction (and whether their needs are being met): Are your participants attending regularly? Are they completing the program? The purpose of these measures is the possibility for course correction: if the answer to any of these questions is “no,” you can use this information to improve the program quickly and position the participants for success. 

Process Measures: An Important Partner in Program Evaluation

It might be easy to dismiss process measures as mere “bean counting” – just meaningless numbers. But because they are shorter term, process measures are great indicators of whether you are heading toward your desired outcomes. (And, as mentioned above, they can help you course-correct.) They are partners to outcome measures, not replacements of them. Working in tandem, process measures and outcome measures can provide you with the important feedback you need to determine the effectiveness of your program. But most of us will want to also measure our outcomes. Outcome measures are the 

Outcome measures

Outcome measures evaluate the effect the program has/had on its target population, and can be assessed during or after the program is complete. Outcome measures assess the impact your program had on participants in three primary ways:

1. What they will have: Will they gain knowledge, improve skills, or develop a change in attitude? What will your participants have at the end of the program that they didn’t have at the beginning? 
2. What they will do: How will they use the knowledge, skills, or changes in attitude they gained through the program? 
3. How they will be better off: In the long term, how does this program make their lives better? 

These outcome measures can be short-term or long-term. For example, consider a college-entrance-focused program that exposes students to careers in stem. Through participation in this program, these students will learn about stem careers as a possibility for themselves. They will take science classes. They will be more qualified to attend college and potentially have higher earnings. Some potential outcome measures for this program could include: 

1. Short-term: Students will gain knowledge about careers in STEM fields. (What they will have)
2. Medium-term: Students will take science classes in high school. (What they will do)
3. Long-term: Students will go to college and pursue STEM majors. (How they will be better off)

Of course, you could choose to measure outcomes long after the college stage, including how many students gain employment in STEM fields or how many have higher earnings. However, there are two potential obstacles: (1) longitudinal studies take time and are often not cost effective and (2) it may or may not be appropriate to measure these outcomes, as it could be invasive to the participant to gather the relevant data. Once your program is complete, participants have a right to their privacy and are no longer obligated to engage with you — and in some sensitive cases, it would be a violation of their privacy to follow up with them. However, if you have consent from your participants and they are reporting voluntarily, it may be appropriate to continue gathering data. Just ensure you are prepared for the additional time and expenses for this type of follow-up.

Would you like to learn more about developing outcome measures? Consider reading my post on selecting measures for nonprofit performance management. Or contact me to discuss your desired outcomes and how I might help you develop ways to measure them.