Program evaluation is a subject that often causes angst among nonprofit leaders. Though the leaders in your organization may agree it is important to track the efficacy of your programs, the methods for doing so can lead to confusion or even conflict. Throw in some grant reporting requirements, and the whole thing can feel completely unmanageable.
The solution? A data dictionary.
What is a data dictionary, and why do I need one?
A data dictionary is an up-to-date list of all the measures your organization is currently using to evaluate programs. Creating a data dictionary not only collects all your measures in one place, but it also helps avoid separate programs re-creating measures that are already defined. It is also useful for maintaining communication between development and program managers so program managers are not surprised by grant reporting requirements.
What does a data dictionary look like?
Though it can seem complicated, a data dictionary can easily be created using Excel (or a similar spreadsheet program). Here’s how to create a data dictionary in three easy steps:
1. Create a worksheet for each program.
For the purposes of the data dictionary, a “program” is defined as a set of activities with a target participant and a start and end date. Be sure to have a separate worksheet for each in your Excel workbook. For example, imagine you are running an early education program that involves two distinct populations: parents and babies. Though they are participating in the same program, each population has their own own activities and outcomes; therefore, each one should be tracked on a separate worksheet.
2. Format each worksheet to include the following columns:
Measure name: A clear and brief name that starts with # or %.
Cadence: How often this measure should be calculated and charted. (If there are specific dates, e.g., the end of the fiscal year, record that here.)
Target: What the measure should be or how it should change from period to period.
Data Source: What dataset or survey the item comes from.
Variable: The specific variable or field in the dataset that the data comes from. (See the data dictionary example here.)
Method: The calculation you need to do to maintain validity. (For example, if you calculate the graduation rate by dividing the number of students that completed by the number enrolled and not the number that attended on the first day, be sure to specify that in your calculation.)
Owner: Who “owns” this data? Multiple people might help get this data, but one person is ultimately responsible for calculating and reporting it.
Report: Where this data will be shared. What are you going to do with this data? What reports need to include it?
3. Populate it!
Fill in the data you are already tracking, using the columns as a guide. (Remember: one line per datapoint!) Recording your evaluation measures and having to define them carefully will help you eliminate uncertainties and refine your processes for evaluating your programs. Plus, you’ll have the added benefit of seeing what data you already track.
How do I Use the Data Dictionary?
Once you’ve created your data dictionary, you’ll find it benefits how you communicate about your programs both internally and externally.
Internally, a data dictionary helps standardize definitions across programs. For example, one organization might agree on a single definition of “graduation rate” across all programs so they are easier to compare or aggregate – which will eliminate redundancies, increase buy-in for the measures, and maintain their significance. This will make it easier to identify not only where changes may need to be made, but also which successes can be celebrated!
For grant writing and reporting, your data dictionary is a valuable resource for (1) identifying which measures you’ll be tracking and (2) reporting progress on those measures throughout the life of the grant. When you apply for grants, try to use measures already in your data dictionary; but if you have new reporting requirements with a new grant, be sure to have all the relevant parties meet to add the measures to the dictionary and assign owners so that everyone knows what they’ll need to do and when. With the measures carefully defined, with input from program staff, you’ll know that they are meaningful and precise, making it easier to use them for improvement or to share your successes.
Are you ready to start your data dictionary? Click here to see an example for easy guidance. Even if you don’t have a lot of measures defined yet, getting your data dictionary started is a great first step toward developing a robust evaluation program. And stay tuned – next we’ll talk about using a data agenda – a plan for developing those measures.