Evaluation can be a mysterious process for the uninitiated and often even for those with extensive experience. How do you get started and what should you know before you get started? This section prompts you to think about the types of things you will need: money, time, evaluation expertise, support from program managers and staff, and in some cases, official approval for your evaluation.
If you would first like to learn more about evaluation (what it is, why it is important, and what makes for a good evaluation), please see "Evaluation: What is it and why do it?"
What types of resources will I need to invest in the evaluation?
Do you have sufficient experience to carry out the evaluation?
Environmental educators can successfully conduct many evaluation activities themselves. There are times, however, when an external evaluator can be of great assistance, such as in the design of instruments or data analysis. If you have never conducted an evaluation before, but expect to do more in the future, it may be worthwhile to invest the time and money to increase your evaluation expertise. Alternatively, if the requirements of your evaluation exceed your evaluation capacity or if you need an evaluation to be perceived as objective, hiring an external evaluator may be the way to go.
How much time are you willing and able to dedicate to the evaluation?
This question is particularly relevant if you are planning to do the evaluation in-house. Even a small evaluation can take more time than you think. Remember that the internal evaluator is likely to have other competing organizational responsibilities, as are the program managers, staff, and volunteers who may be called upon to help carry out the evaluation.
How much are you willing to spend on the evaluation?
The cost of hiring an external expert to evaluate your program can be prohibitive for some environmental education (EE) programs depending on the expert and the scope of the evaluation. While internal evaluations can save money, remember to factor in the in-house costs associated with staff time, copying, mailing, collecting and analyzing data, etc.
To learn more about the resources you will need for your evaluation, see:
Best Practices Guide to Program Evaluation for Aquatic Educators (.pdf)
Recreational Boating and Fishing Foundation, 2006
Beginner Intermediate
In Chapter 2, the section entitled "Capacity and Resources" provides advice on how to build organizational and staff capacity for evaluation and how to estimate the budget, time, and other resources you will need. Several pages also explain how to create an evaluation budget as well as a detailed timeframe for completing your evaluation.
Budget enough for your evaluation!
How do I find and work with an internal or external evaluator?
Evaluators - whether internal or external to your organization - play an important role in shaping and managing the evaluation as well as in interpreting its results. Identifying, choosing, and working with one of these individuals is an essential part of the evaluation process. For more information on this topic, please visit our page on Finding and Working with an Evaluator.
How do I involve program managers, staff and others?
An effective way to build stakeholder support for your evaluation is to actively involve them in the design and implementation of the evaluation. Consider inviting board-members, program staff, managers, community members, and other key stakeholders to participate on an evaluation advisory panel, or on the evaluation team.
Once you identify the individuals whose support you want, you may want to create a forum to generate discussion and promote interaction among these stakeholders (UNDP, 2003).
Ask them what aspects of the program they would like to see evaluated, and discuss the potential benefits of evaluation for different groups. Consider offering stakeholders the opportunity to attend a training seminar, course, or other event on evaluation, so they will be prepared to participate meaningfully.
Participatory evaluation is an approach that encourages active engagement in, and support of your evaluation. By fostering participation, you will build ownership of the evaluation, and ownership of the program itself.
Anyone initially uninterested in participating could be reminded about how the evaluation process or results might help them, or the program. You are unlikely to have difficulty gaining the support of those who are accountable for the program, or have a stake in its success (such as board members, donors, managers, and staff), especially if they are given the opportunity to help shape the evaluation. Even if you hire an external evaluator to undertake your program evaluation, in-house participation in the process can still take place and is highly recommended.
How do I obtain approval for the evaluation and consent from participants?
Most evaluations collect data from individuals. Protecting the rights and privacy of these individuals and treating them with respect are the responsibility of the evaluation team. Institutional Review Boards (IRBs), help evaluators to ensure that these things happen. Typically, you need to prepare an application that includes a description of how you plan to conduct your evaluation as well as the consent forms and data collection instruments you will use. To find out more, please visit our page on Evaluation Consent and the IRB Process.
References
United Nations Development Programme. (2011). Results-Based Management Handbook: Harmonizing RBM concepts and approaches for improved development results at country level. Retrieved April 9, 2015 from: http://www.ilo.org/public/english/bureau/program/dwcp/download/undg_rbm1011.pdf