Formal Structured efforts that are part of the education system, (including K-12 schools, colleges and universities), and that usually take place in classrooms or schools. (For example, a biodiversity curriculum used in 8th grade science classes.) Nonformal Nonformal education settings are typically less structured than formal settings and tend to give participants more choice in deciding what to do and how to participate. Workshops, seminars, clubs, service groups, zoos, tours, youth groups, and nature centers are all typical nonformal educational settings. Informal Informal learning settings typically have the least structure. In these settings participants must make decisions about what to learn and how to participate. This is sometimes referred to as incidental learning. Mass media, interpretive activities, signage, kiosks, etc. are settings where informal learning could occur.
Lesson/Unit/Curriculum Material: Informal learning settings typically have the least structure. In these settings participants must make decisions about what to learn and how to participate. This is sometimes referred to as incidental learning. Mass media, interpretive activities, signage, kiosks, etc. are settings where informal learning could occur. Issue Investigation/ Service Learning: Service learning is an approach to teaching that combines academic learning with community or environmental service. Issue investigation refers to any process that involves students in the exploration, analysis, and the development of potential solutions regarding an environmental issue. This category also includes action research, inquiry learning, and similar teaching methods. Field Trip/Guided Tour: An educational experience that takes place outside of the formal school setting Interpretive Exhibits This includes exhibits, as well as signage common in non-formal and informal educational settings such as zoos, museums, nature centers, etc. Club/Interest Group This includes activities/programs sponsored by various clubs/groups. These programs are typically nonformal or informal in nature and take place after/out of school. Camp/Residential Program Includes day camps or overnight camps and similar programs of various lengths. Workshop/Course Refers to instructional classes for the purpose of professional/personal development. This category also includes university courses, as well as other types of more extensive professional development. Outdoor Adventure/Recreation A structured recreational activity or series of activities that take place outdoors (e.g., canoeing, rock climbing, ropes courses, etc.). Out of/After School Any activity/program/event that does not take place during normal school hours, but is associated with Pre-K-12 students and likely to be affiliated with a school. Mass Media Campaigns Campaigns that target segments of the general public and include the use of brochures, flyers, magazines, and/or other media.
Agriculture/Gardening Programs focus on agriculture and/or gardening and other related issues, such as composting. Animals/Plants/Wildlife Programs focus on specific animals/plants/wildlife, as well as the flora/fauna of an ecosystem or region. Biodiversity Programs discuss the diversity of species on Earth, the importance of biodiversity, threats to biodiversity (e.g. habitat loss), endangered species, species extinction, and conservation. Career Choices Programs provide participants with information about careers in environmental/conservation related fields. Climate Change Programs address the science and impacts of, and socio-political responses to global warming. Ecology/Ecosystems Programs focus on the science of ecology and on how various habitats and ecosystems function. Energy Programs focus on energy, energy conservation, alternative energy, or similar topics. Earth Sciences Programs address geology, weather, the water cycle, and other related topics. Human Health Programs address issues such as asthma, lead, obesity/ physical activity, and other hazards to human health. Pollution Programs address environmental hazards affecting land, air, or water (e.g. acid rain, non-point source pollution, etc.). Recycling/Waste Reduction Programs focus on recycling, waste management, waste reduction, etc. Sustainability Programs discuss issues related to sustainability and sustainable development, such as human population, water use, transportation, etc.
Lesson/Unit/Curriculum Material Informal learning settings typically have the least structure. In these settings participants must make decisions about what to learn and how to participate. This is sometimes referred to as incidental learning. Mass media, interpretive activities, signage, kiosks, etc. are settings where informal learning could occur. Preschool (preK) Children typically 5 and under, not yet enrolled in elementary school. Primary (K-5) Children, typically ages 6 to 11. Secondary (6-12) Adolescents, typically ages 12 to 18. College/Continuing Education Adults who are in college or participating in a continuing education or professional development course. Teachers Formal educators at preK-12 institutions. Nonformal Educators Educators working in nonformal settings, such as zoos, museums, nature centers, etc. Other Professionals Adult professionals not included in other audience categories, such as conservation professionals, program managers, business professionals, etc. Adult/Community Groups A group with a specific mission, which may or may not be directly affiliated with an educational institution (e.g., Lion’s Club, Jaycees, etc.). Youth Groups A group composed of young people with a specific mission, which may or may not be directly affiliated an educational institution (e.g., Girl Scouts, Boy Scouts, 4-H, FFA, etc.). Other Groups/Individuals Segments of the general public not included in the other categories, ranging from individual visitors to families to senior citizens. Underserved/Minority Groups Individuals not frequently served by environmental programs and/or members of minority groups.
Evaluation Plan Justifies the purpose of the evaluation and describes the evaluation approach and design. It details what will be done, how it will be done, who will do it, and when it will be completed. Adapted from: http://www.ojp.usdoj.gov/BJA/evaluation/glossary/glossary_e.htm Needs Assessment Determines the need for a project or program by considering aspects such as available resources, extent of the problem and need to address it, audience interest and knowledge, etc. This is also known as a front-end evaluation.. Process/Implementation Evaluation Examines the implementation of a program, focusing on the effort invested in the program and its direct outputs. The information gathered is then used to improve the program. For example, an implementation evaluation might measure how the program is being delivered, who participated, and whether they were satisfied. http://www.gse.harvard.edu/hfrp/pubs/onlinepubs/pcce/ecc.html Outcome Evaluation Outcome evaluation shows the program’s direct effects on specifically defined target outcomes, and provides direction for program improvement. For example, outcome evaluation may show that a program was (or was not) successful in changing participants’ knowledge, attitudes, or behavior. Impact Evaluation Impact evaluation seeks to assess the broader, longer-term changes that occurred as a result of a program. These changes typically occur within the entire community, organization, society, or environment.
Internal Refers to evaluators who are also employees of the organization conducting the program. Adapted from: Worthen, B.R., Sanders, J.R., & Fitzpatrick, J.L. (1997). Program Evaluation: Alternative approaches and practical guidelines. New York: Longman Publishers, USA. External Refers to evaluators who are not employed by the organization conducting the program, other than to implement the evaluation. Adapted from: Worthen, B.R., Sanders, J.R., & Fitzpatrick, J.L. (1997). Program Evaluation: Alternative approaches and practical guidelines. New York: Longman Publishers, USA.
Participants Evaluation findings describing how many individuals are participating, who these individuals are, and how participation changes over time. Activity Implementation Evaluation findings describing how a program is implemented and to what extent the program is being delivered as intended. Satisfaction Evaluation findings describing audience/stakeholder satisfaction with the program or specific components of the program.
Knowledge Refers to participants gaining knowledge as a result of the program. Typically this would require participants to recall something from memory, comprehend its meaning, be able to explain it, and be able to apply it. Adapted from: UNESCO. (1978). Declaration of the first intergovernmental conference on Environmental Education, Tbilisi, 1977. Attitudes/Values Refers to participants altering their attitudes/values as a result of the program. Attitudes/values are typically determined by asking participants to report how they feel and/or what they believe about an issue. Skills Refers to the verbal, mental, or physical abilities participants gain by engaging in the program. In EE, these are typically the thinking and action skills relevant to identifying, preventing, and resolving environmental issues. Adapted from: Bennett, C. (2006). KASA. Retrieved June 1, 2006 from: http://citnews.unl.edu/TOP/english/level3.html UNESCO. (1978). Declaration of the first intergovernmental conference on Environmental Education, Tbilisi, 1977. Behaviors Refers to behavioral intentions or actual behaviors that are formed or occur as a result of participation in the program. These intentions or behaviors are ones assumed to ultimately benefit the environment.
Environmental Positive environmental changes attributable to the program. These impacts are observable and/or measurable, and can be small-scale or large-scale. This includes both improvements in aspects of environmental quality and prevention of environmental degradation. Educational Positive changes in education contexts based on the program. For example, increases in students’ academic achievement, improvements in teachers’ practices, or enhancements to physical learning settings. Health Positive changes in human health and health care resulting from the program. Human health outcomes may include lower rates of disease, disability, injury, and mortality, etc. Health care outcomes may include environmentally healthier facilities for health care, improved rates of correct diagnosis of environmental health problems, and greater communication of environmental health risks between doctors and patients.
Quantitative Information presented and/or summarized through numbers. This includes counts, frequencies, ratings, percentages, and other numeric information. Adapted from: Jacques, D. (1989). Dimensions of Evaluation. Retrieved May 06 from: http://www.brookes.ac.uk/services/ocsd/2_learntch/evaluation.html Qualitative Information described through words, images, or actions. Qualitative data can be obtained through interviews, transcripts, observations, photographs, videotapes, written narratives or similar sources. Jacques, D. (1989). Dimensions of Evaluation. Retrieved May 06 from: http://www.brookes.ac.uk/services/ocsd/2_learntch/evaluation.html
Existing Data/Documents A review and analysis of existing program records and other information collected by the program. These items were not originally intended for the purposes of the evaluation, but may be a valuable source of program information. Adapted from: http://www.gse.harvard.edu/hfrp/content/projects/afterschool/resources/ost_terms.pdf Focus Groups A method of group interviewing designed to yield information through guided interaction among group members. Adapted from: Worthen, B.R., Sanders, J.R., & Fitzpatrick, J.L. (1997). Program Evaluation: Alternative approaches and practical guidelines. New York: Longman Publishers, USA. Interviews A series of orally-delivered questions designed to explore individual participants’ attitudes, knowledge, opinions, etc. Adapted from: Patricia Wheeler, Geneva D. Haertel, and Michael Scriven. (1992). Teacher Evaluation Glossary, Kalamazoo, MI: CREATE Project, The Evaluation Center, Western Michigan University http://ec.wmich.edu/glossary/glossaryList.htm Logs/Journals/Student Work Logs and journals are used to capture participants’ experiences during a program. Typically, participants are asked to record thoughts or actions on a daily or weekly basis. The data is often qualitative in nature, but quantitative data can also be collected. Student work refers to artifacts created by participants during the program, such as art, writing assignments, etc. Observations/Visitor Tracking A data collection strategy in which the behavior of subjects is observed and examined. Standardized Tests Assessment methods designed to be administered, scored, and interpreted in the same way regardless of when and where they are administered. Adapted from: Joint Committee on Standards for Educational Evaluation. (2003). The Student Evaluation Standards, Thousand Oaks, CA: Corwin Press. (used with permission of publisher) http://ec.wmich.edu/glossary/glossaryList.htm Questionnaires/Surveys A form consisting of a series of questions used to collect information about program participants’ knowledge, feelings, intentions, satisfaction, background, etc.
Pretest An instrument administered before the program or activity has begun. It is compared with the results of a posttest to show the effects of the program/activity. Adapted from: www.ojp.usdoj.gov/BJA/evaluation/glossary/glossary_i.htm. Posttest An instrument administered after the program or activity has ended (Center for Program Evaluation, 2006). It is often compared with the results of a pretest or retrospective pretest to show the effects of the program/activity. Adapted from: www.ojp.usdoj.gov/BJA/evaluation/glossary/glossary_i.htm Retrospective Pretest This pretest is given after the program ends, rather than before the program. Typically participants are asked to think back and answer questions about what their knowledge, attitudes, skills, or behaviors were like before the program. Adapted from: http://www.gse.harvard.edu/hfrp/eval/issue30/spotlight.html Retention/Follow-up Test An instrument administered not immediately after the program has ended, but sometime later. Retention/Follow-up tests are used to determine whether the program produced durable changes in participants. Comparison/Control Groups The comparison or control group is a group of individuals who do not take part in a program, but who are ideally are similar to individuals who do (treatment group). Comparison and treatment group results are compared to determine whether the changes in program participants are due to the program or other factors. Multiple Treatment Groups When similar groups of individuals experience different programs to determine which program may be most effective. Or when different groups with different characteristics experience the same program. Random Assignment Participants are assigned to treatment or comparison groups such that each individual has an equal chance of being selected for either group; their assignment to either group is determined by chance. http://psychology.about.com/od/rindex/g/def_randomassig.htm www.mh.state.oh.us/oper/research/pubs.ta.research.glossary.html Interpretive Exhibits This includes exhibits, as well as signage common in nonformal and informal educational settings such as zoos, museums, nature centers, etc.
Descriptive Statistics Descriptive statistics include summary measures such as frequency, mean, mode, median, and others. http://www.socialresearchmethods.net/kb/statdesc.htm Inferential Statistics Inferential statistics are used to make inferences about the whole population, extending beyond the immediate data alone and include correlations, t-tests, Chi-squared tests, ANOVAs, ANCOVAs, regressions, etc. Field Trip/Guided Tour: An educational experience that takes place outside of the formal school setting Interpretive Exhibits This includes exhibits, as well as signage common in nonformal and informal educational settings such as zoos, museums, nature centers, etc.
Key Incident Analysis Key incidents or major events are used to draw conclusions. This may include the use of quotes and examples. Content Analysis w/ Categorization of Responses A method of analysis that summarizes qualitative data by grouping responses into categories. Adapted from: Worthen, B.R., Sanders, J.R., & Fitzpatrick, J.L. (1997). Program Evaluation: Alternative approaches and practical guidelines. New York: Longman Publishers, USA. Content Analysis w/ Descriptive Statistics A method of analysis that summaries qualitative data by grouping responses into categories and then uses descriptive statistics (frequencies, means, etc.) to organize, display, and describe these results. Adapted from: Worthen, B.R., Sanders, J.R., & Fitzpatrick, J.L. (1997). Program Evaluation: Alternative approaches and practical guidelines. New York: Longman Publishers, USA. Content Analysis w/ Inferential Statistics A method of analysis that summaries qualitative data by grouping responses into categories and then uses inferential statistics (correlations, t-tests, etc.) to draw conclusions and inferences from these results. Adapted from: Worthen, B.R., Sanders, J.R., & Fitzpatrick, J.L. (1997). Program Evaluation: Alternative approaches and practical guidelines. New York: Longman Publishers, USA.
