Types of Evaluation Designs

This section describes different types of evaluation designs and outlines advantages and disadvantages of each. Many alternative designs can also be created by adding a comparison group, follow-up test, retrospective pretest, and/or intermediate testing to the designs identified below.

Posttest only

 

evaldesign posttest

 

Data are collected at the end of the program.

Advantages Disadvantages
This type of evaluation design is useful when time is an issue or participants are not available before the program begins Because there is no pretest or comparison group, it is difficult to determine:
  • the magnitude of the outcome (e.g., how much did knowledge increase compared to what participants knew before)
  • whether the outcomes are due to the program or due to some other cause

Retrospective Pre & Posttest

 

evaldesign-retrospective

 

Data are collected at the end of the program. Participants are asked to assess their current level of knowledge/attitudes/skills/intentions AFTER experiencing the program and to reflect on their previous level of knowledge/attitudes/skills/intentions BEFORE experiencing the program.

Advantages Disadvantages
Useful when a true pretest is impossible Some may find it difficult to remember how they thought/behaved prior to the program

Pre & Posttest

 

evaldesign-prepost

 

The same instrument is used to collect data before the program begins and again at the end of the program.

Advantages Disadvantages
  • Relatively simple to implement
  • Controls for participants' prior knowledge/attitudes/skills/intentions
  • Provides better evidence of the effectiveness of the program compared to prior designs
  • Cannot account for non-program influences on outcomes.
  • If self-reporting is used rather than objective measures, posttest scores may be lower than pretest scores. This occurs when participants overestimate their knowledge/attitudes/skills on a pretest but accurately assess their knowledge/attitudes/skills on the posttest.

Pre & Posttest with Comparison Group

 

evaldesign prepostcomparison

 

Data are collected before the program from two groups. One group participates in the program and the other does not. Data are collected from both groups once the program has ended.

Advantages Disadvantages
  • Provides the most assurance that outcomes are actually the result of your program.
  • Allows you to more accurately assess how much of an effect the program has.
  • Can demand more time and resources.
  • Requires access to at least two similar groups.

Pre & Posttest with Follow-Up

 

evaldesign prepostfollowup

 

 

Data are collected before the program begins, at the end of the program, and again at some point in the future.

Advantages Disadvantages
  • Allows you to see if the program has lasting effects
  • Can provide valuable information about long-term impacts
  • Tracking and contacting participants demands time and resources
  • Cannot account for non-program influences unless a comparison group is also tracked

Intermediate Testing & Posttest

 

evaldesign intermediate

 

Data are collected at multiple points during the program (T1 and T2 in diagram) and again at the end of the program.

Advantages Disadvantages
Allows you to track participants’ progress as they move through the program
  • Best suited for longer programs
  • Doesn’t account for non-program influences


Note: Use of these designs is not limited to quantitative methods. Qualitative methods can also be used with these designs.

 

References

 

Adapted from Bennett, D.B. (1984). Evaluating Environmental Education in Schools: A practical guide for teachers. Retrieved July 26, 2006 from http://unesdoc.unesco.org/images/0006/000661/066120eo.pdf