Chapter 9: Fundamentals of Program Evaluation

Summary

This video, produced by the Children's Hospital of Philadelphia, introduces the fundamentals of program evaluation. It explains the importance of evaluating programs, highlights key terms like performance goals and smart objectives, and addresses common reasons why organizations neglect evaluations. The video emphasizes how evaluation helps determine program effectiveness, optimize resource use, and support data-driven decision-making. By asking critical questions and utilizing tools like logic models, viewers can improve their programs and contribute to future success. The video also prepares viewers for deeper learning in subsequent chapters on different types of evaluations.

Takeaways

• 😀 Program evaluation helps determine whether your program is effective and can guide future decisions and improvements.
• 😀 It’s important to identify who cares about the outcomes of your program, including stakeholders and funders, to ensure its success.
• 😀 Funders often require program evaluations to understand if a program works, how it works, and whether it can be replicated in different settings.
• 😀 One of the biggest challenges in program evaluation is gathering and analyzing information, but proper tools and resources can overcome this.
• 😀 Evaluation is a systematic process that assesses the merit, worth, and value of a program’s activities, characteristics, and outcomes.
• 😀 SMART objectives are essential in program evaluation as they make goals specific, measurable, achievable, realistic, and time-bound.
• 😀 Program evaluation is often neglected, but it provides essential data to improve program delivery and target audience needs.
• 😀 Top reasons people avoid program evaluation include lack of understanding, fear of negative results, and the perception of it being time-consuming.
• 😀 Effective program evaluation can lead to data-driven decision-making, resource optimization, and potential policy changes.
• 😀 Program evaluation can help clarify the 'who, what, where, when, why, and how' of your program and provide actionable insights for improvement.

Q & A

What is program evaluation, and why is it important?

Program evaluation is a systematic process of collecting information about a program’s activities, characteristics, and outcomes to assess its overall effectiveness and inform future decisions. It is important because it helps determine whether a program is achieving its goals, makes the best use of resources, and provides data-driven insights for improvements or policy changes.

Who cares about the outcomes of a program?

At the very least, the program staff should care about the outcomes. Additionally, stakeholders, funders, and any organizations or individuals who are invested in the success of the program, such as those providing financial support or community partners, also care about the outcomes.

What are the key questions to ask when starting a program evaluation?

The key questions to ask include: Who cares about the outcomes of the program? Who is requiring the evaluation? What challenges might arise in gathering and analyzing information? These questions help clarify the evaluation's purpose and guide the process.

What is a SMART objective in program evaluation?

A SMART objective is a specific, measurable, achievable, realistic, and time-bound goal that outlines what a program intends to accomplish. For example, a SMART objective could be, 'By the end of 2017, there will be a 20% increase in teens who report they will not text and drive after receiving the educational program.'

What are some of the common barriers to conducting program evaluations?

Some common barriers include misunderstanding the evaluation's purpose, fear of negative findings, lack of evaluation skills, and the perception that evaluation is an add-on activity rather than part of the program planning process. Other barriers include time constraints, perceived high costs, and prior negative experiences with evaluations.

Why do funders require program evaluations?

Funders require program evaluations to understand whether a program is effective, how it works, why it works, and whether it could be replicated with other populations or in different settings. This helps ensure that their financial support is being used effectively and contributes to positive outcomes.

What is a performance indicator in program evaluation?

A performance indicator is a measurable benchmark that shows whether the program is achieving its performance goals. For instance, the number of teens who report they will not text and drive after participating in an educational program serves as a performance indicator.

What is the role of program evaluation in making decisions about future programming?

Program evaluation provides critical data that helps decision-makers assess the effectiveness of a program, understand its impact, and identify areas for improvement. This data-driven approach supports informed decisions about continuing, modifying, or expanding the program.

What are the top three ways program evaluation can help immediately?

Program evaluation can help by: 1) Determining whether the program can make a difference, 2) Making the best use of scarce resources, and 3) Leading to policy changes and support through data-driven information.

What are the different types of program evaluations, and how do they differ?

There are three main types of program evaluations: formative, process, and outcome/impact evaluations. Formative evaluation focuses on the design and development phase, process evaluation assesses the implementation, and outcome/impact evaluation looks at the effects and long-term results of the program.