Evaluation of M and E Used

Summary

In this presentation, Edu discusses the monitoring and evaluation (M&E) methods used in Nigeria's Malaria ACSM program, aimed at malaria prevention and control. The framework integrates both process and outcome indicators, including community engagement, media impact, and behavioral shifts related to malaria. Strengths highlighted include the use of a theory of change, mixed-methods research, and alignment with national health systems. Areas for improvement include limited community-led M&E, attribution challenges, slow adaptation, and a lack of focus on marginalized groups. Edu concludes with a call for feedback and further discussion on enhancing the framework.

Takeaways

• 😀 The Nigeria Malaria ACSM framework aims to strengthen advocacy, communication, and social mobilization for malaria prevention and control in Nigeria.
• 😀 The framework has contributed to increased awareness, funding, and collaboration in Nigeria's fight against malaria.
• 😀 Monitoring and evaluation (M&E) is a critical pillar in the framework, ensuring the effectiveness of ACSM activities in supporting malaria prevention and treatment goals.
• 😀 The M&E system used both process indicators (e.g., IPC sessions, media spots, community dialogues) and outcome indicators (e.g., knowledge of malaria transmission, bednet usage, treatment uptake).
• 😀 Data collection methods include quantitative surveys (e.g., malaria indicator surveys), routine data from health facilities, qualitative methods (e.g., FGDUs, in-depth interviews), media monitoring, and advocacy documentation.
• 😀 A theory of change-based evaluation was employed, linking media use, training, and interpersonal communication to intermediate outcomes like improved knowledge and long-term goals like reduced morbidity and mortality.
• 😀 A mixed-methods approach (quantitative and qualitative) provided a more nuanced understanding of the program's impact, aligning with best practices in evaluation.
• 😀 Embedding data into national health systems promoted sustainability and institutional learning, enhancing the broader impact of the framework.
• 😀 Strengths of the M&E system include its focus on process and outcome monitoring, allowing for measurement of activities and behavior changes.
• 😀 Limitations include limited community-led M&E, challenges in attributing behavior changes solely to ACSM efforts, the lack of real-time adaptation, and missing an equity lens to focus on marginalized groups.

Q & A

What is the main focus of the Nigeria Malaria ACSM strategic framework?

The Nigeria Malaria ACSM strategic framework focuses on strengthening advocacy, communication, and social mobilization for malaria prevention and control, with the aim to increase political support, improve public behavior, and enhance coordination among stakeholders.

How does the M&E system in the Nigeria Malaria ACSM framework contribute to malaria control efforts?

The M&E system tracks both process indicators (activities conducted, such as IPC sessions, media spots, and community dialogues) and outcome indicators (shifts in knowledge, attitudes, and practices related to malaria), ensuring effective malaria prevention and treatment goals.

What are process indicators, and can you give examples from the framework?

Process indicators measure the activities conducted during the program. Examples include the number of interpersonal communication sessions, media spots aired, community dialogues facilitated, and advocacy meetings held.

What outcome indicators were used in the M&E system for the Nigeria Malaria ACSM framework?

Outcome indicators include shifts in malaria-related knowledge, attitudes, and practices (e.g., knowledge of malaria transmission, bednet usage among pregnant women and children under five, uptake of ACT, and prompt care-seeking behavior).

What methods were used for data collection in the malaria ACSM framework?

Data collection methods included quantitative surveys (e.g., malaria indicator surveys and demographic health surveys), routine data collection from health facilities, qualitative methods like focus group discussions and in-depth interviews, media monitoring, and advocacy documentation.

How did the mixed methods approach contribute to the evaluation of the ACSM framework?

The mixed methods approach, which combined both quantitative and qualitative data, allowed for a more nuanced understanding of the program's impact, aligning with the triangulation principle in program evaluation.

What is the significance of integrating data into national health systems in the ACSM framework?

Integrating the data into national health management information systems promotes sustainability, facilitates institutional learning, and ensures that malaria prevention efforts are aligned with broader national health goals.

What strengths were identified in the Nigeria Malaria ACSM framework's M&E system?

Strengths include the use of a theory of change-based evaluation, the mixed methods approach, embedding data into national systems, and effective process and outcome monitoring, all of which contributed to the program's overall success.

What are the key limitations and areas for improvement identified in the Nigeria Malaria ACSM framework?

Key limitations include limited community-led M&E, challenges in attributing behavior changes directly to the ACSM efforts, limited real-time adaptation in the feedback loops, and a lack of focus on marginalized or vulnerable groups, such as people with disabilities.

What could have been done to improve the community involvement in the M&E system?

Greater integration of participatory M&E would have empowered communities to identify and track indicators that mattered to them, enhancing local engagement and ownership of the malaria control efforts.