by Isadora Pagy, Monitoring and Evaluation Analyst at IDIS
Theory of Change (ToC) is a highly valuable tool for both strategic planning of an initiative and impact assessment. The process of constructing it can be challenging and raise many questions, so we have compiled a series of tips to assist you.
First and foremost, it is essential to understand what Theory of Change is, its purpose, and how to systematize it.
When we use ToC to evaluate the impact of a particular initiative, it supports a deeper understanding of the causal relationships within the project or program in question. It provides a summary of the target audiences that benefited from the initiative, the activities that were carried out, the expected impacts, and the strategic objective to be achieved.
Understanding who the stakeholders are
Thus, the systematization process begins with mapping out the stakeholders or interested parties involved in the project. We seek to understand who the direct beneficiaries, indirect beneficiaries, implementers, partners, and investors of the initiative were. With this information, we define who will be consulted in constructing the Theory of Change (ToC).
1 – Differentiate between direct beneficiaries and intermediary actors. For example, many education projects conduct activities with schools but aim to impact the children attending classes. It’s possible that educational managers are intermediaries in benefiting students but not direct beneficiaries. Many projects struggle due to a lack of information and data on their direct beneficiary audience, stemming from not clearly defining their primary audience early in the intervention, which in turn hinders effective data collection and monitoring essential for impact evaluation.
The importance of a well-defined scope
Next, we move to defining the activities carried out by the project. It’s crucial to analyze the project’s materials and documents to gain insights into what was initially proposed versus what actually occurred. In alignment with this, interviews with the initiative’s technical team are conducted to understand how the activities were implemented and if there were significant deviations.
2 – Be curious! Understand what happened in practice beyond what is documented. Question the data and seek a deeper understanding of the project.
3 – Have a well-defined scope. If there were significant methodological changes, it may not be feasible to include all of an organization’s initiatives. Not all initiatives may contribute to achieving a single, consistent long-term objective.
Active listening
Once the activities are defined, we investigate what changes occurred in the lives of those who participated in the project or program. At this stage, interviews with beneficiaries are conducted to delve beyond immediate outcomes and explore medium- and long-term impacts generated by the intervention. Pay attention to the following:
4 – Practice active listening. Create a comfortable space for interviewees to openly discuss the impacts they experienced and avoid influencing them by presuming impacts that were not mentioned.
5 – Beware of double counting. Particularly in evaluations involving monetization, avoid counting the same impact twice by structuring a causal chain between mapped results and impacts. For example, in an initiative aimed at generating employment and income through vocational training, participants report increased technical knowledge in computing and higher income from jobs obtained through this new knowledge. Monetizing both changes (knowledge and employment) would double-count the impact of the same causal chain, thus overestimating the project’s impact.
Finally, we understand the strategic objective that the initiative attempted (and possibly achieved) through its activities. This can be achieved by reviewing previously collected data and addressing key questions: What was done? For whom? Why? With what ambition?
The technical team may not always share the same vision regarding the strategic objective, but by consulting key stakeholders and answering these questions, the objective-setting process can be facilitated. This leads to the final tip:
6 – Don’t forget to validate your understanding with the executing team. Presenting the ToC to those who implemented the project on the ground is crucial. It ensures everyone is on the same page and facilitates a well-defined scope, which will ease subsequent steps in your impact evaluation.
The process of constructing a ToC is as important as its outcome. By following these steps, evaluators gain better insights into beneficiary access and project data availability, allowing for the reassessment of scope and evaluation design. This reduces the likelihood of future setbacks that could strain evaluation in terms of time and resources.
Having completed these steps, we can systematize the Theory of Change visually and use it as a guiding tool for the remainder of the evaluation, whether conducting qualitative or quantitative research, for instance.
