Many community-serving organizations struggle to capture the impacts of their work. Reasons include the lack of a formal system or structure by which impacts can be aggregated, dispersed programming that touches upon a wide range of community issues, lack of staff capacity to conduct evaluation, lack of time to dedicate to impact data collection, and limited evaluation resources to draw from. Perhaps equally challenging for those of us who do community work, certain community impacts can feel intangible—in the sense that they can neither be seen nor touched—such as improved leadership skills, increased social capital, and stronger communication. The result is that community-serving organizations often fail to effectively capture the impacts of their efforts on the communities they serve.
To develop an effective system for measuring impacts or outcomes, one first must distinguish between outcomes and outputs. Outcomes refer to measures of change in people’s quality of life or their community, such as decreased poverty levels or greater acceptance of leadership roles. Outputs, on the other hand, refer to short-term activities that are conducted and often quantitatively presented, such as the number of workshops held, and the number of people engaged. To truly measure outcomes, or the results of an organization’s work in a community, one needs to clarify what the intended outcomes or results of work are and what inputs and outputs will lead to the desired outcomes: the so-called theory of change.
In the field of evaluation, the "theory of change" refers to a systematic approach used to map out how and why a desired change is expected to occur in a particular context. It outlines the underlying assumptions, interventions, and pathways through which a program or initiative aims to achieve its intended outcomes. As the adage goes, “How can you know if you have arrived if you don’t know where you are going?”. There are several frameworks that can help organizations map out their theory of change. One commonly used framework is the logic model. A logic model is a structured and visual tool used to illustrate how inputs, activities, outputs, outcomes, and impacts are interconnected within a program, project, or intervention. Essentially, the logic model framework has one work backward, beginning with defining long-term outcomes: the desired changes or improvements sought by a particular effort. Then, it outlines the necessary steps to facilitate these changes and identifies the inputs required to ensure the behaviors and actions that will ultimately lead to the desired outcome.
Once an organization has clarified its short-, medium-, and long-term outcomes, it is critical to establish indicators. An evaluation indicator is a specific, measurable variable used to assess the performance or progress of an initiative toward the desired outcomes. It essentially serves as a benchmark for tracking achievement.
If an organization has mapped out its theory of change—by defining the desired outcomes, what indicators to track, and what measures need to be taken to achieve the outcomes—they are ready to assess what methodologies to use to garner the data that they need to demonstrate progress towards the outcomes.
Below are a few of the more accessible evaluation tools and methodologies that organizations might consider to measure impact, including:
- Surveys to gather feedback from participants or stakeholders to assess changes in knowledge, attitudes, behaviors, or perceptions.
- Key Informant Interviews: Structured or semi-structured interviews with key informants, beneficiaries, or stakeholders to gather qualitative data on outcomes and impacts.
- Focus Groups: Group discussions among stakeholders to explore experiences, perspectives, and impacts collectively.
- Storytelling: The use of narratives, anecdotes, or case studies to communicate the findings, outcomes, and impacts resulting from a program or effort. Storytelling integrates qualitative elements to convey human experiences, successes, challenges, and lessons learned.
- Document Analysis: Review of documents, news articles, reports, or records to track progress, changes, or achievements.
These tools and methods, whether used independently or in conjunction, offer a robust understanding of an organization's impact. Although they are among the more accessible options, their effective implementation demands skill and expertise. Some organizations opt to develop in-house capabilities for evaluation, while others benefit from partnering with experienced third-party entities. This collaboration helps mitigate potential biases, particularly when the organization assesses its own outcomes.
For more information on measuring the outcomes of community-based work, visit the Padlet or contact Charlie French at Charlie.French@unh.edu
