I came across a very interesting and concise article by Fred
Carden from RTI in the December 2013 American Journal of Evaluation, which
makes plain and simple some important points—whether you’re working from the
perspective of sustainability, systems strengthening, country ownership, or
learning and adaptation.
Carden’s introductory quote captures his key point about
evaluation succinctly: “it’s not about your project, it’s about my country.”
While I would prefer critiquing the emphasis for “international development aid evaluation” instead of Carden’s “development evaluation” (which I
think is always relevant—aid or no aid*), Carden makes three simple but
important points:
(1) Evaluation will endure but “[international]
development [aid] evaluation” is not a permanent field of practice.
(2) Evaluation needs to look at
systems, not projects; and
(3) Evaluation requires local
expertise.
I think our work on the Sustainability Framework has
certainly taken this look at systems before projects to heart, even if it has
been a challenge. We have used local expertise, but whether we have given
sufficient leadership to local expertise is more questionable. We’ve recently
had another experience with this type of local system perspective:
Ilona
recently assisted a Gates Grantee in laying the foundation for an evaluation
approach looking at the local and national systems rather than just the
project, in this case by developing a theory of change. A theory of change (TOC)
is a type of logic model that articulates an expected outcomes pathway, the
causal relationships, and the underlying assumptions that relate to the broader
social, cultural, political, economic, or institutional environment behind the
process of change. TOCs are under-utilized tools that can be very helpful in
mapping out, in Carden’s words, the “constellation
of activities [and we would argue also, of actors] that create change and
betterment” in a system. If conceived with that perspective in mind, a TOC
encourages a broader view of change beyond the immediate project that
encompasses (and is grounded in) the realities of the context and therefore
allows key elements outside the project boundaries to be explored and included
in the evaluation of a change process. By clarifying how particular activities
are expected to produce particular outputs and outcomes (and the relevant
assumptions), the TOC helps in framing the evaluation questions to provide
meaningful insights or evidence of program success and identify possible
counterfactual explanations, as relevant.
Such approaches and the logic behind them are going to
continue to grow in relevance. Carden articulates this logic very concisely and
clearly—we recommend you take a look at his short (less than 3 pages!) article. http://intl-aje.sagepub.com/content/early/2013/07/12/1098214013495706.full
Cheers,
Eric
[*] I would suggest
just one language adjustment to the first point and state that “international development evaluation”
is not a permanent field of practice. Development—as in social and human
development-- needs to continue here, there, everywhere and at all times; and
all stakeholders from government to civil society need to rely on good
evaluation to learn and adapt.
