The no-blueprint blueprint to development

Eric Sarriot 

Note: I moved last August to Save the Children in Washington DC, where I now work as Sr Health Systems Strengthening Advisor. I look forward to continued engagement with my friends and former colleagues at ICF/CEDARS.

It’s a little known secret that I own a network of low altitude satellites which monitor every workshop and conference on sustainable development in the world. Like every day.

I took a random representative sample of all 784 workshops and meetings on development that were held last Tuesday and did a textual content analysis. The following exchange was recorded verbatim 1,458 times, which corresponds to 1.86 times per meeting with a 95% C.I. from 1.26 to 2.18:

• “Thank you to the panel for presenting an interesting approach, but I don’t think this can be applied with a cookie cutter in every [country | province | district | commune | village],” said one participant from ‘the field’. To which a first panelist replied:
• “Obviously, we are not proposing this as a blueprint. There is no blueprint to this complex issue.” A second panelist interjected:
• “I totally agree, there is no blueprint. Our approach needs to be adapted to the context.”

STOP!!!!!!!!!

My satellite monitoring is equipped with internal logic contradiction sensors, and these sensors were systematically triggered by this last statement.

Funny story actually about these sensors--when I had first installed the logic analysis program, I struggled a bit. I was getting error messages like “logic routines not applicable to development work,” and “analytics must be supported by either evidence or shallow catchphrases supporting comfortable intellectual habits.” I had to upgrade the software to accept logic again. I’ll spare you the details of programming, but it involved encoding into bits Aristotle, Descartes, and Einstein’s thought experiments. Hard work, but I’ve digressed.

The root of the internal logic contradiction is the simple fact that it’s only blueprints that you need to adapt to context. So, if you’re going to adapt to context, don’t tell me that there’s no blueprint. Say: “we intend to adapt the blueprint for the context.” And that leaves unsolved the question of what to do, when there’s actually no blueprint. But let’s take it a step at a time.

The beauty of blueprints

‘Blueprint’ is actually a metaphor in development—not a real thing. Martin Reynolds of the Open University in the UK regularly points out that we should not “mistake the map for the territory.” So, let’s start with what a real blueprint is actually for.

A blueprint is a document which details the way to build something, and shows how to arrange different sub-systems (drywall, electrical, water pipes, ventilation, etc.) of a structure. It’s great to have a blueprint, because someone has thought through and tested configurations of these sub-systems and made sure that they all work together to provide integrity and functionality to the structure. It spares you from [new metaphor coming] reinventing the wheel each time, and taking advantage of evidence-based best practices. Consider my neighborhood, a lot of houses were built on the same pattern in 1940. Only small variations due to topography existed when the houses were built, but people have been building additions, knocking down walls and modifying them ever since, so every house is now a little bit different from the next one.

I want to finish my basement and need to figure out how to do it. Lucky for me, my neighbor did her basement and let me look at how she did it. I’m happy with what she did and I’m going to use it as a blueprint for my own basement. Since our two houses are not exactly identical (our basement stairs were put in different places for one), I will have to adapt her blueprint to the specific context of my house.

So, blueprint: great. Adapting to context: of course. It’s not either or. It’s the latter because of the former. If there was no blueprint, I would not be adapting my neighbor’s approach, I would have to imagine something different for my basement.

And the same applies to global health. Consider just a couple of examples:

IMCI (the integrated management of childhood illness) was a blueprint. One could argue that because people always ignored the health systems strengthening element of IMCI, basically because the blueprint was not respected, IMCI was considered as a failure. I know this is a long debate. Another example? iCCM (integrated community case management), infection prevention and control, prevention of post-partum hemorrhage in health facilities and with misoprostol in communities, the childhood immunization schedule—all countries adapt those strategies or intervention packages, but there is an unmistakable blueprint. Even some more complex non clinical interventions have blueprints, sometimes tacit or enshrined in legal documents and policies. You want to run an NGO (non-governmental organization) to deliver a public good? Well, there’s a dominant blueprint that you need to have executive leadership, held accountable to some sort of a board, and a financial accountability and oversight structure. After that it gets messier, but those parts seem to be based on a blueprint that is accepted for the robustness and risk mitigation they provide to the organizations. And—again--it always has to be adapted to context.

So, before I tear some of this down, let’s recap the major points so far:

• Blueprints are useful and they can help us be efficient and avoid re-inventing things that have been tested and validated through empiricism and accumulated human experience and wisdom. 
• All blueprints need to be adapted to context. It would be utterly ignorant not to adapt to context, the worst kind of hubris. Even the science of management has long accepted contextual management as a requirement. I don’t think that I need to get into an inventory of the ‘white elephants’ of international development at that point. (Do we love our metaphors or what?) 
• So, please don’t ever brag again about adapting to context. And do me a favor; next workshop you attend, when the panelist says, “there is no blueprint. Our approach will be adapted to the context”, please rough him up a bit and just make him stop. It really messes with my satellite monitoring analytics, and I can’t have any of that.

When there’s actually no blueprint
Let’s go back to our panelist and participant from ‘the field.’ [Spoiler alert: I may caricature the differences in perspectives to stress my point.] The panelist actually has a blueprint, a plan, an idea, an intervention, which he believes is now tested and proven to be able to deliver a public good. Variation in contexts is a challenge--an adaptation and implementation challenge--to be able to deliver what he knows can work and to take it to scale. The statement “there is no blueprint”, we now know, only serves to control the unpleasant complexity and skepticism of the participant, but what the panelist really wants to apply is definitely a blueprint. Adapting to context is a bone he is throwing to these pesky field people who don’t know any better and would have us to boutique projects all the time.

The field participant, on the other end, is immersed in a context. She is richly informed about the geography, history, politics, micro-social and societal reality of that context. The level of complexity increases with the level of attention paid to details, and our participant has seen over and over again when approaches cooked outside of her context have failed on the cliffs of that complexity. (This metaphor at no extra-charge.)  What she really is hinting at is not that a solution needs to be adapted to context, but that a solution needs to be developed, created, imagined, and invented in the specific context where the problem is identified.

Those two views are not solved by the adapting to context platitude; they represent very different approaches to problem-solving. So, who’s right?

Well, no surprise here; the answer is… it depends.

As we have just seen, there’s a beauty and value to blueprints. But there’s also a world where contextual design and innovation dominate. And it is underappreciated in central / global spheres of decision making of global health. As many things are, there is a continuum to navigate, but the dominant model of our work is blueprint thinking. The necessary and productive intellectual discussion about where blueprints fail us and when we need a different type of thinking is too systematically squelched. This could be due to power differentials between the center and the periphery of all our systems, and to blind spots emerging from our different points of observations.[1]

There are a couple of models out there describing where complex takes over complicated in the problem definition and solutioning space. One of the most famous is David Snowden’s Cynefin model, which represents problems from simple, to complicated, to complex, on to chaotic. When problems are complicated, best practices can be identified and promoted through protocols and, yes, adaptable blueprints. But when we enter the space of complexity, emergence takes precedence over best practices. I once tried to map out how complexity increases in the definition of global health problems, based on work by Geyer and Rihani. It turned into this table, which might provide a concrete illustration.

The more your problem is on the right side of the table, the more useful will be a blueprint, if used with smart adaptation to context. But as you move to the left, the value of the blueprint decreases. At some point, adaptation is no longer the solution. Creation, invention, context-based design become the requirements. This means that you start from the perspective of the context actors, as opposed to that of the global experts.

Figure: increasing level of complexity in problem definition from right to left (Source: http://link.springer.com/article/10.1007/s11213-014-9329-9)

A clarification: I am not claiming that this is the way to determine whether a blueprint is appropriate or not. But I suggest that more often than not, problems on the left side of the table will not be amenable to blueprints, even if they may incorporate sub-issues where a valid best practice or blueprint is available.

In conclusion, let’s acknowledge that “there’s no blueprint; we need to adapt to context” is an illogical statement, used sometimes with the best intentions, but also too often as the expression of a central-planner bias preventing an intellectual debate that we badly need. In the absence of a blueprint for figuring out whether a blueprint can be used, maybe we can start by listening to the question of the field participant with a little less condescension and a little more intellectual curiosity.

So, make sure to bring that up at the next workshop. And remember: my satellites are watching!

[1] I’ve probably been led on this trail of thinking following a presentation that I made in June 2016. The topic of it was about blind spots in global health, specifically blind spots to self-organization. Definitely some overlaps. The summary of the presentation is available here: http://www.witpress.com/elibrary/dne-volumes/11/4/1273

Designing programs that work better in complex adaptive systems

Ann Larson

Social Dimensions

ann.larson@socialdimensions.com.au

I became interested in complex adaptive systems (CAS) in 2013. I lead a team identifying lessons from 18 programs scaling up women’s and children’s health innovations. It became clear that a critical success factor was how effective the implementation team was at recognising and responding to challenges. With colleagues, I learned about the properties of CAS, examined if they were present in several case studies of national scale-ups, and uncovered the effective and ineffective responses to the unexpected turn of events (Larson, McPherson, Posner, LaFond, & Ricca, 2015). As a result of this work, I see properties of CAS operating everywhere.

A consensus on how to create change within a CAS is emerging, based on experience and backed by a growing body of research.  This presentation briefly describes some of the most commonly stated principles, and then asks the question, ‘why are these practices not informing the design of programs, especially in international development?’ It appears that this is not due to lack of knowledge or interest. Instead, it arises from the nature of donor organizations and the power relations between those who commission, conduct and assess the designs on the one hand and the government officials, local NGO staff, front line staff and community members on the other hand.

Next, the presentation gives an overview of complexity-sensitive design approaches that could improve projects’ implementation and impact. There are many promising methods being trialled.  However, they should be accompanied with a large yellow sticker: USE WITH CAUTION. Recent reviews suggest that they are difficult for stakeholders to understand and conduct (Carey et al., 2015) and are not congruent with donor requirements (Ramalingam, Laric, & Primrose, 2014). Importantly, pilots of the use of systems thinking to design programs are not validated; we do not know if they actually create accurate descriptions of how systems work or contribute to improved outcomes (Carey et al., 2015).

This presentation, originally given at an evaluation conference, argues that evaluators and the process of evaluation should be central to complexity-sensitive design. First, the information used to inform designs needs to unite rigorous, generalizable evidence and nuanced experience of working within the specific context. Evaluators regularly draw on both sources of knowledge. Second, these design approaches need evaluating. What value do they offer over conventional methods and are they really as appropriate, effective and efficient as their proponents promise?

Presentation is available here. 

References:

Carey, G., Malbon, E., Carey, N., Joyce, A., Crammond, B., & Carey, A. (2015). Systems science and systems thinking for public health: a systemic review of the field. BMJ Open, 5, e009002. doi: doi:10.1136/bmjopen-2015-009002

Larson, A., McPherson, R., Posner, J., LaFond, A., & Ricca, J. (2015). Scaling Up High-Impact Health Interventions in Complex Adaptive Systems: Lessons from MCHIP. from http://www.mcsprogram.org/resources/scaling-high-impact-health-interventions-complex-adaptive-systems-lessons-mchip/

Ramalingam, B., Laric, M., & Primrose, J. (2014). From Best Practice to Best Fit: Understanding and Navigating Wicked Problems in International Development Shaping Policy for Development Working Paper. London: ODI.

Scenario Planning for Development

CEDAR's Sustainability Framework has long talked about the importance of planning for various scenarios by taking the long view in project planning, management, and evaluation activities, reiterated again in a recent paper by Eric Sarriot et al., A causal loop analysis of the sustainability of integrated community case management in Rwanda which also studied scenarios.

Source: http://neomer.us/wp-content/uploads/2012/11/page_scenarioplanning.png

As Wilson Center's New Security Beat writes in its article Scenario Planning for Development: It's About Time, "scenario planning systematically looks at existing and emerging trends and their plausible - though sometimes unlikely - combinations in order to reduce risk. It's an exercise that does not produce single point predictions, but examines a range of possible situations to help prepare for the unexpected."

It's interesting, and heartening, to see USAID changing its approach to development over the last few years by incorporating longer term goals, "adaptive programming" to better respond to external influences such as natural disasters, disease outbreaks, or shifts in governance structures, and increasing focus on "exit pathways". New Security Beat writes of these adaptive approaches that are increasingly embraced, in line with and in response to the rise in uncertainty, preparedness/response shortfalls, and growing complexity. This indeed seems a direction to impact development work for years to come.

Read more about scenario planning and its increasing use by USAID here.

Resources for adaptive management practices and cost-effectiveness in development

We recently added two new documents to our CEDARS Center resource repository to help development professionals think about adaptive management practices and program cost-effectiveness during implementation, planning, or evaluation.

Have a read through below and click through the links. As always, should you have comments or questions (or additional resources we can share with the sustainable health and human development community), do not hesitate to reach out to us.

Navigating Complexity: Adaptive Management at the Northern Karamoja Growth, Health, and Governance Program [document available here]

This paper, by Engineers Without Borders Canada, under contract with Mercy Corps (MC), is a case-study regarding adaptive management practices within Mercy Corps’ (USAID-funded) project, Growth, Health & Governance Program (GHG). The paper covers building the culture necessary for learning and adaptation, discusses some tools and processes that support adaptation, and some implications for funders and practitioners. Throughout the document, culture is emphasized as the most important factor to be successful in adaptive management and provides strategies and attitudes deemed necessary in achieving this culture. The tools and processes are presented with the purpose of reinforcing the described culture. One of the tools, the Results Chain, is an interesting way of conceptualizing the path to reaching the goals of the project and is similar to a results framework.

A blog post, with a summary is available here: http://usaidlearninglab.org/lab-notes/navigating-complexity-adaptive-management-northern-karamoja-growth-health-governance

Cost-Effectiveness Measurement in Development: Accounting for Local Costs and Noisy Impacts [document available here]

This policy research working paper from the World Bank Group, Africa Region, can help individuals think about cost-effectiveness within their programs from implementation, planning, or evaluation perspectives. As evidence from rigorous impact evaluations grows in development, there have been more calls to complement impact evaluation analysis with cost analysis, so that policy makers can make investment decisions based on costs as well as impacts. This paper discusses important considerations for implementing cost-effectiveness analysis in the policy making process. The analysis is applied in the context of education interventions, although the findings generalize to other areas. First, the paper demonstrates a systematic method for characterizing the sensitivity of impact estimates. Second, the concept of context-specificity is applied to cost measurement: program costs vary greatly across contexts -- both within and across countries -- and with program complexity. The paper shows how adapting a single cost ingredient across settings dramatically shifts cost-effectiveness measures. Third, the paper provides evidence that interventions with fewer beneficiaries tend to have higher per-beneficiary costs, resulting in potential cost overestimates when extrapolating to large-scale applications. At the same time, recall bias may result in cost underestimates. The paper also discusses other challenges in measuring and extrapolating cost-effectiveness measures. For cost-effectiveness analysis to be useful, policy makers will require detailed, comparable, and timely cost reporting, as well as significant effort to ensure costs are relevant to the local environment.

You can find additional information on this paper here: http://documents.worldbank.org/curated/en/2014/09/20196499/cost-effectiveness-measurement-development-accounting-local-costs-noisy-impacts-cost-effectiveness-measurement-development-accounting-local-costs-noisy-impacts

Learning from a post-project evaluation study, applying systems thinking and addressing complexity in community health

By Michelle Kouletio

It doesn't happen enough, but every once in a while a seed of a sustainable health intervention is planted in the ground.  In this case, the seed was planted at the doorstep of a mayor's office in northwestern Bangladesh, amid narrow, busy roads with open sewers of common of bustling secondary cities. Among the middle class families live the extreme poor.  Along with several other challenges, the poor are failed by the government health system whose facilities are overwhelmed by patient volume and whose outreach workers do not serve.   While national policy assigned elected municipal leaders with responsibility for ensuring coverage of equitable health services, these leaders were provided little guidance nor resources.

With support from the USAID Child Survival and Health Grants Program, Concern Worldwide worked in this and other municipalities in Bangladesh to empower municipal leaders to develop a replicable model for social mobilization in complex urban environments. What made this project unique from so many well intended and ambitious community health projects was the embedment of a systematic sustainability planning and monitoring system that established a shared vision and measurement framework across the Mayor’s office, elected representatives, service providers, social leaders and health volunteers along with the project team.

As the technical advisor for Concern Worldwide, I had the privilege of backstopping this initiative.  It took quite of bit of extra effort, particularly in developing practical capacity measuring tools from scratch and maintaining regular reviews at the neighborhood and municipality level. Sustainability planning also required tackling structural barriers and inter-ministerial relations which could have otherwise been ignored in a conventional project. However, the benefits of this deeper analysis and shared accountability approach resulted in real improvements in equitable health outcomes and an enduring political mobilization approach that allowed the population to continue reaping benefits years after the project closed.

Two recently published articles on this work further validate the importance of the hard work of the Concern Worldwide staff in Bangladesh and their contribution to the developing body of evidence on adaptive health systems and sustainability planning:

Advancing the application of systems thinking in health: sustainability evaluation as learning and sense-making in a complex urban health system in Northern Bangladesh

Community Health Systems as Complex Adaptive Systems: Ontology and Praxis Lessons from an Urban Health Experience with Demonstrated Sustainability

Post-script from Eric Sarriot:

These two papers coincide in their release to form a useful series on the Concern CSHGP Bangladesh experience. The first one is part of a larger and important Supplement of Health Research Policy and Systems on Systems Thinking in Health, coordinated by Taghreed Adam of WHO’s Alliance for Health Policy and Systems Research.

Link to the Supplement.

Link to Dr Taghreed Adam’s editorial.

Link to Dr David Peters’ introduction, The application of systems thinking in health: why use systems thinking?

Putting vulnerability first – the need to revise targeting and delivering aid to promote sustainability and resilience

By Debra Prosnitz, MPH

The Humanitarian Policy Research Group (HPG) recently released a policy brief on resilience: Political flag or conceptual umbrella? Why progress on resilience must be freed from the constraints of technical arguments (1). Reading this brief, I began to reflect on the relationship between resilience and sustainability. In the context of development, resilience should address ability to cope and recover from crisis, and the sustainability should address the process of strengthening social capital to sustain progress in health, social progress, etc. (2). If sustainability is a process, perhaps resilience is one of the outcomes. I decided that the importance of this topic deserved a CEDARS blog update.

In this short (4-page) brief, Simon Levine succinctly captures two approaches to addressing resilience, brings to light the stalemate and consequent inaction in which they are stuck, and suggests a new way forward in theory through action (3).

Levine discusses two “broad arguments” for addressing resilience, both of which, he argues, distract us from the underlying need to identify and understand vulnerabilities of individuals and communities, and find ways to address these vulnerabilities: The political argument convinces us that something needs to be done - “since the shocks and stresses that cause crises cannot be prevented, the task is to ensure that people are better able to cope when things do go wrong,” and the technical argument calls for a refined definition of resilience and new approaches for addressing it because what has previously been implemented has not adequately addressed the complexity and challenges of resilience, such as climate change. While we may not yet know the best way to define resilience or the best strategies and approaches to address it, we must not let this stop or delay efforts to do so.  We know that vulnerabilities exist and should be addressed, and should move forward by identifying and understanding vulnerabilities of individuals and communities, and findings ways to address these vulnerabilities.

Resilience then—even if imperfectly conceptualized--can already be enhanced by ensuring that “vulnerabilities are the center of development policy and investment,” and that marginalization be addressed and minimized through development; resilience should not become another sector of development. Thus, Levine suggests that the definition of and theoretical framework for resilience can be developed and refined through action (rather than delaying action pending conceptual clarity).

While Levine addresses the complexity of development and the “structural, institutional and bureaucratic obstacles” to making change to how development aid is targeted and delivered, I wish he had made a clearer link between resilience and sustainability. I’m envisioning resilience as a pillar of sustainability, because while we can’t predict which shock will occur, the occurrence of shocks and changes following even the best of our interventions is almost a certainty. Further, neither resilience nor sustainability can be achieved by external intervention and leadership alone. Community involvement should be a prerequisite for approaching both, with communities and individuals leading efforts to identify vulnerabilities and define resilience.

Levine’s brief is a glimpse into larger bodies of work he has published on this subject (4), which can also be found on the HPG website. Whatever your stance on resilience as a concept, this paper is an important reminder of the old adage not to let the perfect become the enemy of the good; we should not be complacent in our thinking about models of aid delivery, and should be actively thinking about and advocating for changes in the way in which aid is both targeted and delivered. Vulnerability should be the primary criteria, with aid targeted toward the most vulnerable first and delivered in a way that bolsters resilience.  

----------

(1) S. Levine, Political flag or conceptual umbrella? Why progress on resilience must be freed from the constraints of technical arguments. Policy Brief 60. Humanitarian Policy Research Group (London: ODI, 2014).
(2) Sarriot, et al. Taking the Long View: A Practical Guide to Sustainable Planning and Measurement in Community Oriented Health Programming. Macro International, Inc. (Calverton, MD, 2008) defined sustainability as “a process that advances conditions that enable individuals, communities, and local organizations to improve their functionality, develop mutual relationship of support and accountability, and decrease dependent on insecure resources…(and) enables local stakeholders to play their respective roles effectively, thus maintaining gains in health and development…”
(3) As our fearless CEDARS leader Eric Sarriot summed it up “praxis can improve without a perfect epistemology.” 
(4) S. Levine, Assessing resilience: why quantification misses the point. Humanitarian Policy Group Working Paper. (London: ODI, July 2014); and 

S. Levine and I. Mosel, Supporting Resilience in Difficult Places: A Critical Look at Applying the ‘Resilience’ Concept in Countries Where Crises Are the Norm, HPG Commissioned Paper for BMZ (London: ODI, 2014).

Emergence of Sustainability in a Complex System

On January 26, CEDARS was featured at a Microlinks Breakfast Seminar, which turned out to be a great success, as well as a unique learning experience for everyone.

Eric Sarriot, joined by Sudhir Wanmali, Sharon Arscott-Mills, and Owen Calvert, presented a simple definition of "Complex Adaptive Systems", described how the concept of Sustainability fits into this definition, and provided real-world examples of its use.

Following the presentation, a thought-provoking Q&A session took place amongst both the panel and the audience. One question, in particular, asked how complex systems in food security could be incorporated into Pastoral Systems, and the bulk of the discussion focused on data collection challenges, the need for accurate indicators, and stakeholder organization in these types of systems.

Watch Eric's follow-up interview:



For more information, view the full presentation with audio or visit the Microlinks website.

Read a detailed summary of the event on ICT 4 Agriculture's Blog: Sustainability Lessons from HSS: Implications for FS and ICTs Projects.


Cheers,

The CEDARS Team

Some critical thinkers about complexity in development assistance

Thanks to Matt Dettman for pointing me to a new link,  now referenced in our resource page.

The idea of development happening in complex adaptive systems is so new, that the bloggers of 'Aid on the Edge of Chaos' are highlighting a speech on global resilience from 1970!

I'm going to keep an eye on this blog and see what other nuggets can be gleaned.

Cheers

Eric