Designing programs that work better in complex adaptive systems

Ann Larson

Social Dimensions

ann.larson@socialdimensions.com.au

I became interested in complex adaptive systems (CAS) in 2013. I lead a team identifying lessons from 18 programs scaling up women’s and children’s health innovations. It became clear that a critical success factor was how effective the implementation team was at recognising and responding to challenges. With colleagues, I learned about the properties of CAS, examined if they were present in several case studies of national scale-ups, and uncovered the effective and ineffective responses to the unexpected turn of events (Larson, McPherson, Posner, LaFond, & Ricca, 2015). As a result of this work, I see properties of CAS operating everywhere.

A consensus on how to create change within a CAS is emerging, based on experience and backed by a growing body of research.  This presentation briefly describes some of the most commonly stated principles, and then asks the question, ‘why are these practices not informing the design of programs, especially in international development?’ It appears that this is not due to lack of knowledge or interest. Instead, it arises from the nature of donor organizations and the power relations between those who commission, conduct and assess the designs on the one hand and the government officials, local NGO staff, front line staff and community members on the other hand.

Next, the presentation gives an overview of complexity-sensitive design approaches that could improve projects’ implementation and impact. There are many promising methods being trialled.  However, they should be accompanied with a large yellow sticker: USE WITH CAUTION. Recent reviews suggest that they are difficult for stakeholders to understand and conduct (Carey et al., 2015) and are not congruent with donor requirements (Ramalingam, Laric, & Primrose, 2014). Importantly, pilots of the use of systems thinking to design programs are not validated; we do not know if they actually create accurate descriptions of how systems work or contribute to improved outcomes (Carey et al., 2015).

This presentation, originally given at an evaluation conference, argues that evaluators and the process of evaluation should be central to complexity-sensitive design. First, the information used to inform designs needs to unite rigorous, generalizable evidence and nuanced experience of working within the specific context. Evaluators regularly draw on both sources of knowledge. Second, these design approaches need evaluating. What value do they offer over conventional methods and are they really as appropriate, effective and efficient as their proponents promise?

Presentation is available here. 

References:

Carey, G., Malbon, E., Carey, N., Joyce, A., Crammond, B., & Carey, A. (2015). Systems science and systems thinking for public health: a systemic review of the field. BMJ Open, 5, e009002. doi: doi:10.1136/bmjopen-2015-009002

Larson, A., McPherson, R., Posner, J., LaFond, A., & Ricca, J. (2015). Scaling Up High-Impact Health Interventions in Complex Adaptive Systems: Lessons from MCHIP. from http://www.mcsprogram.org/resources/scaling-high-impact-health-interventions-complex-adaptive-systems-lessons-mchip/

Ramalingam, B., Laric, M., & Primrose, J. (2014). From Best Practice to Best Fit: Understanding and Navigating Wicked Problems in International Development Shaping Policy for Development Working Paper. London: ODI.

A review of "Systems science and systems thinking for public health: a systematic review of the field"

By Eric Sarriot

A recent publication in BMJ titled Systems science and systems thinking for public health: a systematic review of the field by Gemma Carey et al. describes findings from a systematic review of current literature on systems science research in public health, with a focus on specific “hard” and “soft” systems thinking tools currently in use. A review of the literature sub-selected for analysis in this paper revealed the absence of some pertinent articles that may have enriched the discussion, but as the authors acknowledge, quoting Williams and Hummelbrunner, “holism is ‘somewhat of an ideal. In reality all situations, all inquiries are bounded in some way.”

An interesting application of systems thinking can be found in David Peters and Ligia Paina’s paper on the Develop-Distort model. The Develop-Distort model paper does not reference the great thinkers of Soft Systems or Systems Dynamics, which could be why it did not qualify to be part of this systematic review, yet it is also of great interest. With this model, and other emerging ones, the question then becomes whether new tools and methods, that abide by key principles, should and could fit into the constantly evolving field of systems thinking. Of course, this question in and of itself, does pose some bias.

The review by Carey et al. continues by ascribing sub-selected literature with four types of systems thinking categories:

• Position pieces: the literature in this category mostly advocates for greater uses for systems thinking in public health;
• Papers with an analytic lens: most articles here maintain the caveat that once analysis using a systems thinking approach is complete, many researchers revert back to previously used analytic tools, likely due to a lack of practice and training in systems methodologies;
• Bench-marking of best practices: where systems thinking is used to evaluate public health practice – with some articles evaluating the best practice based on whether it abides by systems thinking principles, rather than whether the application of systems thinking advanced thinking and performance; and
• Systems modelling: modelling of real-life or dynamic processes using systems thinking.

While the discussion is fairly long, it makes several good points, including that systems thinking is not a panacea and should not be approached as such, that there is a need for greater verifiability of models, and last but not least, that there is a need to improve skills of public health researchers in systems methods and thinking. The authors then move to discussion on the value of soft system methodologies emphasizing how metaphors can be used as a useful heuristic. The authors describe this evolution in thinking as a challenge to how health policy makers define “evidence,” and conclude with a note that systems thinking in health will improve if and as we learn to ask the right questions of systems science, and play down some of the accompanying rhetoric.