Monday, January 4, 2021

2021 moving forward but remembering what we have learned: There is a humanitarian-to-development nexus. Can we navigate it better?

There is a humanitarian-to-development nexus. Can we navigate it better?

by: Elijah Olivas (with Eric Sarriot and Will Story)

As everyone, I am looking for better things in 2021 -- among them, to remember some of what we learned last year. And no, this is not about Covid-19, although I am sure some of it will apply.

Just over a year ago, we released two case studies of systems effects of successive emergency health and nutrition (EHN) projects of Save the Children a little while back. (Sudan, and Pakistan) And we are hoping to have a cross-cutting analysis paper out soon in Health Policy and Planning. This work took over 3 years to complete, but it provides the opportunity for some reflection, which is worth bearing in mind as we start a new year.

The immense value of asking questions
At the beginning of this project, Save the Children was looking to explore unknowns such as, how have past Save the Children humanitarian projects in two countries (Pakistan and Sudan) evolved and affected changes over the years? And what does it mean to truly strengthen a health system? To understand more, we put a team together, with researchers and practitioners from the University of Iowa and from Save the Children, partnering with CORE Group with OFDA funds in the completion of the studies (read all papers and presentations on the HDTF page).

Together, given the many uncertainties about the impact of past projects and the meaning of health systems strengthening (HSS), we set out simply to learn. We analyzed the series of recovery projects following two emergency health and nutrition (EHN) crises: natural disasters and conflict in Pakistan, and the aftermaths of civil war in Sudan. We looked for evidence that any projects strengthened, or weakened, health systems or the ability for an affected community to transition from recovery to long-term development.

The wisdom of the crowd – building on prior research and concepts
We had very few assumptions before beginning our research, but those we had, we held firmly, by looking around us and making sense of what we found. We held, from our own work and others’ research, that the concept of “community-inclusive systems for health” meshed well with both Save the Children’s projects and the ideas that we were articulating with other authors. Those ideas were later published in another paper: “Beyond the Building Blocks: Integrating Community Roles into Health Systems Frameworks to Achieve Health for All”. Other important sources of information to build on included the evolving uncertainties surrounding HSS, as mentioned by authors like Grace Chee, Sara Bennett, and Josefien van Olmen, and ultimately a systematic review of systems strengthening in humanitarian projects published by the Institute of Tropical Medicine in Antwerp under UHC2030.

With these pieces of information, we moved forward, scanning the EHN and HSS literature with the sole goal of learning. 

What we found out
The recovery efforts following a humanitarian crisis are often divided into two types, based on the way financing for such programs is typically available. On one hand, there are short-term recovery efforts, and on the other, long-term development programs. Despite each having different goals, priorities, and funding requirements, both are crucial for repairing and strengthening systems against future crises. In fact, each represents two ends of what we call a ‘humanitarian-development nexus’ – a progression from one stage of recovery to another. Though practitioners and researchers on each end recognize this nexus, financing differences keep the two ends divided.

HSS is a discipline that can help us bridge this division. Its focus is the interrelation of the largest contributors to a population’s health, such as health care delivery, health financing, and policymaking. For HSS to effectively bridge the two ends of the nexus, shorter-term recovery must be designed to pave the way for longer-term development. As part of our analysis, we examined how past Save the Children recovery projects – intentionally or unintentionally – helped or hindered such longer-term development down the road.

Cautious recommendations
Although our analysis was limited in scope, with attention to crises in only two countries, we were still able to uncover some important lessons for future relief projects. Namely: There is a need for strong, shared information and learning systems among all partners in a recovery effort. Crises like these are continually evolving. Funders, governments, community members, and different aid organizations may each have different, and rapidly changing, priorities for recovery, creating a tempestuous sea of overlapping interests. The only way to weather such a storm of complexity is through collaborative learning among all partners – adapting and evolving to the constant change in resources and priorities in the crisis-affected area.  Such collaboration can range from new partner collaborations to shared data agreements and priorities, all things which present their own challenges given common drivers of practice.

We offer our analysis as a modest, first step. Increased urgency for immediate humanitarian relief must not mask the importance of longer-term development, just as the need for local ownership of development cannot eclipse emergency priorities. To connect these two mindsets as a ‘nexus’ of humanitarian-development work, and to help areas affected by crisis more quickly and effectively transition toward stability, we must continue learning from the effects of past relief projects on development. Some additional lessons from our analysis include:
  • It is possible to look ahead while addressing immediate challenges
  • It is possible to course correct
  • Governments will continue to play a central role in what they nurture and what they won’t
  • The ‘ecosystem’ of players within development work has a critical role to play – it’s not just coordination, it’s signaling and providing boundary references
  • Individual emergency projects will struggle to heed our call for more evaluation and learning, but large implementers and donors could play a greater role in identifying and resourcing critical learning questions, whether for evaluation or research
We boldly joined our voices to those of others who have been making recommendations on the ‘transition’ from shorter- to longer-term recovery, but we also took to heart some of our lessons and sought to integrate them in a new approach to designing WITH emergency programs to better identify project contributions for systems strengthening.


Our work continues... Welcome to 2021.


Elijah

visual: Unknown Author; licensed under CC BY-NC-ND

Sunday, November 17, 2019

Collective action for improving health services. Why the evidence will continue to be mixed and what to do about it.

2016-2019 has seen a proliferation of large studies, donor reviews and evaluations published on social accountability (especially in health), with seemingly contradictory findings. Yet, if we go behind the findings, there are key common lessons for practice, including those studies with null results. These lessons include, but are not limited to, the role of high-quality community-based facilitators, the value of collaborative approaches, and contributions to system strengthening and relationship building. We make a start in exploring these lessons across different studies. We aim to encourage new syntheses[1], going beyond randomized field trials. These are, badly needed to help guide practitioners and challenge researchers to explore questions, which build further knowledge, not on whether it works, but how, why, and in what circumstances.
Introduction

Social accountability[2] as a practice has different definitions, different frames of reference, different audiences and purposes. Different families of human development practice may lay claim to it, from human rights and social justice advocacy; democracy, governance, and participation; organizational science and systems dynamics; to more programmatic concerns for systems strengthening, quality of service delivery, and equity. Different framing will place social accountability as a goal, or as a process. Both operate on different timescales and levels of complexity. More importantly, both framings - goal and process - are legitimate.
Governments may react differently to these different agendas and framing, and, additionally, different layers of government may look at social accountability differently based on where “systemic pressure” is applied. We could also underline that a participatory process bringing clients and providers together to collaborate and problem solve, may generate genuine expression of social accountability (one party feeling accountable to the other within a dialogical relationship) without “social accountability” having ever been articulated as an approach, much less as a project.
By itself, this complexity can already explain some variance in demonstrated results. We have seen increasing research on the impact of social accountability interventions on access to and quality[3] of services, as well as experimental research on population level outcomes (mortality, malnutrition, etc.). After multiple single intervention randomised control trials, however, debate still rages within research, policy and donor circles, about their definitive impact for development outcomes per se.
Indeed, there was much collective hand wringing last year when two high powered randomised control studies in health landed with null results.[4] Yet another large RCT in India found the opposite - dramatic effects on stunting and vaccination rates.[5] 
Many social accountability practitioners (implementers, donors, and governments) have strong positions on its value from their experience, and practice is moving forward.[6] The risk for the community of interested parties is to see each positive RCT as a final validation, and every negative RCT as a disqualification. Evidence challenges are important fodder for scientific and intellectual debate. Research needs to continue, but we suspect that the final finding of “trial” approaches to evaluating social accountability will be “it depends” for a long time, and for a number of reasons.
Why a traditional reference to evidence will continue to be mixed.
Firstly, it is important to consider which outcomes are examined in these recent studies. In particular, negative findings came from RCTs testing for changes in population health outcomes. A number of donor evidence reviews, on the other hand – notably a DFID macro-evaluation, USAID’s Marshalling the Evidence review, and this year’s 3IE review suggest consensus for positive impact on intermediate service outcomes.
It’s when we draw the long causal chain to try and measure the development outcome that it becomes challenging. A blog is not going to satisfice the question of why that is, but there are inherent limitations to chasing the statistical significance of broad and somewhat distal systems interventions on health outcomes.
Secondly, the devil - or in this case, learning from RCTs - is in the detail. Social accountability is messy; we need to learn from all opportunities, while understanding the limitation of each study. RCTs are powerful tools, but relying on their finding without discussion of theory, context and model limitations, is probably one of the worst things we can do in applying science to development issues.[7],[8],[9] The complexity involved in genuine social accountability mechanisms include many necessary, but not sufficient context and process conditions, which may demand more dynamic and contextually grounded forms of investigation, from realist evaluation, to developmental evaluation. (All of which also have limitations, no doubt about it.)
Third, theory is important, and we are challenged to imagine management, political science, sociological, or systems science theories, supported by evidence, which would posit that complex human endeavours can succeed and endure without accountability. This accountability can take different forms (administrative, legal, financial, social), which can complement, or even substitute to another over time on a particular issue.[10] Some leading thinkers about social accountability have suggested that the argument should simply be framed around accountability itself.[11]
Furthermore, if accountability is a first principle for human systems, then the questions might have to move from “Does it work?” to “How do we balance different types of accountability to optimize results?”[12] “What conditions allow for social accountability to be stepped up when administrative accountability starts to lag?” and “How much can we get from institutionalized social accountability mechanisms?” “What are central mechanisms from data to negotiation, which optimize the interaction between state and citizen in the delivery of public health services?”
Finally, trial methodologies framed around testing survival of patients at a time T, may need to evolve seriously to consider the dynamical processes (at times T1, T2…Tn) through which an intervention (labelled “social accountability” or unlabelled but creating the space for it) can alter social structures and processes, which may—under certain preconditions—themselves contribute to: (1) improving a public service, (2) establishing a norm for sustaining a public good responsive to the expressed needs of citizens, (3) providing standing capacity to identify and respond to new challenges and conflictual situations, (4) bringing a new progressive framing—hence unanticipated by original researchers—in the conditions of delivery of a public service, and (5) what else?
We certainly have not circled the block on research and evidence on social accountability, but tried to shift a bit the direction of our collective discussion.
But now, what lessons can be taken from the range of studies mentioned above, whether they had positive or negative findings?
Here are some of our top suggestions:
  1. Community collective action occurs, with even minimum levels of support and encouragement. 
On this, the T4D / Harvard qualitative study (forthcoming) which will complement the published RCT showed how even a minimalist, time-bound intervention, while not ideal, still led to an average of five ‘social actions’ per village across 200 villages in Tanzania and Indonesia. That was approximately 1,139 social actions. If we consider what it takes to enable communities to speak up, then this is quite a positive outcome for the communities involved. The thorough qualitative work that has accompanied this RCT, mirrors much of what is captured in the grey literature for this work, and this makes it particularly valuable. Without this, the summary message that “it didn’t work” [to effect long causal chain health outcomes] could be seriously misinterpreted.
  1. Quality facilitation is essential.
Social accountability approaches need facilitators engaged with communities to support the civic education and empowerment processes required for communities to negotiate with their governments. Investment in facilitators is crucial, and has now been highlighted through several RCTs, notably, the India study earlier identified. The importance of facilitation has also been highlighted over many years by INGOs. A qualitative report for USAID in 2016 highlighted that progress in health service function was less where facilitation weaker. [13] 
As practitioners, we honestly hope the India study puts to bed what practitioners have advocated for a long time – and that budgets for facilitation will follow. The Indonesian Government and many state governments in India understand this and fund their own facilitators, albeit with their own challenges, indicating a critical ongoing need for non-government facilitators.[14] Indeed, a global call for community development facilitators in every village is the single compelling goal of the 5-year-old Community Led Development movement (CLD) made up of 70+ non-profits interested to progress community empowerment across agency and indigenous/international divides.[15]
  1. Non-confrontational collaboration is important and is valued by communities.[16]
There’s a large space for debate and definitional issues in addressing what is and is not “confrontational” in social accountability. We are not going to dive in at this point, aside from the short digression as an endnote.
The point that we want to note here is the positive finding for the value of non-confrontational collaboration, which comes from two in-depth studies, undertaken over the same period in Indonesia by Harvard University and for the World Bank’s Global Partnership with Social Accountability. It is also highlighted from a 3IE systematic review on citizen engagement, which found that:
“Citizen-service provider engagement is more effective when implemented through phased, facilitated processes that are framed collaboratively, as opposed to one-off accountability meetings that are interpreted as confrontational.”
This argues at a minimum that there are certainly ample time and space for the added-value of conflict-sensitive, problem-solving approaches to social accountability
A digression, if you allow--When governments, unfortunately not-so-rarely, decide to be blind and deaf to the aspirations of people, or of particular groups of people, giving power and voice to people will be more political and more confrontational at some point. This reflects a difference in framing of social accountability, as ‘projects’ versus ‘movement’. We are not going to debate here whether one form of social accountability is better than the other, or the possibility of an approach to social accountability “in the middle”.[20] We believe that different degrees of ‘confrontation’ will be needed at different times and in different places. Projects working within the framework of bilateral and multilateral agreements will likely be biased toward pragmatic and technical contributions—changing systems seemingly ‘from within’. Social accountability movements trying to defy bias, prejudice, and injustice remain much needed in the world, and will continue to be a force for change putting pressure on public administration. This tension is not exclusive to social accountability,[21] but certainly social accountability is a space for living with this tension. Projects have to tread carefully on political issues, while movements naturally seize political platforms more forcefully. We do not believe in disqualifying one or the other approach, or pitting them against each other. We are seeking progress through a “yes, and” dialogic approach.

  1. The first social accountability project supported by the Global Partnership for Social Accountability in Indonesia 5 years ago (completed last year) showed evidence of positive effects on systems strength, and relationship building with a network of groups and institutions, with shifts in power dynamics.
This obviously deserves more unpacking than we have space for here. Nonetheless, this is the first GPSA evaluation to capture it through a systematic concurrent evaluation, based on an initial theory[17] refined from the accrued evidence. GPSA has seen evidence for these types of changes in other projects, as have several other INGOs. An earlier Columbia University study in 2017 also highlighted positive system strengthening and development coordination effects in Zambia from the same intervention methodology.
Conclusion
These lessons will not be new to many practitioners. Indeed, they appear sometimes obvious after many years of field experience. However, the inherently political nature of social accountability interventions and their intangible processes make their research, monitoring and evaluation complex[18]. To forge ahead, we--practitioners, researchers, policy-makers, and donors--consequently need different syntheses approaches, with rigorous and realist frameworks, recognizing multiple sources of learning and knowledge, and not pseudo-experimental black box approaches, to forge ahead.[19] As more sophisticated questions are devised in context, as multiple knowledge sources are recognised and alternative methods supported, we hope to see deeper, more nuanced learning develop for social accountability. This learning will especially be needed from assessments within the different social and political contexts, to inform the adaptation and evolution of social accountability mechanisms. This is true for citizens within these contexts, and for governments committed to ambitious Sustainable Development Goals.


Sue Cant & Eric Sarriot                 Nov. 2019
Sue Cant is the Social Accountability Adviser for World Vision International. This blog post is the opinion of the authors, but the thinking evolved through many one on one discussions and, in particular, a group convened earlier this year, who we would like to acknowledge: Courtney Tolmie, Florencia Guerzovich, Marta Schaff, Vicky Bowdell and, in particular, Angela Bailey. These colleagues helped improve the text, and we maintain sole ownership of its shortcomings.

Notes:
[1] Note, excellent 3IE synthesis in 2014 (Rogers, at al 2014)
[2] Collective action or transparency and accountability or multi-stakeholder governance, or put simply, any activity that helps citizens make government accountable with a tangible result.
[3] Quality in the broadest sense possible includes dimensions of equity, appropriateness, etc.
[8] R4D’s rich discussion of its own study findings (which were largely negative) has been an exemplar of commitment to intellectual rigor. Tolmie C et al., 2018 Presentation to American Evaluation Association ‘Speaking Truth to Power’ Conference, November 3, Cleveland Ohio
[10] Who can argue that the social accountability generated by the #MeToo movement is leading to more legal and even administrative accountability? In global health, one of us witnessed first-hand a participatory community engagement process lead to the expression of community grievances for more balanced ethnic and linguistic representation of the communities served by care providers. Once public administration responded, this became part of the way to manage the district and administratively address this element of population needs. Sustainability of the outcome meant that social mobilization no longer was necessary to maintain the service outcome. Social accountability (those words were not used) had led to institutionalized administrative norm, at least on this one pain point.
[12] For quantitative researchers, maybe a Bayesian approach needs to be suggested—what do we already know about the requirements for accountability in human systems? How does that inform our next research question?
[13] USAID, 2016, The Maternal and Child Survival (MCSP) Program, MCSP Community Component/Tanzania Community Scorecard Documentation Report, December.
[14] Forthcoming, Guerzovich et al
[16] This is not to say that contestation is not important. It is, but there are different approaches at different times that may work. 
[17] This was informed by a systematic realist review.
[18] Not to mention their implementation, as mentioned above
[19] Realist synthesis is one such methodology
[20] See forthcoming paper by Florencia Guerzovich and Maria Poli, which we will link to when available.