Thursday, October 25, 2018

Social Accountability -- 2 short videos on a community scorecard for health services in Malawi

First a video featuring the excellent Helen Mwale (then with Save, and now with MSH) produced on the fly last year by the SC Malawi team, about the Community Scorecard approach -- Save the Children was one of the implementers of the USAID/SSDI project - in this case using the scorecard tool developed originally by CARE, and used globally by dozens of organizations.
[Of note: we had an interesting session at HSR2018 with Gail Snetro, Paula Valentine, and Allison Foster in Liverpool contrasting the scorecard and the Partnership Defined Quality approach. Both have critical elements, and my personal bet is that the methods might continue to converge. I also suspect that their differences actually provide a way to be more responsive to context, by allowing emphasis on different parts of a rich process. I'm also eager to hear about a full day session on social accountability, also at HSR2018, organized by Renee Loewenson.]
Enjoy this quick orientation video from Save the Children (2017):

And here's another (2013) CARE Malawi video, as part of the 'power to the people' series:


And here's a link to a series of presentations by speakers in a special session on Social Accountability held in 2017 at Save the Children's with the DC Health Systems Board. View videos.

Wednesday, August 22, 2018

The emergency-to-development continuum: left to right, or right to left?


As global health and international development professionals, our assumption is often that the continuum between emergency and development goes:
Emergency –> to –> Development,
left-to-right.

As global health and community health professionals, we have to or we tend to look at the situation as technicians, clinical technicians, social and behavior change technicians, health systems governance, and management technicians, leaving the questions of why fragility occurs to social change activists and advocates. We are now more often paying attention to public and private sector accountability to citizens, through the lens of the necessary governance and leadership of health systems, clearly delineated in a 'building block'. And we focus this attention on the need to ensure quality of services, the requirement that health systems be responsive to clients, so that utilization increases and our interventions reach scale. We trade very cautiously on these democracy and governance issues – actually, we generally like to focus on governance and say little about democracy. Our individual beliefs, motivations, and passions are still heavily humanitarian, socially-oriented, and democratic. But during office hours, we couch them in skilled technical language. After all, in terms of improving health outcomes, the best performing countries are not necessarily the most democratic ones.

Some of us at least, and certainly in polite company, enter the world as neutral technicians, even if we are passionate about democracy and global justice. And there’s a reason for it: our legitimacy and credibility stems from this neutral technicity; it allows us to come in, respond to human suffering, and act on the levers of health systems behaviors – including at community level – to redress the performance of health systems. We inherit a world full of emergencies, and now we are questioning how we can get better at moving along the continuum from left to right: Emergency – to – Development. That’s our job; that’s where we make a difference; and our activist, rambunctious, political self does well to leave the stage to the community health or health system professional that we are.

But what if the world was not going left to right, but more and more, in so many places, right to left:  development-back to-emergency, via a path of chaos?

Let’s consider examples related to both internal and external influences:
  •        A country successfully moving on MDG and now SDG indicators, but creating substantial anti-democratic spaces: regional imbalance, presidency-for-life, ethnic blind spots if not repression, leading to an undercurrent of popular discontentment, one day resistance, uprising, violence, implosion of the national model…    
  •       A relatively stable country with a functioning health system becomes a pile of rubble, sent back generations in its development due in large parts to foreign interventions and acts of wars, including by industrialized nations, who happen to also be well-meaning donors…
  •       Global economic models come to prominence, with long ramifications and ripples in terms of the economic choices of nations, the potential corrupting factor of massive amounts of funds, displacement of social investments away from the poor, and increased national blind spots about the needs of the poor and vulnerable.
  •       Unmitigated environmental destruction and global climate change hit large regions severely, leading to increased humanitarian crises, aggravated social, economic, and ethnic upheaval, increased numbers of refugees, and corollary decreased ODA funds for development.

If these are rare and unfortunate occurrences, then we are better off to leave this to the activists, peacemakers, and social justice advocates of the world. The world is still going left-to-right; our entry point into technical issues will allow us to play our role for progress, while ‘staying in our lane’.
But if these are trends and not exceptions, if more often than not such factors lead the world to go:
Emergency <– to <– Development,
right to left,...

...then our cautious neutrality may lead us to see a lot of our work undermined or even wrecked to ruins tomorrow. We will face constant degradation on a large scale, persistent and intractable emergencies, our left to right emergency-to-development continuum will revert to theory, overtaken by downward spirals and de-development. If this is the case, then our neutrality does not facilitate our work, but rather undermines the sustainability of any progress we achieve.

So, which is it? Left to right? Or right to left? And what does it mean for us?

Thursday, July 19, 2018

The sustainment index - a new metric to inform ex post sustainability evaluations.

Greetings -

Reeti Hobson and I have recently published a paper in BMC Health Services Research proposing a measure of sustainment -- the sustainment index.


I'm an unlikely candidate for jumping on one quantitative indicator to solve complex questions, but this is -- as the title of the paper states -- a 'simple metric for a complex outcome'. It is free of causal inferences; it is simply a measure of how much an indicator improved from Time 0 to Time 1 continued to progress from Time 1 to Time 2 (presumably after a transition phase / end of a project). 

We call it the 'sustainment index' and the paper explains this choice of language. 

We've already made a strong case that empiricism about sustainability needs to move from 'is it sustainable?' -- binary, either reductive or existential -- to two types of questions:
  1. Looking forward: what are the chances that progress will continue under new conditions (sustainability)?
  2. Looking back: how much of initial progress has continued after transition and can we start to understand why and how (sustainment)?
The sustainment index is simply here to provide a reliable and valid measure to this "how much" question.

The story of its development links back to an unsuccessful attempt at an unsolicited proposal, where the concept of this metric was developed. It's really put out as a invitation to researchers to start testing the metric. If you've already collected baseline, endline, and post project service or outcomes' data, it's extremely easy to plot out the sustainment index. If you're designing an ex post study, consider its value as a dependent variable.

An interesting property of the index, which is not discussed in the paper, is that while the performance of a health indicator is affected by the initial conditions, baseline (i.e. improvements are harder to achieve when the baseline is higher), the sustainment index--since it tracks the evolution of the speed of progress -- does not seem affected in the same manner. The graph below (not in the paper) shows 3 trajectories from 3 different baseline points of a health indicator, with the same sustainment index.

If we've made some serious researchers out there curious... then our job is done.

All feedback - and study reports -  welcome.



Eric [esarriot AT savechildren DOT org]
Thanks to my former ICF/CEDARS home for getting this through completion, including Reeti Hobson, my co-author, and Jennifer Yourkavitch. Thanks to our reviewers.

Saturday, April 21, 2018

Alma Ata Principles are turning 40

Johns Hopkins U and Future Health Systems have launched an anniversary event and movement on the theme of the 40th anniversary of Alma Ata, the enduring significance of its principles, and the challenges to its more comprehensive, some would say holistic, vision.

Visit the AlmaAta40 site and register to contribute.

Here is a 20 min interview of Dr Bishai that I carried out for our Save the Children team and the CORE Group.

Cheers all,

Eric

Saturday, January 13, 2018

Abt and Save the Children event on Governance and Accountability for Strong Community Health Systems event


As we recognize the importance of community health systems in expanding access to care, especially to underserved populations, governance has become a key consideration towards realizing Universal Health Coverage. 

Abt Associates, Save the Children US, and UNICEF organized a panel discussion on the latest initiatives to improve governance for community health.

Some time stamps on this event



@10:15 – start - Bob Fryatt (Abt) intro, referencing the “institutionalizing community health conference” which took place in J’burg a year ago.

@12:50-30:45 Tim Evans (World Bank) – important comments on the need for decentralized self-organization [my words] in health systems, particularly community health systems management, the plurality of perspectives and visions and negotiations, and unintended factors affecting centralized planning to achieve universal health coverage, and the necessity of autonomy of local service delivery units.  Q&A after that.

@49:15 introduction of first panel – UNICEF (kindly acknowledging “whichever organization was involved in organizing this event"), Global Financing Facility, and USAID private sector innovations in HIV/AIDS care and treatment. I was particularly interested in the overview of the GFF and USAID’s Joe Tayag’s (@1:12) examples and discussion of needs-based approaches to specific key populations and their aspirations.

@1:41 Second panel with Carolyn Gomes (Caribbean Vulnerable Communities Association), Lisa Tarantino (Abt) and your servant (Save the Children). @1:42 - Carolyn made a powerful case for the role of community based organizations in HIV/AIDS and how the bottom up can lead to change upstream. @1:54 - Lisa provided 3 interesting examples in community health and health governance and health financing.  And if you want to see me in full effort, go to @2:05 and hear all about ‘Beyond the Building Blocks’.



Sunday, February 12, 2017

Fiction or non-fiction in international development

I will not denounce the author of this iconoclastic drawing (Steve Hodgins), and I take all responsibility for the text that it inspires.

"Fiction" or "non-fiction"? Would we consider our wok in global health, global development as one or the other? Easy question. Right?

Well, I've long had this word stuck in my brain: "pretense". Maybe that's why Steve's drawing triggered me to post it and write a comment.

We often think of corruption as "them", these people who will take personal benefit from public efforts. This district manager who collected 90 days of per diem in just a month (truth or urban myth? but not impossible at all). This West African country where most workshops take place in a hotel in the capital, but just outside the administrative boundary of the capital city, so that everyone gets travel per diems. These many places where we allow "sitting fees" to discuss policy and programs with officials, because presumably they hold their population hostages, and, either we care so much about the populations, or we care so much about the success of our projects -- who knows? Maybe it's a mixture of the two. But anyway, these cases of corruption are--in a post-colonial mindset--"their" problem.

We talk far less of the corruption of pretense, which this picture illustrates so well.

"How does it relate to our work, really?" you may ask.
Of course all of us sometimes embellish a bit the promises of our tools, methods and proposals. But that's par for the course. We're only human. Development is a vocation, it's also a job, and -- yes -- a little bit of industry. What are you going to do? We ask.

But you see, words matter. And when our rhetorical inflation keeps "scaling up", then our words get further and further away from reality. At some point... that disconnection from reality matters. Maybe there's only so much that alternative facts can deliver...
That's why some people care deeply about evaluation by the way -- evaluation is nothing more than a methodological quest to move away from marketing and rhetoric to rediscover a little bit of truth when we can. (I often argue about the need for a diversity of methods of evaluation, and the appropriate nature of evidence given the question-context, but I never argue about its fundamental importance.)

So how bad can this wandering from "non-fiction" to "fiction" be?

I'll give just one example, because on this example I feel pretty confident that I have looked at the good, the bad, and the ugly of our work and evidence.

Here's a clear sign that you've entered the "fiction" section of international development: when you read somewhere about "ensuring sustainability", you know that you're in the fiction section:
- "ensuring sustainability" is like promising a free lunch, free money, or predicting the future. Mystics can promise it, but there's no space in scientific language for such a hyperbolic phrase. So when a donor uses it, it simply means "make us dream", or "we haven't thought so much about this, but we please tell us something that looks good". And when the proposal writer writes a paragraph about "ensuring sustainability", given that sustainability is about 1,001 necessary but not sufficient conditions, it only means "here's a selection of factors of sustainability that we think you will like as a story". Basically, it's pleasing fiction. To be sure, someone will have to come up with a list of indicators, because in that fiction section of the library someone has come up with a list of indicators of sustainability which are -- you know it - "SMART". Of course, there's not one study in the world to validate this list of indicators, but remember: we are in the fiction section. As long as there's a fit between the empty promises of the proposal and the empty request of the RFP, it's still a good score on the proposal. All is good.

Now, projects happen in the real world. And there's a lot of pressure on them. Project managers are making 34 decisions a day (I made up that number just now; I like it), and they have a very demanding job. Usually -- and I don't blame them -- slowing down the project to address all the demands of negotiation, consultation, conflict management, and forward thinking required by sustainability is not really something that they can do. Their decision space is more about reducing the unintended effects on, than maximizing, sustainability.
Six months from the end of the project, there will be a lot of attention paid to "handing over" and "transitioning". Too late, I'm afraid.
And the final evaluation will state, as it has stated 4,685,904 times before (I love this new world where we can make up our own numbers!) that -- wait for it:
"sustainability...
should...
be ..
thought..
about..
from ...
the...
<you know it's coming, right?>
beginning."

Bingo!

But you see - until that moment - we were looking at sustainability in the fiction side of the library.

That's a form of corruption of our mission and efforts -- not the only one -- but think about this cartoon next time you see the words "ensure" and "sustainability" in the same sentence. (And maybe you'll find other examples where we corrupt our best judgement and professional thinking.)


Eric

Thursday, January 26, 2017

The no-blueprint blueprint to development

Eric Sarriot 

Note: I moved last August to Save the Children in Washington DC, where I now work as Sr Health Systems Strengthening Advisor. I look forward to continued engagement with my friends and former colleagues at ICF/CEDARS.

It’s a little known secret that I own a network of low altitude satellites which monitor every workshop and conference on sustainable development in the world. Like every day.

I took a random representative sample of all 784 workshops and meetings on development that were held last Tuesday and did a textual content analysis. The following exchange was recorded verbatim 1,458 times, which corresponds to 1.86 times per meeting with a 95% C.I. from 1.26 to 2.18:
  • “Thank you to the panel for presenting an interesting approach, but I don’t think this can be applied with a cookie cutter in every [country | province | district | commune | village],” said one participant from ‘the field’. To which a first panelist replied:
  • “Obviously, we are not proposing this as a blueprint. There is no blueprint to this complex issue.” A second panelist interjected:
  • “I totally agree, there is no blueprint. Our approach needs to be adapted to the context.”

STOP!!!!!!!!!

My satellite monitoring is equipped with internal logic contradiction sensors, and these sensors were systematically triggered by this last statement.

Funny story actually about these sensors--when I had first installed the logic analysis program, I struggled a bit. I was getting error messages like “logic routines not applicable to development work,” and “analytics must be supported by either evidence or shallow catchphrases supporting comfortable intellectual habits.” I had to upgrade the software to accept logic again. I’ll spare you the details of programming, but it involved encoding into bits Aristotle, Descartes, and Einstein’s thought experiments. Hard work, but I’ve digressed.

The root of the internal logic contradiction is the simple fact that it’s only blueprints that you need to adapt to context. So, if you’re going to adapt to context, don’t tell me that there’s no blueprint. Say: “we intend to adapt the blueprint for the context.” And that leaves unsolved the question of what to do, when there’s actually no blueprint. But let’s take it a step at a time.

The beauty of blueprints
‘Blueprint’ is actually a metaphor in development—not a real thing. Martin Reynolds of the Open University in the UK regularly points out that we should not “mistake the map for the territory.” So, let’s start with what a real blueprint is actually for.

A blueprint is a document which details the way to build something, and shows how to arrange different sub-systems (drywall, electrical, water pipes, ventilation, etc.) of a structure. It’s great to have a blueprint, because someone has thought through and tested configurations of these sub-systems and made sure that they all work together to provide integrity and functionality to the structure. It spares you from [new metaphor coming] reinventing the wheel each time, and taking advantage of evidence-based best practices. Consider my neighborhood, a lot of houses were built on the same pattern in 1940. Only small variations due to topography existed when the houses were built, but people have been building additions, knocking down walls and modifying them ever since, so every house is now a little bit different from the next one.

I want to finish my basement and need to figure out how to do it. Lucky for me, my neighbor did her basement and let me look at how she did it. I’m happy with what she did and I’m going to use it as a blueprint for my own basement. Since our two houses are not exactly identical (our basement stairs were put in different places for one), I will have to adapt her blueprint to the specific context of my house.

So, blueprint: great. Adapting to context: of course. It’s not either or. It’s the latter because of the former. If there was no blueprint, I would not be adapting my neighbor’s approach, I would have to imagine something different for my basement.

And the same applies to global health. Consider just a couple of examples:

IMCI (the integrated management of childhood illness) was a blueprint. One could argue that because people always ignored the health systems strengthening element of IMCI, basically because the blueprint was not respected, IMCI was considered as a failure. I know this is a long debate. Another example? iCCM (integrated community case management), infection prevention and control, prevention of post-partum hemorrhage in health facilities and with misoprostol in communities, the childhood immunization schedule—all countries adapt those strategies or intervention packages, but there is an unmistakable blueprint. Even some more complex non clinical interventions have blueprints, sometimes tacit or enshrined in legal documents and policies. You want to run an NGO (non-governmental organization) to deliver a public good? Well, there’s a dominant blueprint that you need to have executive leadership, held accountable to some sort of a board, and a financial accountability and oversight structure. After that it gets messier, but those parts seem to be based on a blueprint that is accepted for the robustness and risk mitigation they provide to the organizations. And—again--it always has to be adapted to context.

So, before I tear some of this down, let’s recap the major points so far:
  • Blueprints are useful and they can help us be efficient and avoid re-inventing things that have been tested and validated through empiricism and accumulated human experience and wisdom. 
  • All blueprints need to be adapted to context. It would be utterly ignorant not to adapt to context, the worst kind of hubris. Even the science of management has long accepted contextual management as a requirement. I don’t think that I need to get into an inventory of the ‘white elephants’ of international development at that point. (Do we love our metaphors or what?) 
  • So, please don’t ever brag again about adapting to context. And do me a favor; next workshop you attend, when the panelist says, “there is no blueprint. Our approach will be adapted to the context”, please rough him up a bit and just make him stop. It really messes with my satellite monitoring analytics, and I can’t have any of that.

When there’s actually no blueprint
Let’s go back to our panelist and participant from ‘the field.’ [Spoiler alert: I may caricature the differences in perspectives to stress my point.] The panelist actually has a blueprint, a plan, an idea, an intervention, which he believes is now tested and proven to be able to deliver a public good. Variation in contexts is a challenge--an adaptation and implementation challenge--to be able to deliver what he knows can work and to take it to scale. The statement “there is no blueprint”, we now know, only serves to control the unpleasant complexity and skepticism of the participant, but what the panelist really wants to apply is definitely a blueprint. Adapting to context is a bone he is throwing to these pesky field people who don’t know any better and would have us to boutique projects all the time.

The field participant, on the other end, is immersed in a context. She is richly informed about the geography, history, politics, micro-social and societal reality of that context. The level of complexity increases with the level of attention paid to details, and our participant has seen over and over again when approaches cooked outside of her context have failed on the cliffs of that complexity. (This metaphor at no extra-charge.)  What she really is hinting at is not that a solution needs to be adapted to context, but that a solution needs to be developed, created, imagined, and invented in the specific context where the problem is identified.

Those two views are not solved by the adapting to context platitude; they represent very different approaches to problem-solving. So, who’s right?

Well, no surprise here; the answer is… it depends.

As we have just seen, there’s a beauty and value to blueprints. But there’s also a world where contextual design and innovation dominate. And it is underappreciated in central / global spheres of decision making of global health. As many things are, there is a continuum to navigate, but the dominant model of our work is blueprint thinking. The necessary and productive intellectual discussion about where blueprints fail us and when we need a different type of thinking is too systematically squelched. This could be due to power differentials between the center and the periphery of all our systems, and to blind spots emerging from our different points of observations.[1]

There are a couple of models out there describing where complex takes over complicated in the problem definition and solutioning space. One of the most famous is David Snowden’s Cynefin model, which represents problems from simple, to complicated, to complex, on to chaotic. When problems are complicated, best practices can be identified and promoted through protocols and, yes, adaptable blueprints. But when we enter the space of complexity, emergence takes precedence over best practices. I once tried to map out how complexity increases in the definition of global health problems, based on work by Geyer and Rihani. It turned into this table, which might provide a concrete illustration.

The more your problem is on the right side of the table, the more useful will be a blueprint, if used with smart adaptation to context. But as you move to the left, the value of the blueprint decreases. At some point, adaptation is no longer the solution. Creation, invention, context-based design become the requirements. This means that you start from the perspective of the context actors, as opposed to that of the global experts.


Figure: increasing level of complexity in problem definition from right to left (Source: http://link.springer.com/article/10.1007/s11213-014-9329-9)

A clarification: I am not claiming that this is the way to determine whether a blueprint is appropriate or not. But I suggest that more often than not, problems on the left side of the table will not be amenable to blueprints, even if they may incorporate sub-issues where a valid best practice or blueprint is available.

In conclusion, let’s acknowledge that “there’s no blueprint; we need to adapt to context” is an illogical statement, used sometimes with the best intentions, but also too often as the expression of a central-planner bias preventing an intellectual debate that we badly need. In the absence of a blueprint for figuring out whether a blueprint can be used, maybe we can start by listening to the question of the field participant with a little less condescension and a little more intellectual curiosity.

So, make sure to bring that up at the next workshop. And remember: my satellites are watching!




[1] I’ve probably been led on this trail of thinking following a presentation that I made in June 2016. The topic of it was about blind spots in global health, specifically blind spots to self-organization. Definitely some overlaps. The summary of the presentation is available here: http://www.witpress.com/elibrary/dne-volumes/11/4/1273