Country Ownership and its Measurement - Part 3 of now 4

This post is long overdue. In Part 1, we considered a bit of the history behind the current emphasis on country ownership—at least it was current when I started; in Part 2, we spent a little time on the crucial question of why we would want to measure country ownership, and we discussed some implications:

When it comes to ownership, the old saying “if you can’t measure it, you can’t manage it” begs the question: “should you be managing someone else’s ownership?” Since the obvious answer is “no,” we need to realize that developing, cultivating, or allowing ownership happens within the tension of desired transitions in roles between different entities. If we avoid the pretense that we are somehow “objective” and outside of the game of this transition, as donors, technical advisors, or evaluators, Part 2 concluded on the following question :

If ownership grows or withers from the net result of an interaction, a dialogue, a transfer of resources, an exchange in capabilities, the negotiation of roles in decision-making, then what is the point of ownership measurement if it exclusively focuses on the recipient? If I am the provider of assistance, or the donor, or the policy adviser, does it make sense for me to try and measure the recipient's ownership without questioning my role in this process?

This last blog is an attempt at the what and the how of measurement. Remember that we are dealing largely with monitoring and evaluation of country ownership, rather than research. In the former case, information (signals) has to be produced in a time frame and with a frequency aligned to management processes so that it can be used to inform management decisions. This is very different from a research exercise.

I hope to convince you that how we measure country ownership cannot be dissociated from why we measure it. And this has profound consequences.

Let's start at the very beginning (Von Trapp, Maria. 1965). Measuring ownership is going to require solving a great many questions, but first and foremost it's going to require making two choices. These choices are always made, but they are not always explicitly made with respect to alternatives and opportunity costs: 

1. The first thing we do with something complex is to break it down (reduce) it to elemental components. And we're going to have to decide which 'parts' of ownership we're going to focus on.

We saw in Part 1, how PEPFAR identifies four main components to ownership, namely political will and ownership, institutional ownership, capacity, and mutual accountability. I discussed briefly how there are circuitous relationships between these different things (ownership is part of stewardship, which is part of or the same as proper governance, which is part of capacity, which is part of ownership, for example). There’s also a nested Russian Dolls element as well (political ownership needs to translate into institutionalization of key values, processes, etc, which needs to translate into operational capacity and capability, which can be broken down into a number of capacity areas).

This is not a critique of any single assessment tool or conceptual model, but simply a recognition that, for one thing, we are trying to assess something which cannot be fully assessed, given that it involves layers of both institutional and human intentions, actions and reactions. We are trying to capture observable signals about something, which is at the same time social, psychological, institutional, and political. When trying to assess a whole by breaking it down, we gain in precision (reliability) in measuring specific pieces, but we lose in validity because of the glue and connecting pieces we are forced to discard as we proceed. It may be acceptable, even necessary, to do this, but we need to face that this is what we are doing.

There are other ways to look at ownership which are as valid as PEPFAR's approach. And much like capacity, each component of analysis needs to be broken down into more pieces. Here are some categories and sub-categories of analysis that have been proposed and used:

• The four components above (political will, institutional ownership, capacity, and mutual accountability) are examined in terms strategy, resource allocation, operational planning, implementation, and M&E;
• Other efforts have looked at 'readiness to implement country owned solutions' by considering country leadership, ownership and advocacy; conditions in the policy and planning environment; institutional dynamics: management, coordination and implementation; as well as the culture of learning and knowledge-based practices;
• Mutual accountability relates to a host of relationships: citizen, donors, internal accountability, with a number of sub-domains, notably finances;
• Ownership also translates at different system levels. In one of our efforts to assess ownership at district level, we had to consider operationalization of the national HIV plan at the district level (very close to the concept of institutionalization); institutional coordination of HIV care and treatment activities; and congruence of expectations between levels of the health system.
• We have also tried looking at stages of transition in roles (shared or divided between "external" and "internal/country" planners, implementers). Elements under assessment included health planning; service delivery; specific management functions (finances, supply chain, laboratory, human resources; supply chain management; laboratory management); training; health information; and capacity building.

As you can tell, we were trying to save some of that glue and connecting pieces that fall off the side. The point of this list is to show the breadth of elements that can be considered, and the potential depth of each element, not to mention layers (geographic and institutional) of systems and sub-systems. We break down the concept to simplify our life, and then realize that it's "bigger on the inside" (Who, Dr., 2007).

Nonetheless, this is the first choice and the first step of awareness:

• After we've mapped out the boundaries of our exercise / assessment / research, how are we going to break down the concept of ownership?
• How much of the glue and connecting pieces are we ready to lose in the process?
• How far down are we going to drill down?
• What will be the level of effort required by our measurement effort depends on our answer to these questions.

2. The second question is about our measurement process, and how 'emic' (from the inside) or 'etic' (from the outside) we will be in measuring ownership. Ok, in simple words: who's perspective are we taking in assessing ownership: that of external agents, or that of the purported owners? Are we trying to be objective or participatory?

You may have noticed that I gleefully dodged the issue of what type of metric we construct--this would require a longer treatment, and for now many approaches are on the table and none have been validated better than any other. Among the options:

• Scorings and ratings about perceptions of ownership in various components;
• Multi-item response indices based on discrete numerical scales (1-5; 1-10 usually), or anchored scales (where each numerical response corresponds to a descriptive textual "anchor" or "word-picture");
• Additive scales based on meeting a number of criteria (yes/no);
• Full qualitative analyses;
• Etc.

It is important to pick one approach, design, validate and use it appropriately, but the question I want to deal with here is far more fundamental. Let's say that you've stuck with four components of assessment of ownership used by PEPFAR at some level of a national social service system (health district, school, social welfare department, etc.), the key question will be whether you take one of two paths:

a- Attempt to measure ownership [of a program, policy, or project, in an institution or system] as objectively as possible, without bias and influence from the agents inhabiting the structures being assessed; or

b- Measure ownership with the actors and stakeholders-subjects of the process of taking ownership.

Now the first temptation—I use the word temptation because it usually leads to sins of program design –the temptation is to go for, "we'll blend the two approaches, and get a middle of the road tool."

I want to use a metaphor to illustrate how the temptation of a ‘middle of the road’ approach can be self-defeating.

Imagine that the methodological space we must travel from one measurement approach to another (objective versus participatory) is a landscape that we are going to walk across. In Figure A below, this landscape is a flat plain, where we can easily move from one point to the next, and stop wherever appropriate. There is an objective end of the spectrum (some sort of reliable external audit) and a subjective one (maybe asking five key informants, "how much ownership do you think there is in the province on X? Thank you."). The path between the two ends is flat, so there is no cost, or very little, to moving and stopping along. If a compromise is needed, all we need to do is move the design cursor to the ideal fair and balanced position to enjoy the best of both worlds.

This would be very nice, indeed, and it’s a pleasant illusion when your livelihood depends on maintaining the illusion. But I suspect that most choices about measurement are made on a different type of landscape. Imagine that we are not faced with  a nice clean plain, but instead have to walk through a mountainous landscape, one that looks more like Figure B. The path between the two ends is rugged, with peaks (B4, B6, B8) and valleys (B3, B5, B7, B9). It’s hard to push over a peak, and once you do, you roll down very fast into the next valley. Consequently a small inflection in design one way or the other will lead to quickly losing a lot of the features of the previous design [side note for the geeks, the peaks are “tipping point”].

What is important--essential--to understand here is that if we most likely live in a B-type world, rather than an A-type world,  we don’t get to make small adjustments to our hearts’ content, but rather we observe (or ignore) that small adjustments to our methods have big consequences. As soon as we move across points B4, B6, or B8, what results is not a small change in the balance of our methods, but a jump which can be far more consequential than we think (i.e. from B5 to B7).

(Note that our discussion has focused only on the ownership question, leaving aside that of the change process, called transition. We’ve had interesting efforts working with PEPFAR/CDC trying to move forward on this, but I don’t think it’s quite cooked and ready to serve yet. Only to say that providing information about the two sides of the equation makes a lot of sense and only strengthens the argument for a learning approach.)

Stay tuned for the next and final (!) entry on this blog series focusing on country ownership which will explore the practical implications of this discussion.