The clearest representation of healthcare is a doctor sitting with a patient. Whether it’s a routine checkup or a serious procedure, direct interaction is central to how health care happens. If you want to improve health outcomes, this interaction is a good place to start: putting new technology in the doctor’s bag, ensuring vaccines or essential medicines reach patients, or simply building new clinics.
But, of course, we know that there are hundreds of other people who make that interaction possible: diagnosticians, administrators, insurers, janitors, receptionists, truck drivers, cooks, regulators, researchers…the list is endless. These are the people who compose a health care system. And the design of such a service delivery system can dramatically impact the effectiveness and efficiency of the care provided.
The global health sector is paying increasing attention to the systems that ensure care reaches people who need it most. Last month, experts from several major NGOs and donors gathered at the Center for Global Development to talk about investments in global health and how an enabling environment can make investments more effective. Though a bit meandering, the conversation frequently came back to the importance of systems strengthening, as I discussed recently on Humanosphere.
However, the event left a big open question around measuring the impact of systems strengthening efforts. This measurement is challenging because health systems are incredibly complicated. They involve a wide range of public and private actors, their performance depends on countless exogenous factors, and they take different forms in every country depending on the population being served and the other institutions in place.
Health systems strengthening may be able to learn from measurement in broader governance reform efforts. There is already a great deal of overlap between the two. When I posed the question of impact measurement to a few of the panelists, Karen Cavanaugh of USAID commented that other development sectors are probably ahead of healthcare in terms of systems-level indicators.
Generally, governance indicators can take two different approaches:
Measuring governance itself usually means looking at the rules and processes of government—for example, noting whether certain laws are on the books, whether the legislature passes a budget, or whether certain agencies exist and have funding—as well as how government functions in practice. This sort of measurement can be fed smoothly into diagnosis (you can quickly identify the gaps) and reform (you can design and advocate for changes). However, because governance is measured against how government “should” operate, this carries a normative bias that can be hard to justify. It may well miss many positive capacities of a governance system that fall outside its framework, and potentially encourage isomorphic mimicry (the creation of institutions that look like they meet international standards, while performing entirely different functions).
Measuring the outcomes of governance, on the other hand, guarantees a focus on performance. What did the government actually accomplish? This necessarily pushes into other sectors; governance itself accomplishes nothing, except as it improves health outcomes, educational opportunities, employment, peace, or some other end. For example, Matt Andrews has proposed under-5 mortality or birth registration rates as possible governance performance indicators. Though these performance indicators can lead to cross-country comparisons and support advocacy, they aren’t well-suited for diagnostics or attribution for reform efforts.
Clearly, neither approach to indicators is fully satisfactory on its own.* We need each to serve as a check on the other. Measuring outcomes ensures that our measures of governance itself are actually relevant, while measuring governance itself gives us more actionable guidance than outcomes alone.
Translating this back to health systems, we do need both kinds of indicators. The outcomes are somewhat straightforward, and they are the same outcomes that health care is always concerned with, such as reducing disease incidence or saving disability-adjusted life years. Health systems-level indicators (analogous to measuring governance itself) also exist (see the World Health Organization’s handbook on indicators for health systems). Improving a governance system requires a combination of both types of indicators, supported with research to ensure that they are actually linked in a given context.
The need for both kinds of indicators also points to the need for humility about what indicators can accomplish. Indicators are great tools for managing linear, cause-and-effect relationships, but governance and system reforms are far from linear. Relationships between system elements are often complex, dynamic, and dependent on feedback loops that can be easily misunderstood. In such a context, indicators merely point the way to problems. Designing solutions requires deeper engagement, including a certain amount of iteration and muddling through.
This deeper engagement doesn’t lend itself to indicators or a value-for-money mentality, and it can seem far removed from the direct interaction between doctor and patient. But herein lies the hard work of systems strengthening.
* Bill Savedoff refers to this distinction as governance determinants and governance performance. Somewhat analogously, a recent ODI paper from Marta Foresti and Leni Wild call this form and function. Others refer to it as rules-based and outcome-based, with slight variations in emphasis.
