Enabling Data-Driven Innovation for Global Development

A longstanding criticism of the international development field is its unrealistic desire for certainty and over-dependence on linear planning. In a complex and changing world, many of the institutional policies and systems that prioritize rigid forward planning, strict fiscal discipline, and routine quantitative evaluations end up hamstringing programs and undermining their impact. A new movement seeks to counter this pattern: “Adaptive management” is the practice of adapting and iterating programs continuously, throughout implementation, in response to observed shifts in progress. Especially in a new era of technology-supported, data-driven programming, adaptive management is more feasible than ever before.

That’s why the Global Development Lab at USAID is committed to increasing the practice of adaptive management throughout the Agency. The Lab was created with the mandate to harness the power of science, technology, innovation, and partnerships to accelerate development impact across the Agency’s work; it realized early on that USAID’s ability to innovate and exponentially increase its impact is largely influenced by its ability to practice adaptive management. However, it was clear that a number of institutional barriers prevented programs from working adaptively. And it was even less clear, in the complexity of USAID’s operations and contexts, which of these institutional barriers were the most impactful, and how USAID might best prioritize strategies for change. To find a way forward, the Lab engaged Reboot to analyze these barriers and advise on strategies for addressing them.

Reboot’s signature approach to institutional ethnography blends qualitative methods to understand formal processes and informal practices alike. At USAID, we focused on understanding the “positive deviants”—those staff and partners at missions who are overcoming institutional barriers and practicing adaptive management in some form or another to amplify their development impact. We were particularly interested in those using digital data or technologies to support implementation, as this is an area where the Lab is uniquely positioned to support.

Select services for this project included:

• Institutional Ethnography

  We examined 17 programs across three missions (Indonesia, Uganda, and Zimbabwe) that were pioneers in their approaches to evidence-based, adaptive programming. Researchers explored each program at a granular level across its implementation cycle, reviewing program documents and conducting semi-structured interviews with stakeholders involved in delivering or benefitting from each program. We gathered data from more than 200 respondents, including USAID staff, implementing partners, government partners, and beneficiaries. We prioritized capturing actual practice instead of pro forma procedure. And we developed a rich, nuanced portrait of the personal, contextual, and institutional factors that shaped the work of USAID’s missions and of its partners.

• Communications Design

  Given the complexity of the organization’s operations, we created visual frameworks to share the insights and implications of our ethnography with stakeholders across USAID. Through clear, compelling visual design, we presented an internal model for USAID highlighting five key inhibitors to adaptive management. Our visual “maps” showed the relationships between various actors at USAID as well as the larger ecosystem in which the Agency works. Most importantly, we illustrated the relationships between institutional incentives, culture, data systems, human resource policies, and staff capacities that were at the heart of our ethnography; this made it easy for stakeholders to quickly grasp the implications and assess the impact of various interventions.

• Strategy Development

  Over the course of this engagement, we provided strategic advisory to USAID’s missions and the Global Development Lab, as well as to the Bureau for Policy, Planning, and Learning. We surfaced opportunities for individual missions—already innovators in their own rights—to refine their policies and practices. To spread these insights, we provided briefings and reports and supported discussions with USAID executives across the Agency. We advised USAID on how this work could be carried forward in new initiatives, including through new processes and tools. Working especially with leaders in USAID that were developing new policies and program guidance for the organization, we helped identify and resource options for enabling adaptive management across the Agency.

While most of our specific findings are internal to USAID, our ethnography also has important implications for the larger development field. Most importantly, we created a detailed portrait of the dynamic of “upstream (ac)countability,” a common practice of measuring any program’s success primarily on the degree to which it achieves pre-established, quantitative targets set for the benefit of oversight bodies. Because those oversight bodies are disconnected from the work on the ground, these targets are often oversimplified and represent what is “countable” instead of what truly indicates impact. Our research showed that the most successful programs—the positive deviants—are those with the flexible incentives and structures necessary to pivot away from pre-established targets when evidence from implementation offers new information.

Our work led to in-depth discussions with USAID executives on institutional barriers to innovation, especially in light of a rapidly shifting development landscape, and focused on opportunities to tackle the structural issues the Agency faces instead of just the symptoms. We also identified specific, near-term opportunities to address culture, management structures, procurement practices, human resources policies, and approaches to technology. Our recommendations have since informed leadership, organizational design, and change management dialogues at USAID; by providing a detailed portrait of those “positive deviants” who are creating change despite the odds, our work helped advance evidence-based strategies for USAID to maximize its impact.