Developmental Evaluation for Public Sector Innovation

When the Digital Strategy Unit in the Office of the President of Mexico set out to implement an ambitious new program in government innovation, the team also took the innovative step of engaging a developmental evaluator, seeking a partner to provide advisory support and evaluate its work in real time. This evaluation would both inform ongoing implementation, and contribute broader lessons to the field. The team invited Reboot to fill this challenging role by closely accompanying the implementation of one of its most significant programs, Innovation Agents, an open government and civic technology fellowship. 

The Digital Strategy Unit’s desire for such a transparent and responsive approach to learning matched the program’s ambitious vision: To make the Mexican government a platform for innovation. Innovation Agents worked toward this goal by attracting experts from outside of government to partner with “insiders” from five government agencies. Together with support teams, these pairs of “Innovation Agents” set out to develop technology-based solutions to pressing problems. As the Digital Strategy Unit’s leadership guided the five teams and managed this complex program, Reboot provided guidance on strategic planning and program management, helping them to navigate the many unexpected challenges that attend the trailblazing work of government reform.

Reboot approached the project by prioritizing personal interactions with program staff and project teams, investing time in the relationships and experience necessary to truly understand the nitty-gritty details every step of the way. We also advised the Digital Strategy Unit through structured reflection workshops as well as ongoing dialogue and weekly advisory meetings. Reboot was aware of the interaction between our dual roles as evaluators and advisors. Recognizing that our involvement influenced the process as we evaluated it, we developed guidelines to structure our role and consciously monitor our potential influence.

Select services for this project included:

• Developmental Evaluation

  From program launch to wind-down, Reboot conducted design research on each of the five Innovation Agents teams, interviewing them one-on-one, and observing them at work. Our in-person, high-frequency interactions earned our researchers the trust of respondents and gave us deep familiarity with the ongoing function of the program; it also kept us abreast of the rapidly-changing implementation context (a hallmark of pilots). We also looked beyond the Innovation Agents program at the Digital Strategy Unit’s broader portfolio, conducting research into three concurrent open government programs; this additional research helped validate emerging findings from the program against other approaches, and strengthened the analysis to be more relevant to a broader practitioner audience. The findings from this research fed into regular advisory meetings with the Digital Strategy Unit leadership and helped guide implementation.

• Workshop Design and Facilitation

  Reboot conducted four structured reflection workshops with various groups within the Digital Strategy Unit, each covering a specific point in implementation. For example, an early strategic communications planning session helped participants understand how to frame their program’s value for different target audiences; a later workshop helped determine how to close out the program for maximum impact, and to plan the design of the next generation of Innovation Agents.

At the end of the pilot phase, the projects within Innovation Agents as well as the broader program portfolio of the Digital Strategy Unit had both successes and disappointments to share. Our research offers valuable insights into the pitfalls that held some of the pilots back: How to find the “sweet spot” of political support; how to tap into non-traditional revenue streams (and avoid the suspicion inherent to non-traditional funding); how to manage the time commitments of participants; how to collaborate with government “insiders” as an “outsider,” or vice versa; these and other challenges manifest in distinct ways for government innovators in other places and contexts.

Through the hard work of the Digital Strategy Unit and the project teams, these practical lessons learned are now resonating with civic innovators around the world. At events like the Open Government Partnership’s 2015 European Regional Meeting, practitioners expressed excitement and feelings of validation as they recognized their own experiences in the research. Reboot is also sharing a companion resource, “Implementing Innovation: A User’s Manual for Open Government Programs.” The guide speaks specifically to the people working at the frontlines of public sector innovation. Reboot will share these lessons at the 2015 Open Government Partnership Global Summit, and looks forward to continuing the conversation with innovators and reformers worldwide.