Collaborative Learning Networks and systems change: Tackling a complex measurement challenge

Editor’s note: The following blog is the first installment in a three-part series exploring measurement and learning for Collaborative Learning Networks in accompaniment with the recent publication of Results for Development’s Collaborative Learning Networks Measurement & Learning Framework.

Over the past few years, there’s been a growing push for international organizations in the global development and social impact community to support local systems strengthening through a catalytic-yet-indirect role.

As such, many international non-governmental organizations (INGOs) have shifted from being the “doer” toward a more intermediary or facilitator role. By making this shift, INGOs are often working to convene, connect and support networks of partners who are more directly fueling change in their local systems. One such example of this shift is Peace Direct’s “Nine roles that intermediaries can play in international cooperation.”

This is an overdue — yet positive and sustainable — shift for the sector; however, it poses a particular measurement challenge for INGOs who want to understand and, where possible, estimate their catalytic contributions to systems-level change.

How might we measure and assess such indirect and distributed contributions to long-term system change? And how might we learn from and communicate the successes or failures of this empowering approach to drive impacts at local and global levels?

In this blog series, we will explore the challenges and opportunities associated with assessing the progress and impact of Collaborative Learning Networks (CLNs) for systems change, as well as Results for Development (R4D)’s specific approach to facilitate change with local actors within and across countries.

Why measure and learn about Collaborative Learning?

This question has been front of mind for R4D staff as we searched for a way to record the progress and contributions to impact of our 15 years of working through Collaborative Learning Networks (CLNs).

CLNs convene diverse stakeholders — government officials, sector experts, civil society actors, technical partners — from different countries or regions around a common or shared purpose in a way that allows them to learn from one another while co-creating knowledge and tools to address common challenges. Complementing the expert-driven Technical Assistance (TA) models, it involves collaborative action or implementation-oriented research across countries and sectors, applying peer learning and working directly with relevant in-country institutions that can help develop the supportive ecosystems needed to bring about long-term systems reform.

Since 2009, R4D’s portfolio of CLNs has expanded and diversified, experiencing heightened demand during the COVID-19 pandemic as country leaders sought insights from their counterparts in other countries on navigating the rapidly evolving pandemic landscape while sustaining routine services.

Most CLNs in R4D’s current portfolio work in health, including the Joint Learning Network for Universal Health Coverage; Linked Immunization Action Network; and the Strategic Purchasing Africa Resource Center (SPARC). Some CLNs work on education and/or other social sectors, e.g. PEERSS and SALEX. But Collaborative Learning is a highly adaptive approach that can be applied for joint problem solving within and across public sectors and regions, from local to global levels and across regions.

For example, JLN for Universal Health Coverage is a CLN of policymakers and practitioners from 30 member countries that have been actively engaged for over a decade in co-developing pragmatic solutions to implementing universal health coverage reforms through practitioner-to-practitioner learning. Collaborative Learning is an intangible, silo-busting, and deeply participatory approach. This means it’s very effective in supporting changes in local systems, but it is very difficult to define through standard indicators. Like other approaches that support systems change, it is a joint and collective effort that contributes to shared impact.

What’s more, the impacts are layered or happening at different levels.

CLNs generate network-level outcomes in terms of the knowledge and solutions gained and implemented together across borders, influencing in-country systems change agents’ policies and practices. Moreover, CLNs generate systems-level changes in building commitment to — and ownership of — a shared vision and pathway for reform. This commitment and ownership also help build momentum for change in the specific contexts of countries’ systems (i.e.., reforms within a country’s health systems). Third, CLNs seek to generate cross-network learning and knowledge outcomes that influence global development thinking and practice (e.g., best practices and models for funding and support of locally led, demand-driven and contextually relevant solutions).

Funders, practitioners and network members or managers alike all have a need to understand these multi-layered outcomes and impacts. Network managers want to ensure their network is growing, maturing and supporting Collaborative Learning and knowledge generation at the network level. Funders want to have evidence that CLNs are an effective and sustainable model for joint problem solving. Practitioners want to implement best practices for shared learning with their peers at local, regional and global levels to support continuous improvement of their technical, social and political work. Examined together, these learning needs are vast and varied yet intricately tied together.

Addressing the question

Looking at the multiple layers, many players and the associated mix of measurement and learning needs, we understood that we didn’t have a pre-existing approach to address the inherent complexity. Rather, we usually relied on a standard monitoring for accountability reporting to funders and ad-hoc feedback from partners and network members on success. This feedback often came in the form of anecdotes, stories or workshop feedback form responses.

We decided to develop a bespoke, fit-for-purpose measurement and learning framework that is both robust and versatile.

In partnership with Collaborative Impact and with funding support from the Hewlett Foundation, R4D embarked on piloting such a framework as a collaborative co-creation process. Together with two pilot CLNs, we spent several months designing, stress-testing and refining the framework to meet the demands of the complexity described above while maintaining enough ease-of-use for broad application. We wanted to ensure the framework can be applied to networks of all sizes, geographic scopes, sectors and levels of maturity.

For those who fund, design, lead or work through CLNs to strengthen local systems and are wondering how best to measure progress and impact, this framework is for you. We are thrilled to release this Collaborative Learning Network Measurement & Learning Framework as a flexible tool for organizations to adapt and apply as useful to their given network contexts. What’s more, we hope sharing this framework will spur further conversation around measurement challenges and opportunities as more organizations shift to play an intermediary facilitative role seeking to amplify the transformative power of locally led coalitions and networks.

In upcoming blogs in this series, R4D and Collaborative Impact will delve into the content of the framework and the process we took to co-create and stress-test it with the Linked Immunization Action Network and the Strategic Purchasing Africa Resource Center (SPARC).