A call for evidence on strengthening mixed health systems
Findings from our evidence review
[Editor’s note: This is the third post in a series of blogs sharing experiences from R4D’s work on mixed health systems. The authors of this series hope to encourage debate and the sharing of ideas on the topic of mixed health systems and public-private sector engagement for universal health coverage (UHC). This blog, in particular, explores the evidence around mixed health systems. The series also highlights some principles for improving mixed health systems and how governments can support private providers’ integration into government plans. Catch up on the series, and read the first and second posts, “3 principles for improving mixed health systems” and “Pursing Public-Private Engagement Options in mixed health systems”]
When you look at the field of mixed health systems, much of the evidence that is out there is built from practice and experience. Recent blog posts from R4D on this topic focus on what we have learned about improving mixed health systems and the roles governments can play in these engagements, mostly stemming from our experience as a technical partner, process facilitator, and intermediary working with public and private change agents on health systems strengthening.
In this post, we’ll provide a primer on the evidence base for mixed health systems, and share an evidence review by the numbers — and what they might mean for researchers and practitioners, including:
- The percentage of articles that do not mention universal health coverage
- The percentage of articles that study a specific public-private engagement
- The percentage of studies that look into factors that may help or hinder public-private engagements
But what does the research say?
To answer this question, it may make sense to start with what we mean by “mixed health systems” and why understanding the evidence may require us to look beyond just this term.
First off, mixed health systems (MHS) as a term is still used relatively infrequently. It refers to a health system in which a mix of public and private providers deliver health related goods and services. In the past, the global health field has talked more about “public-private partnerships,” a more limited contractual concept. Instead, evolving mixed health systems involves several elements and deliberative processes, including: building, nurturing and maintaining trusted public-private engagements and supporting the legal, political, economic and organizational enabling environment.
As the field moves more toward the concept of mixed health systems, a full review of the evidence would not be complete without considering what the research on public-private partnerships say on this topic.
Even with the myriad of theoretical and empirical research that has looked at public-private partnerships, learning in this space has often been rooted more in the experience of practitioners than what is coming out of journals. And there are at least two good reasons for that. First, context matters — and the best approaches and responses to strengthening mixed health systems are likely to be ones that respond to specific factors related to the place, actors and problems where they are being implemented. Second, the very thing that makes systems strengthening work appealing — understanding complex adaptive systems and the “behavior, of diverse, interconnected agents and processes” in a holistic way — also makes it all the more challenging to measure and research.
With all of this said, we believe there are still important lessons to be gleaned from existing research on mixed health systems — and gaps in the evidence that are worth further investment. R4D’s Strengthening Mixed Health Systems (SMHS) project, supported by Merck for Mothers, is seeking to help fill some of these gaps by testing and researching approaches to strengthen MHS, specifically around public-private engagement, to improve maternal health and support universal health coverage (UHC). We kicked off this effort with an evidence review, identifying 237 studies that were published in journals and that have sought to reveal important evidence about strengthening mixed health systems and ultimately conducting a deep dive review of 76 papers.
Though the complete results of this evidence review are forthcoming, we’ll unpack some numbers below.
76: the percentage of articles that do not mention universal health coverage
Our search terms included several combinations of “mixed health systems,” “public private” and “maternal health,” so the inclusion of articles that explored UHC was not guaranteed. Even so, the importance of the UHC in the global health field at this time suggested there would be a greater integration of this agenda into the literature, as an outcome or an intervention focus related to mixed health systems. This is especially true given that we restricted our search on articles from the past 10 years, a timeframe that we would have anticipated would include more articles that focused on, or at least mentioned, UHC. Instead, we found the vast majority of articles made no mention of UHC — and the percent that elevated UHC to the level that it was mentioned in the title or abstract was significantly smaller (11).
Is this surprising?
Yes and no.
What may underlie this disconnect?
What might help stimulate more literature in the area of mixed health systems and UHC?
We wonder if a change in what countries are demanding or looking for will help support this research agenda. As countries recognize the need for leveraging the private sector for UHC, they begin to call for not only experiences, but evidence (and ways to translate and contextualize that evidence), for what works.
Additionally, it will be necessary to better integrate research methods, such as implementation research and adaptive learning approaches, which can be produced more rapidly to minimize lag time between action and research, and help policymakers and practitioners understand factors that influence implementation. Implementation research and process evaluation can help policymakers better understand how a given intervention or reform is or is not supporting change, while adaptive learning leverages available data and feedback to adapt a specific approach in real time. Finally, donors and funders can also help stimulate research agendas by coupling support for technical assistance with more rigorous forms of evidence generation on UHC.
29: the percentage of articles that study a specific public-private engagement
Interestingly, most of the literature we reviewed focused on frameworks for public-private engagement or country-wide policies and approaches to working with the private sector rather than specific demonstrations of partnerships or engagements. This means the literature focused either on theoretical concepts or policies whose implementation had yet to be studied, rather than hard evidence. We noted in the introduction to this post that the critical role of context, which presents challenges in terms of conducting evaluations that could provide generalizable lessons for the effectiveness and design of support to strengthening mixed health systems and specifically public-private sector engagements. Taken in tandem with the fact that impact evaluations are difficult to design for health system change and these types of engagements, this could explain why there is less literature that speaks specifically to demonstrations and health system impact of public-private engagements.
Even with these caveats, this feels like a missed opportunity to investigate outcomes related to specific country demonstrations of work to strengthen mixed health systems. Reviewing a pool of analyses on the effectiveness of public-private engagements may never reveal a silver bullet — but may help to uncover trends related to specific context factors or insights into engagement designs and processes that could be valuable to practitioners looking for guidance of what not to do.
Is this surprising?
This, too, is somewhat unsurprising.
What may underlie this disconnect?
As we mentioned above and expand on here, measuring health system change is hard and is still a relatively new field. Measuring complex system change depends on the types of research questions asked as well as on the system context and dynamics, so there is no single best evaluation design to draw on. While some guidance exists, health researchers and practitioners must take flexible and iterative approaches like outcome mapping/harvesting — using systems thinking theories, methods, and tools and designing indicators to measure system change that are most applicable to the context. Additionally, it is also difficult to link health system change to improvements in health outcomes. This is partly due to the longer-term nature of movement in these types of indicators and partly because of the difficulty in proving causality with so many actors, processes, and pathways in the mix.
61: the percentage of studies that look into factors that may help or hinder public-private engagements
Despite there being a relatively small number of evaluations or articles that look at specific public-private engagements, there is much more knowledge (empirical and theoretical and experiential/anecdotal) about what may be linked to better — or worse — public-private engagement performance. This is one place where policymakers and practitioners could benefit from the research — and where the evidence could continue to flourish. Without a simple cookie-cutter approach to public-private engagement, understanding factors related to engagement design, partners, context and technical content that are associated with stronger or weaker performing engagements can provide practical and actionable recommendations for stakeholders working on evolving mixed health systems day in and day out.
An interesting finding from the evidence review is that some of the factors that might matter the most are not all technical but rather softer in nature. While factors related to determining enabling environment improvements in financing and the nuts and bolts of contracting are certainly important and are prevalent in the literature, a big piece of the puzzle seems to be related to the characteristics of and dynamics between the partners themselves. Factors such as the willingness of each party to dialogue and share information, can influence the success of engagements or even determine whether they happen at all. And, within an ongoing engagement, factors such as partner motivations, trust, shared values, communication, and even attitudes may have a significant impact on success. These types of factors may be challenging to assess and measure but could be critical to understanding how public-private engagements can work better to strengthen mixed health systems. And understanding how critical these factors are in determining the success of engagements can help those seeking to support mixed health systems design approaches that seek to foster factors that help — and mitigate factors that hurt.
What next?
These numbers only scratch the surface of the evidence review — and we are continuing to design work with partners in Kenya and India to test new approaches to strengthening mixed health systems for better maternal health and in support of UHC.
As we continue to make sense of where the evidence already is, it is worth noting what we cannot say from our review. The 237 studies that we reviewed for this work focus on the academic literature, and we recognize that much of the knowledge about strengthening mixed health systems to achieve health outcomes may lie in the grey literature. This is a component of the evidence that we seek to come back to, but for now, it is a gap in this work that we need to acknowledge.
Even so, the high-level numbers like those discussed here and the deeper analysis that we are currently writing up represent a big step forward to consolidate what the literature already says about strengthening mixed health systems. As part of our project work, the SMHS team is supporting demonstrations of strengthening mixed health systems in Kenya and India. Using a case study methodology, the project will conduct process evaluations paired with qualitative analysis to answer learning questions for these two countries. This combined with crosscutting analysis of secondary cases identified in part from the evidence review will continue to help fill the gaps in the evidence. One evidence gap, in particular, that we plan to help fill builds on the final statistic above — studying and better understanding what factors are associated with successful and unsuccessful public-private sector engagements. This research includes understanding factors that may make some health systems more “ready” or conducive to stewarding and integrating private health care services.
It won’t be a silver bullet — or even a blueprint. But it may reveal trends for what works better — or worse. And, for some of the hard-to-tackle challenges in health systems strengthening, these trends could help seasoned practitioners lean on models and experiences that show more promise and avoid the pitfalls that many have faced before.
This program was funded by, developed and is being implemented in collaboration with Merck for Mothers, Merck’s $500 million initiative to help create a world where no woman has to die giving life. Merck for Mothers is known as MSD for Mothers outside the United States and Canada.
