Measurement in times of uncertainty

Sitting in my office, in the middle of a global health crisis — where everyone around me has had at least one moment where they have thought, “I don’t know if I can do this” — I had a small and surprising positive thought:

Yeah, but humans are really good at doing hard things.

In the moment, it doesn’t feel that way — because the hard things are, well, hard. Taking care of your kids at home while you also try to achieve the same things you could at work four weeks ago is hard. Telling your loved ones that you can’t see them — for weeks or months or who knows how long — because of some invisible but real and deadly threat is hard. And for all of those people who are still interacting with people on a daily basis — to provide essential and lifesaving services — keeping the people around you, even the ones who are total strangers, safe and healthy is hard.

But as lots of smart leaders have told us — we as people do things not because they are easy, but because they are hard.

So, in times like these, are there things we can do to make it easier to do hard things?

We want to propose that one surprising but important answer is measurement.

Umm…explain please. 

One of the scariest things we face in our lives is feeling lost — geographically, emotionally, metaphorically. You name it. We make decisions to do things that are difficult, but we have an innate desire to get some feedback on how we are doing, whether we are going in the right or the wrong direction, and whether we need to change course.

In the world of a global pandemic, we have good blunt instruments of what we need to do (#StayHomeSaveLives). But blunt instruments are not always good at giving clear guidance on the everyday minutia. Can I go to the grocery store? Can I walk with friends if we are six feet apart? Do I need to wipe down my mail with bleach before opening it?  Or let it sit for 24 hours?  One week? Working in unprecedented environments, one of the ways we know the answers to these questions is to collect data and to track what changes when we do the things we do. The data may suggest things are getting better or much better — or worse. Measuring progress matters.

The same is true for the other hard things people do: reducing corruption, promoting behavior and narrative changes, connecting civil society to achieve collective impact, and pushing for changes in policy and practice from elected officials. None of these ultimate outcomes comes at the end of a simple or linear path, and the routes we try to take to achieve these changes can leave us feeling lost without some guidance regarding the progress we are (or are not) making.

Enter…measurement.

Making measurement easier

Fifteen months ago, R4D began working with colleagues at Open Society Foundations and lots of incredible organizations working on fiscal governance (including the International Budget Partnership, the Open Government Partnership, the Transparency and Accountability Initiative, the Natural Resource Governance Institute, and the Tax Justice Network Africa) to figure out how we might be able to tackle the issue of making measurement work better for the things that are hard to measure. These hard-to-measure fiscal governance issues include budget transparency, trade policies, natural resource and extractive industry governance, and anti-corruption efforts.

One of the first things we learned from conversations with people doing the hard stuff every day is that the first step of identifying indicators to use to measure progress can be one of the biggest roadblocks. While one part of our work has been to develop new indicators where they are missing, we also suspected that there were some good measures and indicators out there already — spread across people’s experience and internal program documents and websites.

We were right; lots of outcomes and outputs that are hard to measure are not uncharted territory. So, we developed a dataset of what we found when we tried to scope out what already exists — and now we are sharing it with (the global) you.

The database includes a range of different types of tools — everything from indicators to indices to measurement toolkits. And it delves into a lot of outcomes that people working on fiscal governance are trying to move the needle on, including health, education and nutrition, among others. So, we have made the database filterable by the types of outcomes that organizations seek to achieve. For example, you can use the database to explore things like awareness raising, influence and power, political will, and contribution to the bigger changes that may require lots of people pushing on the same big boulders.

We hope that you find the database as useful and informative as we have!  Each entry is categorized by outcome area and data collection type, and has link so you can go to the source before you decide to apply or adapt an existing tool or indicator to your own work. We hope that this will make it easier to understand what is out there already — and to make it easier to measure where you are and what that means for progress toward those bigger goals.

And, as all of us are working on difficult issues in an environment that feels uncertain and unclear right now, please reach out if you have questions or ideas for additional resources to build into this database.  We want to help all of us who are not scared off by doing the hard things to do — and measure — those things together.

Photo © Kendra Helmer/USAID