Size doesn’t matter

What is measured, matters or is it measure what matters?

I never remember.

Whichever the case, there is a general fascination with finding the silver bullet (or other such adage) for proving impact. Success, it seems matters most when it can be drilled down to singular items devoid of the nuance and complexity of what made them interesting to start with.

In the world of monitoring, evaluation and learning (and we are pretty fascinating people) there doesn’t feel like enough time or energy is spent talking about metrics and indicators. Or rather, we spend so much time talking about metrics and indicators that we don’t talk about what metrics and indicators are, where they come from, and how they can be most useful in supporting learning, accountability and improvement needs.

The history of measurement and metrics is long and convoluted but its core narrative is one of finding a common language to talk across boundaries about things that defy language. If we all agree, for example that a foot is a foot is a foot — then ten feet will make sense no matter the country, no matter the measurer, no matter the measured. Skip forward a few thousand years (apologies for the butchering of a fascinating history) and we find organisations — funders, evaluators, academics, partnerships — increasingly interested in a common language for identifying progress towards outcomes and impact; and the things that get measured.

The thing about deciding on what gets measured and how to measure it, is that we all have our way of rorting the system to make ourselves feel better, or to construct a positive narrative using available data. When I’m talking about the successes of my latest diet, I will have gained a stone but lost six kilograms. The same goes for organisations wanting to report on progress towards outcomes. In their case, organisations find themselves talking about click-through rates and impressions and lives saved. Metrics are chosen for their numerical qualities, their infallibility and because they’ll be impressive.

We’ve become obsessed not with what we measure, but with dare I say, size.

Deciding what to measure is difficult, no matter the program, concept or intervention. It’s difficult because measurement is about taking a piece of something and comparing it to a standard. It can feel like a good idea to choose the number of times an advertisement was viewed because it’s easy to capture and yields a big number, can be compared to a standard measure (others are doing it), but ultimately it doesn’t mean much if you’re not really interested in how many people saw your advertisement.

In our work, there are two usual places we find people get stuck and I thought I’d focus on them in this series:

1. How to arrive at shared and common measures

2. How to use the information you collect for learning, accountability and improvement

Next week, check back for more and please send through thoughts and feedback on any other topics to include on measurement.