Making measurement useful
A few years ago, I decided that I’d learn French. I was living in Canada at the time, and had convinced myself that French would have some sort of practical value.
For four years, I went along to Alliance Francaise twice a week. I dutifully conjugated verbs, had inane conversations about if I had gone/was going/could go to the cinema, and listened to Edith Piaf on repeat about her life in pink.
Along the way there were mini quizzes, assignments, presentations to the class, feedback from at least 10 different teachers, an immersion trip to the south of France, and even a 3 hour French language diploma that I sat (and passed, merci). There was also the staggering amount of money I spent on it all — which, when added up, probably equated to a small Renault.
There’s lots about my experience that could be ‘measured’: the number of classes I went to; how much money I paid; how well I did on each test; if I passed the exam; if I enjoyed it; or if I could navigate my way across Paris, get the best croissant in town and ask for a size 10 in those black leather lace ups, no not those ones, those ones.
It’s not too dissimilar for people and organisations in the business of social change: there are all sorts of indicators that could be used to measure what we do and the difference it makes. In making choices about what indicators to gather, it’s often useful to consider how we’ll use the information. For most, there are three broad (and interrelated) uses:
Learning: will this information help me understand what’s working, what’s not working, and why? I could have added up all the money I spent on my French classes, but it wouldn’t have helped me determine where I needed more work (which was everywhere, by the way).
Accountability: will this information demonstrate to myself and others that we did what we said we were going to; that our initiative has made the difference we intended it to make; that the investment in this work was good value? My Diploma was evidence I could communicate at a certain level — it didn’t do much to help me b my shoes in Paris.
Improvement: does this information help me in changing what I’m doing? What will I do differently based on this new information? Regular feedback and course corrections, delivered in ways and at times that enable action, seem to be best at fostering improvement. “It’s pronounced poisson not poison” — helpful little nudges.
Understanding who is interested in what information is therefore an important step in crafting a measurement agenda: is it a current or potential funder of our work, a program designer, a participant, a senior manager or perhaps the public? And what is going to be done with the information — is it about learning about our program, meeting accountability obligations, or about making choices that improve our work?
Often all three need to be addressed — and will require a variety of carefully selected and measured indicators.
In the end, I didn’t mind how much money I spent on my French lessons, or how many classes I turned up to. What mattered was that I was able to get across Paris, order that croissant, and buy a really nice pair of blue canvass loafers in a size 6.
A total success, I’d say.
