IngeniousPeoplesKnowledge

Adrian Gnägi from the Swiss Development Cooperation (SDC) has written an interesting blog article on “What is wrong with Managing for Development Results?”, reflecting an essay of Andrew Natsios, former head of USAID.
The article deals with “«Obsessive Measurement Disorder» (OMD), … an intellectual dysfunction rooted in the notion that counting everything in government programs will produce better policy choices and improved management”. We have gone over the top with the well-intended desire to ensure results and to be accountable. But we have set the fox to keep the geese: all the attempts to monitor and evaluate are in themselves becoming one of the biggest impediments for efficiency and effectiveness… Our current M&E practice has in fact become one of the big problems rather than a solution to anything…

Read it on the SDC Blog, it’s worth it!

Already 11 years ago, Meg Wheatley wrote a very similar article on the obsession of measurement, and it’s more topical than ever:
http://www.margaretwheatley.com/articles/whymeasure.html

There is a second article of her speaking to the same issue, a nice illustration from the educational system in the US:
http://www.margaretwheatley.com/articles/largescalechange.html

I have come across this interesting article in “The New Yorker” on what science – and thus we – can really know, how we perceive and deal with reality:

The Decline Effect and the Scientific Method: newyorker.com.

Some of it may not really be surprising if you look at it from a complexity perspective – I really like the study on the behaviour of the mice: you can try to control all variables of an experimental set-up and keep them totally constant over several replications. Your experiment should then reproduce the same outcomes – but it doesn’t! Dramatically! You can’t predict the behaviour or reaction of a living system, no matter how well you plan!

If we try to consider the consequences of what we find in this article and consider that these findings are mainly from “hard science” (like medicine), than how difficult is it to come to a valid finding in a “soft” field like evaluating a development project? Is there anything at all that we can measure and prove? Or is Monitoring and Evaluation largely shadow-boxing, eyewashing, pretending we are able to control something we can’t? How far from “reality” are we with all our studies, evaluations, …? And what value remains then?

“We would like to dethrone measurement from its godly position, to reveal the false god it has been. We want instead to offer measurement a new job – that of helpful servant.” (M. Wheatley and M. Kellner-Rogers)

Measuring for accountability examines “what”, the products, the “things” and assumes that knowledge is a package that can be counted. Measuring for learning makes sense of the “how”, the process rather than the product, and the relations between things. Measuring is not a bad thing in and of itself, the problem emerges when measurement is used for the wrong things. Measuring is good at accounting for what we have done against what we planned and thus should be used for gauging input and outputs. However, one cannot attribute impact to input because of the complexity and therefore measuring impact is almost impossible. Measuring for accountability does not appreciate what is of greatest value and in a knowledge management environment this is likely to be a catalytic conversation or a new idea for innovation.

Within the scientific domain, “measurement reduces and standardises. In order to make sense of complex systems and processes, measurement first uses models and frameworks to reduce them to manageable segments” (Taylor, J. and Soal, S. (2003) Measurement in development practice. CRDA: South Africa, p.4). This is contrary to I-P-K’s approach which encourages complexity by inviting it into one room.

Measurement for learning can be used to improve development practice by creating a picture of what we want to achieve and once the activity is complete by taking time to reflect on why it did not go as planned or why it did not turn out as intended. This reflection and learning should be incorporated into future planning to improve practice. If this does not happen, there is no behaviour change and the project continues as it has been. This cannot for work for any development initiative. Measuring for learning can help ensure that the changes you make to your practice will make you a more effective organisation.

(Text by Margaret Jack, I-P-K)