Title : KPIs considered harmful
Link : KPIs considered harmful
KPIs considered harmful
Anyway, KPIs have always bothered the hell out of me. In fact, the title of this post was going to be "KPIs are a load of bollocks," but I chickened out. My basic thesis is this: show me a performance metric, and I'll show you a performance metric that is being gamed. The basic problem with any KPI is that it is a simple, abstracted measurement of a more complex underlying phenomenon. For example, you might measure the productivity of a call center worker by the number of calls they complete per hour. This seems reasonable; one would expect the productivity of the worker to correlate with this metric. The problems begin, however, when the KPI is used as a way of rewarding and motivating workers.
I always remember a story told to me by an Associate Professor at the Computer Science Department of RMIT University, where I was doing my PhD. This particular person's research area was Genetic Programming (GP), which endeavours to 'evolve' programs to perform a particular task (usually in LISP) by using analogues of the processes of survival of the fittest, mutation, and sexual reproduction. Anyway, one of things he was involved with was the Robocup software league. Essentially, he and his team were trying to evolve programs that could successfully play a game of simulated soccer.
Now, in order to get the 'survival of the fittest' part of GP working, one needs a 'fitness function' that determines how successful a particular program is at performing the task towards which the evolution is targeted. In other words, a KPI. In the case of Robocup, this Associate Professor explained to me that at first they had tried to use the number of goals scored by each player as the fitness function. This makes sense, as better players would tend to score more goals. The problem was that the fitness function wasn't granular enough; the randomly-generated players were so far away from being able to score a goal that the evolution ended up being essentially undirected. So, the clever researchers came up with a more granular fitness function: fitness of a player will be a product of the time spent in possession of the ball and the inverse of the distance of the ball from the goal. Again, this seems to make sense: good attacking players get the ball near the goal. This provides a smoother runway to evolve players that can attack a bit, but not well enough to get an actual goal, on the way to getting to goal-scoring behaviour.
You may be able to guess what happened next. After the evolution was complete, the researchers ran a game of simulated soccer with the most highly fit specimens evolved during their run. This is what they observed: a player would take the ball, move it towards the goal, and just before actually getting it in they would stop and hover around the ball, stopping other players from getting to it. In other words, they optimised very successfully for the KPI, but not for the actual goal. In fact, the players were disincentivised from scoring an actual goal, as this would take the ball from their possession and move it away from the goal. Anyway, the researchers tried with several different fitness functions, and each time a different undesired and unexpected behaviour emerged to take advantage of the metric. Throughout all this, the players did all sorts of interesting and clever things. But they never kicked a goal.
It's not very hard to see that our call center workers above, if their pay or career prospects depended on a calls-handled KPI, might start to act funny. For example, they might begin to terminate many calls before an issue is resolved, in order to be able to receive more calls. This is not a made-up example. When you rely on KPIs too heavily, weird (and usually bad) things happen. In particular, if incentive structures are built around particular KPIs, you are guaranteeing that the organisation will maximise those KPIs. If the KPIs are not very closely tied to what you actually want to achieve, it is most likely that the KPIs will be maximised at the expense of that ultimate goal.
In software development, most obvious KPIs end up encouraging poor behaviour: lines of code written, number of changes submitted, number of bugs resolved, and so on all end up forcing people away from optimal development. LoC count for example, can encourage unnecessary verbosity and copy-paste coding. It is essential to have management with actual insight into the contribution of each person rather than relying on cheap and gameable metrics.
Bottom line is this: management by KPI is mismanagement. KPIs can provide a valuable snapshot of performance if used with care, but in my opinion tying them into an incentive scheme is simply asking for trouble.
Photo by Vitor Castillo / CC 2.0
Such is the article KPIs considered harmful
That's an article KPIs considered harmful this time, hopefully can benefit for you all. okay, see you in other article posting.
You are now reading the article KPIs considered harmful with the link address https://wcest.blogspot.com/2013/04/kpis-considered-harmful.html