Years ago I worked for a company which shall remain nameless. They collected data on everything:
We had to swipe cards to get in the building in the morning and again at lunch. Our computer login and logout times were checked. We had to remain 95% billable (no kidding) while the maintenance workers were tracked based on how long a service ticket was open and how long the task actually took.
For quite a while, they weren't doing anything with this data… and then someone started talking about “continuous improvement”.
The underlying idea of “continuous improvement” – regularly analyze collected data to see what can be improved – makes quite a bit of sense. After all, the concept is no different than optimization issues or bottleneck analysis we do.
Except for one major difference… we're talking about people and physical processes involving people.
When a database query takes longer than normal, there's normally a specific reason why. It may be difficult to find the reason, but it could be something as innocuous as a poorly chosen database index or something subtle and complex as a poorly designed loop or business logic. Regardless, you can often find a way to reproduce the problem – this is the underlying concept of Unit Testing – and determine when the problem has been solved.
When it's a physical process based on the choices and abilities of humans, behavior is not going to be consistent. Two people similarly trained, similarly equipped, and similarly skilled will perform the same task differently from day to day or even hour to hour. It may not always be significantly different but it will different each and every time.
So anyway, my point with all of this is what my employer attempted to do with the data…
The order came down from on high that there were certain “key tasks” that would be tracked and employee evaluations would be based on those. Fundamentally, this could make some sense: determine which things are most important to the job and keep an eye on those closely. But this company took it a step further…
They announced that anyone taking longer than average would be “written up”. And as per company policy, after three writeups, you're fired.
If you don't see the flaw in that one, stop and think about that…
The way an “average” works is that roughly half of the people will be on each side. So half of the people get written up and eventually fired. But due to the slower people being gone, the average shifts lower…
Lather, rinse, repeat.
Collecting data and understanding your processes is a good thing. Knowing what takes how long is a good thing. Knowing who is good (or bad) at certain tasks is a good thing. Collecting data over time to show how your team is changing (improving or not!) is a good thing. Knowing where your team might need more help or training is good.
Making decisions based on any one piece of that data is a bad thing.