What do you do when your manager | project manager | dev manager | other leader asks you for some bit of data that you know is useless? Say that they want something like code coverage progress and test pass rates, but you know that those metrics aren’t truly useful.
Side Note: waitaminute – some of you may be thinking those aren’t useless at all, The way I see it, coverage is only interesting in how you use it to discover potential holes in testing. Test pass rates aren’t interesting to me at all. I can have a 99.99% pass rate, but if the one failing test “accidentally” erases the customers hard drive, I’m far more worried about that single bug than the pass rate. Even if the pass rate is 95%, what I’m most worried about is what types of bugs are in that failing 5%
But people will still ask for those things, because they think they are useful. So, what do you do?
One approach would be to tell them they’re stupid and that you’re not going to give them useless data. Give that a shot and let me know how it goes.
On the other hand, you could just give them what they want – give them test pass rates and coverage information, and go back to work. that’s easy. (note – looking at these two examples is an application (or variation) of my “find the middle” technique).
What you want to do, is give them what they want, but most importantly, give them what they need. The coverage report gets a lot more interesting – and applicable if you include, for example, a list of tests you’ve added as a result of analyzing code coverage. It’s ok to share the test pass rate too, but include breakdown of the bugs causing the failures. You’ve given them what they asked for (pass rates), but you’ve steered the conversation towards risk (what they need). For bonus points, you can look for ways to intermingle the two requests (e.g. are you finding new bugs when adding tests discovered in coverage analysis).
One of the keys for success here is understand (or at least guess) why they want the data in the first place. They probably want some answer – project status, test progress…something. Figure it out, and give them whatever data you think they need to get their question answered. If you’re off track, reload and try again. You will find the right formula. The easy path is to give people what they want, but if you’re serious about improvement, start giving people what they need.