What do you do when your manager | project manager | dev manager | other leader asks you for some bit of data that you know is useless? Say that they want something like code coverage progress and test pass rates, but you know that those metrics aren’t truly useful.
Side Note: waitaminute – some of you may be thinking those aren’t useless at all, The way I see it, coverage is only interesting in how you use it to discover potential holes in testing. Test pass rates aren’t interesting to me at all. I can have a 99.99% pass rate, but if the one failing test “accidentally” erases the customers hard drive, I’m far more worried about that single bug than the pass rate. Even if the pass rate is 95%, what I’m most worried about is what types of bugs are in that failing 5%
But people will still ask for those things, because they think they are useful. So, what do you do?
One approach would be to tell them they’re stupid and that you’re not going to give them useless data. Give that a shot and let me know how it goes.
On the other hand, you could just give them what they want – give them test pass rates and coverage information, and go back to work. that’s easy. (note – looking at these two examples is an application (or variation) of my “find the middle” technique).
What you want to do, is give them what they want, but most importantly, give them what they need. The coverage report gets a lot more interesting – and applicable if you include, for example, a list of tests you’ve added as a result of analyzing code coverage. It’s ok to share the test pass rate too, but include breakdown of the bugs causing the failures. You’ve given them what they asked for (pass rates), but you’ve steered the conversation towards risk (what they need). For bonus points, you can look for ways to intermingle the two requests (e.g. are you finding new bugs when adding tests discovered in coverage analysis).
One of the keys for success here is understand (or at least guess) why they want the data in the first place. They probably want some answer – project status, test progress…something. Figure it out, and give them whatever data you think they need to get their question answered. If you’re off track, reload and try again. You will find the right formula. The easy path is to give people what they want, but if you’re serious about improvement, start giving people what they need.
Good advice, Alan!
It’s pretty hard not to flip the bozo bit on the next VP who decides to ask for metrics without context (“because it’s a best practice, that’s why!”).
And rather than arguing, it’s easier to just give them what they are asking for, and let them feel a false sense of security.
Far better to try to determine (by asking?) what they are really after, and give them what they really need.
I’d go further than Joe-
It’s not just easier to give them what they ask for, it’s required to maintain a positive professional relationship (which in turn might be required for product success) (unless, the asker is junior to you).
Then, if you’re doing PQ (precision questioning), shut up until you switch PQ roles. Only then offer the elaboration.
I agree with you Alan, the questioner probably needs the more detailed and more pertinent details, and given those details, the simplistic metric of “test pass rate” only matters in the context of other pass rates e.g. yesterday’s pass rates.