I think I’m finally caught up and recovered from my brief North American tour last week. While it was fun to present at two conferences plus a customer talk in four days, I missed out on a the second day of both conferences, as well as the opportunity to meet nearly as many people as I would have liked.
I thought I’d write a bit about something I heard more than once last week. I summed up my reaction in a tweet.
I forget how often some companies blame testers for escaped bugs. It’s not their fault.
My flippant follow up to the “bugs are tester’s fault” statement is typically “How can it be test’s fault? We weren’t the ones who put the bugs there in the first place”, but there’s some truth to the remark. Think of it this way – If testers are responsible for letting bugs “slip” through to customers, you have an engineering system where programmers are paid to insert bugs into the software, and where testers are penalized for not finding all of the needles in the haystack.
That doesn’t seem right. Delivering quality software isn’t a game where programmers insert bugs for testers to chase after – everyone on the team has to have some skin in the game on delivering quality (and value) to the customer
There’s also been a lot of buzz recently about the “cost of testing”, where testers (or teams of testers) attempt to justify the investment in testing. I have to admit that this bugs me – I don’t believe in the cost of testing – I believe in the cost of quality, and that the cost (and effort) is shared among the entire software team.
In hwtsam, I wrote:
Many years ago when I would ask the question, “who owns quality,” the answer would nearly always be “The test team owns quality.” Today, when I ask this question, the answer is customarily “Everyone owns quality.” While this may be a better answer to some, W. Mark Manduke of SEI has written: “When quality is declared to be everyone’s responsibility, no one is truly designated to be responsible for it, and quality issues fade into the chaos of the crisis du jour.” He concluded that “…when management truly commits to a quality culture, everyone will, indeed, be responsible for quality.”(STQE Magazine. Nov/Dec 2003 (Vol. 5, Issue 6))
Culture is hard to change, but it’s imperative for making quality software. If your programmers “throw code over the wall” and are surprised when the test team doesn’t find bugs, you should rethink your work environment (or resolve yourself to the fact that your job is to clean up the programmers mess and that quality is nowhere near your control).
As a comment to the “game” you mention in the post: In his (excellent) book “Perfect software and other illusions about testing”, Jerry Weinberg writes about rationalizing the unreasonable, where people try to justify irrational actions in an attempt to make them appear reasonable. A quote from the book:
“for example, a developer is rationalizing if he claims, “I really left mistakes in the program so we’d know if the testers were doing a good job.””
Fortunately, I haven’t come across that kind of mentality in any of the projects I’ve participated in but, undoubtedly, it does exist.
Besides, some bugs slipping through testing doesn’t mean the testers were doing a bad job – they may just have been too busy dealing with the bugs they DID find.
About the cost of testing, I agree with the point you make about it being better to concentrate on the cost of quality instead and my chapter in an upcoming book about reducing the cost of testing is really talking about just that.
It’s good to know the costs related to various activities within a project but it’s not good to place unnecessary emphasis on a single activity as it may lead to losing the big picture and faulty conclusions (it’s better to not use metrics at all than to use bad metrics).
Overall, good post!
…and a great comment – thanks for taking the time to post.
Hi Alan,
A couple of things.
You mention that the concept that everyone owns quality is new. I think it is relatively old. Juran’s (1951) and Feigenbaum’s (1961) quality handbooks were based on this concept. Feigenbaum’s book works out quality control in painstaking and fascinating detail.
“Quality is nowhere near your control”. Very, very true. I think we, testers, often make the mistake of taking responsibility for the quality of the product, the process and *heck why not* the company.
Sometimes it seems like we test while the quality control organisation is missing or not functioning properly and we take on responsibilities that are not ours.
“You should rethink your work environment”. This is essential. One of the first questions you might ask is: ‘how mature is the quality control organisation?’. If there is hardly any quality control then you know you’re probably facing an uphill battle.
I didn’t mean to imply that team wide quality ownership is new. The context of the quote from the book is that in my experience, I’ve seen a shift – not that the concept has changed.
Thanks for the comment – very well stated.
I have here a lot of definitions for quality
http://www.testalways.com/2010/07/28/another-post-about-what-is-quality/
I think though in general quality and testing are areas that intersect but are not the same.
Quality in my view is related to the fact that it also looks nice, it is designed nice, not only works. The working part should be assured by good testing.
Your post is on quality and its ownership. But in the post you seem to be focussing only one missed bugs. That is not all there to quality !!!
If quality is a multidimensional time variant attribute that is valid between a designated user (or user group) and subject (product/service) – bugs (relationship between user and product/service) form one of those several dimensions.
Quality is abstract – the question of ownership of this abstract thing is rather obscure.
Shrini
I thought it was obvious that in this context, I was *focusing on the activity of blaming bugs on testers. A better title possibly could have been “who owns bugs”, but that too is ambiguous without context.
I could have quoted a near infinite number of published definitions of quality, but that wasn’t what the post was actually about.