Peering into the white box

In a conversation last week at STAR, someone was talking about how far testers should debug an issue. Personally, I don’t like it when testers only report the surface of the bug, and I think good testers always dig a little deeper to see what other more interesting issues may be lurking a layer or two below. It can certainly be a bit of a time sink, however, if testers investigate every single bug to the line of code causing the problem. The interesting comment I heard in this particular conversation was the statement that “testers should never look at the code”. The points I heard in defense were all in line with the cons of white box testing – e.g. “testers need to focus on the customer experience, and looking at code ‘taints’ this focus”, or “it moves tester focus to non-essential testing”, etc. I, for the most part, agree, but I think there’s a hidden (but real) value in applying the tester mind to looking at product source code.

In my experience, testers are excellent code reviewers, and think testers should be involved in code reviews much more often than they are today. If you think about it for a moment, it makes a lot of sense. It seems that every week, I read another article or blog post explaining why developers aren’t good testers (“they only focus on the happy path”, “they don’t think of negative conditions”, etc.). There are also numerous articles and experience reports about the value of code reviews for finding bugs earlier.

Now, if you put 2 and 2 together, testers reviewing code must be a great way to find bugs early. When developers review code (and I’ve watched them), they look for the basic general programming errors, but mostly verify that the code does what it’s supposed to do. Testers, on the other hand, question the code – they ask “under what circumstance would this line of code fail?”, or what-if questions like “what if the input values were unexpectedly large?”. Testers also think about how they would test the code – simply asking “how would I test this” often unveils more problems.

I imagine that for some testers, finding bugs in code isn’t as fun as spotting real product failures, but it sure seems to me that some testers reviewing some portion of the product code is a pretty darn good idea.

I’ll tell you how to get started next time.