I’ve been having one of those experiences where a lot of the random bits of flotsam on my radar are sort of converging. But I don’t know if it’s really convergence, or if I’m forcing it. I hope it’s the former, but time will tell.
In my last post, I reflected on a wonderful article from Phillip Armour. His breakdown of “tests that tell us what we know” (0OI tests), and tests that tell us what we don’t know we don’t know (2OI tests) aligns perfectly with something that’s been gnawing at me for a long time.
You see, I’m also bothered (and have been for years) to some extent by one of the more popular testing debates – the scripted vs. exploratory tests…and to top it off, I’m completely confused by the variety of reactions to the “test is dead” statement one of my old friends has been spouting about recently (I warned you about random thoughts floating in my head…)
But it’s kind of coming together for me – some of it as I write these paragraphs.
A part of the problem is that what I call testing is probably different than what you call testing. That’s also true if I turn it around – what you call testing is probably different than what I call testing. The difference is, that I don’t care what you call testing, and some of you
care a lot get really pissed off when somebody calls something testing that doesn’t match your definition.
Testing, in some intersection of our combined testing definitions, contains a lot of 0OI tests. “When you enter 2 + 2 in the calculator, the display shall enter 4”, or “Save the file and ensure it is created”, or “When the user clicks the submit button without a user name, they shall receive an error message”. Many in the test community refer to these as “checks”, and I’m ok with that term, but to me, these are a type of test rather than something different than tests.
Whether you call these tests, checks, or 0OI tests (my current favorite), I don’t want to do them. It’s not just that 0OI tests bore me (because they bore me to tears), but because I think it’s fiscally irresponsible to push verification of mind-numbingly basic functionality so far downstream. In my opinion, developers should do this work, but I guess if any testers find 0OI tests entertaining, I’m sure there’s a lazy developer somewhere who’d like to hire you as his cabin boy.
Short story is that 0OI tests are dead to me.
And then there’s 2OI work – the kind of work good testers love to do. 2OI testing is the land of “I wonder what happens when…”, and “How can I get the program to…” tests. 2OI tests can be manual or leverage tools or automation. Any form of discovery (or new knowledge acquisition) about the software is 2OI testing.
Testers not infatuated with Armour the way I am typically call 2OI testing exploratory testing (true for both manual or automated 2OI tests). In Armour’s recent article on testing that I mentioned in my previous post, he suggests that the sweet spot for 2OI (exploratory) effectiveness is about 50% – meaning that about half of your tests should discover product issues. If you’re finding less, you could vary your techniques, and if you’re finding more, you should back off the effort. I haven’t applied that concept yet when performing testing, but I plan to the next time I conduct a testing session.
But, the way I see it, exploratory testing is dead too. OK – that statement was overly controversial. What’s dead to me, is the term exploratory testing. To me, it’s just testing. It’s all just testing. I wrote a blog post once about this subject (see last three paragraphs to avoid most of my snarkiness) – then one of the top practitioners of ET wrote this post with a similar stance – but it seems every tester in the world is still compelled to proclaim they are “exploratory testers”, because they like to use their brain while they’re testing. All of this is fine, of course, one can use whatever terms they like, but to me, at least, it’s all just testing.
Testers do some of it. Others do some more of it. Some of it’s dead to me, and perhaps none of it is dead to you. It doesn’t matter.
It’s all just testing.