Despite the lame title, this isn’t yet another post attempting to describe what testing is – it’s just something I wanted to share inspired by a small discussion last night (although if you browse the testing blogosphere, you’ll find a lot of good discussion on this topic lately).
I went to Seth Eliot’s talk at SASQAG last night. Afterwards, Seth, Keith Stobie and I were talking (actually, Seth and Keith were talking, and I was listening), and Keith mentioned that some of what Seth talked about wasn’t actually testing. Seth said, "sure it was.", but Keith disagreed. Finally Keith said, "not every bug finding activity is testing." An example was that Seth was talking about analyzing customer log files in order to find bugs, and calling it testing. I suppose technically, that’s an analysis activity and not a testing activity…or is it? Another example are code reviews – they’re a valuable bug finding activity, but certainly not a test activity…
I think it’s easy to blur the line between what testing is, and what testers do. I’m not convinced it’s correct to blur the line (or incorrect for that matter), but I do think it’s a frequent cause of confusion among testers. If you were to ask a tester on my team, for example, what they do, they would say "I’m a tester". But if you asked them to elaborate, they’d say something like, "I review code and product designs, I analyze customer data, I write automated tests, I debug and analyze test failures, I identify risks (and investigate risk mitigation), I measure things, I explore the application, work on cross-group initiatives, I work with developers, I debug, I try out new ideas, I work on quality initiatives, etc.).
I suppose you could look at that in two different ways. One is, "wow, the testers on your team do a lot of stuff," or "wow, the testers on your team don’t do very much testing". Either answer is fine, as the answer depends entirely on your definition of testing, and whether you think testers should only do testing, or whether they perform analysis and prevention tasks as well.
The question is, how much of this is testing? I think I’ve always considered all of these things to be part of the testing world, but I know others see testing as only one or two of those activities. I’m not saying anyone is right or wrong in coming up with their definitions of testing; instead I realize even more than ever that the different views can cause communication problems (as of right now, I’m sort of leaning toward calling very little of this stuff testing, but I may change my mind tomorrow). I think the view differences are largely contextual – on many teams testers do just a subset of that list, and that’s perfectly fine. I suppose that’s just another reason why titles mean little, and experiences and activities mean much, much more.
I was going to ask for reader comments on what testing was, but I don’t think consensus (or verification that there are different views) is important here. What I think is important is that every tester and test team define the role of testing for their context, and do what’s feasible and practical given the product, team, schedule and other factors.