I’ve been gathering some information on test automation systems recently and will probably have a flurry of posts upcoming on automation and related subjects.
Some observations as I browse what Bing has to tell me about the subject of Test Automation:
- Wikipedia tells me that test automation “commonly involves automating a manual process already in place”
- The bulk of the test automation articles are about UI automation
- There are some reasonably good articles warning about the problems one may run into in test automation projects. None of these articles provide solutions or alternatives.
There’s more in my search results, but those three results alone say a lot about what’s wrong with the way (most?) teams approach test automation.
I expect I’ll write more eventually on all of these points, but my anecdotal experience, along with a few dozen web pages and articles tells me that there’s a lot of confusion in regards to test automation.
You see, test design (including the design of how test automation will execute) is really, really hard. It’s hard to write sustainable and reliable tests, and it’s hard to write trustworthy tests. Double the effort required if those tests involve UI automation. Designing good tests is one of the hardest tasks in software development. That’s worth saying again. Designing good tests is one of the hardest tasks in software development.
Compared to everything else in the equation, the “writing code” part of test automation is easy. Massively easy. Almost brain-dead easy. When I compare writing code to composing music, I talk about the creative and challenging aspect of melody, creating a texture and mood, and figuring out where notes and space between notes help with both. There’s also the rote part of the job – chord voicings, doubling, and other parts of filling in the score. The melody and texture choices are the test design of music composition – filling in the rest of the parts is the coding part. Sure, there’s creativity in a countermelody, as much as there’s creativity in a cool algorithm, but it’s still the rote part of the activity.
I fear that testers are worrying too much about the effort and skills required to automate tests – and not enough about what it takes to design reliable, portable, accurate, and trustworthy tests that actually matter.
Tell me I’m wrong.