Comments on: Orchestrating Test Automation

By: 2012 Recap » Tooth of the Weasel

2012 Recap » Tooth of the Weasel — Sun, 16 Dec 2012 05:18:43 +0000

[...] Orchestrating Test Automation [...]

By: To explore or automate… « Software Testing

To explore or automate… « Software Testing — Tue, 02 Oct 2012 12:39:48 +0000

[...] like Alan Page’s blog post on the similar subject: Orchestrating Test Automation Compared to everything else in the equation, the “writing code” part of test automation is [...]

By: Matt Griscom

Matt Griscom — Wed, 22 Aug 2012 22:26:39 +0000

Alan,

great to see you at SASQAG last week!

I think part of the problem is that Test is much less mature as a discipline than Dev is. This follows from Test being lower priority, and less prestigious. Of course, that works for developing games, but for enterprise software you really need to get serious about quality, and test automation is an important part of that.

I’d love to hear more of your thoughts on this topic. I’d love it if you added your comments to a blog post of mine that you find interesting! Like you, I’d like to learn from the experts.

By: A Smattering of Selenium #113 « Official Selenium Blog

A Smattering of Selenium #113 « Official Selenium Blog — Wed, 22 Aug 2012 11:47:27 +0000

[...] Orchestrating Test Automation is essentially why I look for testers who can code rather than coders who [think they] can test Share this:EmailFacebookLinkedInDiggTwitterLike this:LikeBe the first to like this. Leave a Comment [...]

By: Kate

Kate — Tue, 21 Aug 2012 22:56:33 +0000

Alan,

This is so very true. Designing the tests is the critical aspect, and I’ve suffered from far too many managers who don’t get that the combination of skills to both design tests and convince software to perform them is vanishingly rare. (It can be learned – but the people who want to learn aren’t that common either).

From my perspective, automated regression is what assures customers that the critical parts of the software meet regulatory requirements – since my employer supplies commercial software to large businesses, that means compliance with various tax rules, privacy rules, payment data storage rules, and so forth.

By: Jim Hazen

Jim Hazen — Tue, 21 Aug 2012 16:32:04 +0000

Alan,
Where to start… you are correct in your statements and observations. More so than most people will want to admit.
The ‘design’ of a test is the key thing; both manual and automated. The other thing to realize is that ‘automation’ in the typical sense, especially with front-end/UI, is that it is automating the ‘execution’ of the test only. The design, construction, analysis and other tasks that are part of a human doing testing isn’t considered by most regarding automation.
A big thing as you point out are the misconceptions/misunderstanding of what all is involved in ‘automating’ test execution. Too many times people & companies don’t see this type of work for what it is; software development. We need to think about how we build and maintain the ‘testware’, and who is best to do this type of work.
This comes from the problems of the “Automagic” wishful desires and salesjob by vendors. Management doesn’t want to hear that automation is another form of software development. Testers are not Developers, thus you don’t need people (and pay for them accordingly) with those skills and mentality.
You know the story… being at Microsoft you have been through this. But a lot of other companies have not and/or don’t want to due to various reasons (1 being cost and the other ignorance).
I’d love to rant on more, but this is a beer/phone conversation and my fingers are tired enough as is.
Looking forward to other posts you have on this subject. Just remember my favorite saying/axiom: “It’s Automation, Not Automagic!”.

Regards,

Jim Hazen

By: Oren Reshef

Oren Reshef — Tue, 21 Aug 2012 14:41:49 +0000

You are not wrong.
I am mentoring a growing automation effort (in Python) and indeed one of the greatest challenges is the design.
The developer can hack a test in a day, but he will choke when he’ll have to add the other 10,000 variations for it, analyze results and create smart reports.
I always make my automation developers approach their projects the same way as any other software project- start with a high level design, then the details (and code review it, follow coding guidelines, version control it, unit test it etc.)

By: Phil Kirkham

Phil Kirkham — Tue, 21 Aug 2012 12:42:16 +0000

I wish I could tell you were wrong – but a quick perusal of QA Stack Exchange shows it’s all questions of how to access a web element or weddriver not working, questions on STC are nearly all on ‘what is the best tool for automation’ , no questions about how to design the tests.

Looking forward to the rest of your posts on this

By: George Dinwiddie

George Dinwiddie — Tue, 21 Aug 2012 12:12:27 +0000

It’s really really hard to do in isolation, but it’s merely difficult when collaborating with the business and the programmer (what I call the “three amigos”). It still seems hard for people to talk about the “what” instead of the “how.”