There\u2019s a pretty good presentation from the folks at Electric Cloud making the rounds on \u201cwhy your automation isn\u2019t\u201d (and other variations on that title). The premise is, that testers spend too much time babysitting (supposedly) automated tests, and that testers end up doing a lot of manual work in order to keep their automation running. I know nothing about their products, but they have some reasonably good ideas and apparently some tooling to help.<\/p>\n
But it\u2019s not enough. Crappy automation and crappy automation systems is an insanely huge problem. But we tend to ignore it because \u2013 hey \u2013 we\u2019re running lots of automation, and that must<\/em>\u00a0 be good. To be clear,I have nothing against test automation (although I\u2019m leery of a lot of GUI automation<\/a>). Done well, it absolutely aids any testing effort. The other 99% of the time I bet it\u2019s actually slowing you (and your team) down. Please, please, get it through your head that your job as a tester is to test software and not to create the largest test suite known to humankind. .<\/p>\n \u2026and here are some ideas to help you get to automation that doesn\u2019t<\/em> suck.<\/em><\/p>\n The obvious place to start is with your code. Do you treat your test code like production code? Do you do code reviews? Do you run static analysis tools? Do you step through the tests with the debugger to ensure they are doing what you think<\/em> they are? Do you trust<\/em> the results from your tests \u2013 i.e. if a test fails, are you confident that there\u2019s a product bug? If you\u2019re spending hours every day grepping through test results and system logs to try to figure out what happened, your automation sucks.<\/p>\n Now think about how<\/em> your tests are run. Are they automatically built for you every day (or more frequently), and distributed to a test bed of waiting (physical and virtual) machines? Or do you walk around a lab full of computers manually typing in command lines? Or do you just run all of your automation on the spare machine in your office, then upload your results to some spreadsheet on a share where nobody can ever find it. In other words, do your tests execute<\/em> automatically\u2026or do they suck?<\/p>\n What about failures? Do you look at your automation failures in the morning, gather up some supporting data, then enter a bug? Or are bugs entered automatically \u2013 including additional information like logs, call stacks, screen shots, trace information, and other relevant information automatically?<\/em> When a bug is resolved as fixed, do you go run that failing test manually<\/em> to ensure that the bug was fixed \u2013 or does your automation system automatically<\/em> take care of manual (and mundane) tasks such as this for you? How about reporting \u2013 how do you generate the all important \u201cTest Result Report\u201d? Does your automation<\/em> system take care of it, or is it a largely manual<\/em> task.<\/p>\n Do you really have automation \u2013 or just a suck-filled wrapper around some suck-infested tests? <\/em><\/strong>It\u2019s ok, be honest. It\u2019s your choice what to do, but, but I bet the maturity of how<\/em> you test software certainly has much more to do with end user quality than the number of tests you have (mostly because the latter metric is useless, but you get the point).<\/p>\n I don\u2019t have a solution to sell you \u2013 but I can give you an architecture for and end to end true<\/span> automation system for free. The chapter I wrote for Beautiful Testing<\/a><\/em> covers this exact topic \u2013 and I\u2019ve finally got around to posting it here<\/a>. It\u2019s all yours \u2013 read it and comment here if you\u2019d like \u2013 or read it and delete it to free up some disk space. Regardless of my chapter, I still recommend you buy the book \u2013 like the other authors, I don\u2019t make any money, but the proceeds go to buy mosquito nets to prevent malaria in Africa. More importantly, the book is friggin\u2019 cool and I think every tester should own it.<\/p>\n KTHXBBYE<\/p>\n p.s. No idea at all why I decided to use a lolcat title – it just sorta felt right.<\/p>\n","protected":false},"excerpt":{"rendered":" There\u2019s a pretty good presentation from the folks at Electric Cloud making the rounds on \u201cwhy your automation isn\u2019t\u201d (and other variations on that title). The premise is, that testers spend too much time babysitting (supposedly) automated tests, and that testers end up doing a lot of manual work in order to keep their automation…<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_kad_post_transparent":"","_kad_post_title":"","_kad_post_layout":"","_kad_post_sidebar_id":"","_kad_post_content_style":"","_kad_post_vertical_padding":"","_kad_post_feature":"","_kad_post_feature_position":"","_kad_post_header":false,"_kad_post_footer":false,"_kad_post_classname":"","_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[1],"tags":[],"class_list":["post-193","post","type-post","status-publish","format-standard","hentry","category-allposts"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_likes_enabled":true,"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/posts\/193","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/comments?post=193"}],"version-history":[{"count":0,"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/posts\/193\/revisions"}],"wp:attachment":[{"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/media?parent=193"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/categories?post=193"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/tags?post=193"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}