\n
1:<\/span> Test Case Steps:<\/pre>\n<\/p>\n
2:<\/span> 1) Launch App<\/pre>\n<\/p>\n
3:<\/span> 2) Press the Roll! Button <\/pre>\n<\/p>\n
4:<\/span> <\/pre>\n<\/p>\n
5:<\/span> Verify:<\/pre>\n<\/p>\n
6:<\/span> 1) Ensure that none of the numbers is less than 0 or greater than 9<\/pre>\n<\/p>\n
7:<\/span> 2) Ensure that the total field is correct<\/pre>\n<\/p>\n
8:<\/span> <\/pre>\n<\/p>\n
9:<\/span> Repeat this test at least 10 times<\/pre>\n<\/div>\n<\/div>\n
Now, the \u201cautomator\u201d comes along and writes this test:<\/p>\n
\n
\n
1:<\/span> Loop 10 times<\/pre>\n<\/p>\n
2:<\/span> App.Launch (\u201cnumberz.exe\u201d)<\/pre>\n<\/p>\n
3:<\/span> Button.Click(\u201cRoll\u201d)<\/pre>\n<\/p>\n
4:<\/span> \/\/ Validate numbers are within range <\/pre>\n<\/p>\n
5:<\/span> \/\/ Validate correct total<\/pre>\n<\/p>\n
6:<\/span> End Loop<\/pre>\n<\/div>\n<\/div>\n
From what little context I have from the interwebs, this appears to be a common scenario — but we\u2019ve missed the critical aspects of testing this app (testing randomness \/ predictability). What we haven\u2019t done at all in this situation is test design<\/strong><\/em>. To me, test design is far more holistic than thinking through a few sets of user tasks. You need to ask, \u201cwhat\u2019s really going on here?\u201d, and \u201cwhat do we really need to know?\u201d. Automating a bunch of user tasks rarely answers those questions. (Note \u2013 it does answer some<\/em> questions, so if you have a good system and tests you can trust, by all means don\u2019t stop what you\u2019re doing).<\/p>\nWhere I think I have my big disconnect is in the definition of test automation. When I think of test automation, I don\u2019t think of automating user tasks. I think, \u201cHow can I use the power of a computer to answer the testing questions I have?\u201d, \u201cHow can I use the power of a computer to help me discover what I don\u2019t know I don\u2019t know?\u201d, and \u201cWhat scenarios do I need to investigate where using a computer is the only practical solution?\u201d. <\/p>\n
Perhaps test automation is purely the automation of manual tasks, and I\u2019m attempting to overload the word. I know some folks prefer the term \u201ccomputer assisted testing\u201d, and I suppose that\u2019s fine too.<\/p>\n
To me (and I\u2019m sure I\u2019ve used this line before), it\u2019s just testing<\/i>. But please stop<\/strong> thinking of test automation as the step that follows test design, and start thinking of test design first.<\/p>\n","protected":false},"excerpt":{"rendered":" I try to read a lot about testing in blogs, articles, books etc. A few days ago, I came across this quote, and it struck me in an odd way. \u201cCommonly, test automation involves automating a manual process already in place that uses a formalized testing process\u201d The source doesn\u2019t matter, as it turns…<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_kad_post_transparent":"","_kad_post_title":"","_kad_post_layout":"","_kad_post_sidebar_id":"","_kad_post_content_style":"","_kad_post_vertical_padding":"","_kad_post_feature":"","_kad_post_feature_position":"","_kad_post_header":false,"_kad_post_footer":false,"_kad_post_classname":"","_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[1],"tags":[],"class_list":["post-412","post","type-post","status-publish","format-standard","hentry","category-allposts"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_likes_enabled":true,"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/posts\/412","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/comments?post=412"}],"version-history":[{"count":0,"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/posts\/412\/revisions"}],"wp:attachment":[{"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/media?parent=412"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/categories?post=412"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/tags?post=412"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}