{"id":348,"date":"2011-10-24T12:36:02","date_gmt":"2011-10-24T19:36:02","guid":{"rendered":"http:\/\/angryweasel.com\/blog\/?p=348"},"modified":"2011-10-24T12:36:02","modified_gmt":"2011-10-24T19:36:02","slug":"testing-failing-to-succeed","status":"publish","type":"post","link":"https:\/\/angryweasel.com\/blog\/testing-failing-to-succeed\/","title":{"rendered":"Testing: Failing to Succeed"},"content":{"rendered":"
\n

\u2026the paradox of testing: We want to find defects in the software under test, but we do not really want to find them".\u201dPhillip Armour<\/p>\n<\/blockquote>\n

I often speak of Phillip Amour\u2019s Five Orders of Ignorance<\/em><\/a>as they relate to software engineering (note; as of this writing, that link doesn\u2019t work, but I don\u2019t have another link to his paper). I\u2019ve mentioned the orders of ignorance on this blog<\/a>, and also in this article for Better Software<\/a>. <\/p>\n

Although I respect my readers, I know you won\u2019t read those links, so let me briefly cover the orders of ignorance.<\/p>\n

0OI<\/strong> \u2013 Zero-Oh-I is lack of ignorance<\/em>. It\u2019s when you know something. I know, for example, that the first track on the Rolling Stone\u2019s Sticky Fingers<\/em> album is Brown Sugar<\/em>. <\/p>\n

1OI<\/strong> \u2013 lack of knowledge<\/em>. I don\u2019t know (or remember) what the track is following Brown Sugar<\/em>, but I could find the answer quickly.<\/p>\n

2OI<\/strong> \u2013 lack of awareness<\/em>. You have 2OI when you don\u2019t know what you don\u2019t know. I know that there are Stones tunes that I\u2019ve never heard before, but it would be impossible for me to make a list of Stones songs I\u2019ve never heard before.<\/p>\n

30I<\/strong> \u2013 lack of process<\/em>. You have 3OI when you don\u2019t have a suitable method for discovering 2OI (for discovering what you don\u2019t know you don\u2019t know). <\/p>\n

4OI<\/strong> \u2013 meta-ignorance. <\/strong><\/em>You have 4OI when you don\u2019t know about the five levels of ignorance.<\/p>\n

In the Better Software article, I attempted to stress how important the levels of ignorance are to software testers and software testing, and I frequently bring them up in conversations about testing. Recently, my mind was blown when Phillip Armour wrote an article specifically about the application of the levels of ignorance to software testing. A direct link (for those who have access to the communications of the ACM) is here<\/a>. If not, I found a link here<\/a> which should get you to the article.<\/p>\n

And for those of you who don\u2019t click through, here\u2019s a summary.<\/p>\n

When we test to ensure requirements were implemented, or the stories function as expected, we are testing for 0OI. We know what pass and fail looks like, and we can create tests accordingly. The 0OI testing set may be large, but it\u2019s bounded. A passing 0OI test doesn\u2019t expose any new knowledge \u2013 it just proves what we thought we already knew.<\/p>\n

Testers also perform a lot of 2OI testing. We explore or run dynamic tests on a system to help discover what we don\u2019t know we don\u2019t know. Testing for 2OI is an unbounded test set \u2013 the things that a system might<\/em> do is infinite. This is where we need to devise tests when we don\u2019t know what we\u2019re looking for<\/em>. (Note \u2013 we don\u2019t test for 1OI because if we truly knew in advance what we didn\u2019t\u2019 know, we\u2019d resolve the ignorance and conduct 0OI testing. <\/p>\n

Another (shorter) way to think about this is that 0OI tests are knowledge-proving<\/em> tests, while 2OI tests are knowledge-acquiring<\/em> tests. <\/p>\n

Armour goes on to discuss the theoretical information content for a test and optimal failure rates (which I\u2019ll leave to you to read, as I\u2019ve probably already butchered the content already). It\u2019s good stuff full of great quotes, and I hope you check it out.<\/p>\n","protected":false},"excerpt":{"rendered":"

\u2026the paradox of testing: We want to find defects in the software under test, but we do not really want to find them".\u201dPhillip Armour I often speak of Phillip Amour\u2019s Five Orders of Ignoranceas they relate to software engineering (note; as of this writing, that link doesn\u2019t work, but I don\u2019t have another link to…<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_kad_post_transparent":"","_kad_post_title":"","_kad_post_layout":"","_kad_post_sidebar_id":"","_kad_post_content_style":"","_kad_post_vertical_padding":"","_kad_post_feature":"","_kad_post_feature_position":"","_kad_post_header":false,"_kad_post_footer":false,"_kad_post_classname":"","_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[1],"tags":[],"class_list":["post-348","post","type-post","status-publish","format-standard","hentry","category-allposts"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_likes_enabled":true,"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/posts\/348","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/comments?post=348"}],"version-history":[{"count":0,"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/posts\/348\/revisions"}],"wp:attachment":[{"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/media?parent=348"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/categories?post=348"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/tags?post=348"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}