{"id":404,"date":"2012-01-27T18:19:16","date_gmt":"2012-01-28T02:19:16","guid":{"rendered":"http:\/\/angryweasel.com\/blog\/?p=404"},"modified":"2012-01-27T18:19:16","modified_gmt":"2012-01-28T02:19:16","slug":"the-skeptics-dilemma","status":"publish","type":"post","link":"https:\/\/angryweasel.com\/blog\/the-skeptics-dilemma\/","title":{"rendered":"The Skeptics Dilemma"},"content":{"rendered":"<p>For testers, being skeptical is generally a good thing. When someone says, \u201cOur application doesn\u2019t have any reliability errors\u201d, I, for one, am skeptical. I\u2019ll poke and prod and (usually) find something they haven\u2019t thought about. There\u2019s power in skepticism. Last year, I led a team of testers in performing code reviews of production code. My hypothesis was, that while developers perform code reviews thinking, \u201cDoes the code do what it\u2019s supposed to do\u201d, testers think, \u201cIn what conditions will the code <em>not <\/em>do what its supposed to do. You can insert the comment about testers being pessimistic (or overly pessimistic) here, but in general, the tester mindset is to question statements that seem\u2026well, <em>questionable<\/em>. <\/p>\n<p>But it\u2019s easy to go overboard with skepticism. Time and time again, I hear good testers apply their skepticism broadly and liberally. Some (paraphrased) quotes I\u2019ve heard recently include:<\/p>\n<ul>\n<li>\u201cModel-based testing is interesting, but it doesn\u2019t work in a lot of places\u201d<\/li>\n<li>\u201cI\u2019m skeptical of static analysis tools \u2013 sometimes they have false positives\u201d<\/li>\n<li>\u201cMetrics are evil, because someone may use them to measure people\u201d<\/li>\n<\/ul>\n<p>I agree whole heartedly with each of these quotes. However, I worry that folks are throwing <a href=\"http:\/\/en.wikipedia.org\/wiki\/Throw_out_the_baby_with_the_bath_water\">the baby out with the bathwater<\/a>. Model-based testing (just an example), is a wonderful test design technique for stateful test problems. Although occasionally someone will screw up the value of MBT by claiming that it\u2019s the only test approach you\u2019re ever going to need, it\u2019s just another technique (and the <u>perfect<\/u> technique given the proper context). Static analysis tools are also awesome, but aren\u2019t perfect It\u2019s <em>good to measure <u>some<\/u> things <\/em>too, but sure, one can screw it up.<\/p>\n<p>I\u2019m trying to think of anything in my life that works perfectly in every situation, but I\u2019m coming up empty. I run into situations nearly every day where someone has a good idea that will obviously work <em>most <\/em>of the time \u2013 but not always. Given these situations, we could just send them back to the drawing board telling them, \u201cI\u2019m skeptical of your approach, because it won\u2019t work in situation z\u201d, but it\u2019s probably a better idea to have a conversation about the limitations, understand where and when the approach may fail, and discuss mitigation or workarounds. Instead of throwing out the idea of running static analysis tools because of the potential false positives, <em>discuss<\/em> the false positive problem. Find out what causes them. Tweak the configuration. Do whatever you need to do to ensure the <em>value<\/em> of the approach.<\/p>\n<p>Over the years, I\u2019ve found value in some pretty stupid approaches. It seems that we should be able to find more value from some the ideas frequently discounted.<\/p>\n<p>Even if we\u2019re skeptical.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>For testers, being skeptical is generally a good thing. When someone says, \u201cOur application doesn\u2019t have any reliability errors\u201d, I, for one, am skeptical. I\u2019ll poke and prod and (usually) find something they haven\u2019t thought about. There\u2019s power in skepticism. Last year, I led a team of testers in performing code reviews of production code&#8230;.<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_kad_post_transparent":"","_kad_post_title":"","_kad_post_layout":"","_kad_post_sidebar_id":"","_kad_post_content_style":"","_kad_post_vertical_padding":"","_kad_post_feature":"","_kad_post_feature_position":"","_kad_post_header":false,"_kad_post_footer":false,"_kad_post_classname":"","_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[1],"tags":[],"class_list":["post-404","post","type-post","status-publish","format-standard","hentry","category-allposts"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_likes_enabled":true,"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/posts\/404","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/comments?post=404"}],"version-history":[{"count":0,"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/posts\/404\/revisions"}],"wp:attachment":[{"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/media?parent=404"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/categories?post=404"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/tags?post=404"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}