{"id":165,"date":"2010-07-13T10:49:53","date_gmt":"2010-07-13T17:49:53","guid":{"rendered":"http:\/\/angryweasel.com\/blog\/?p=165"},"modified":"2010-07-13T10:49:53","modified_gmt":"2010-07-13T17:49:53","slug":"thinking-about-bugs","status":"publish","type":"post","link":"https:\/\/angryweasel.com\/blog\/thinking-about-bugs\/","title":{"rendered":"Thinking about bugs"},"content":{"rendered":"

I probably haven\u2019t mentioned in a while how nice it is to be back on a product team. The complexity and dynamic nature of software development is something I missed more than I knew, and I\u2019m having fun being back in the flow of shipping a product for a million or so users. One thing that I never got a chance to think about while I was in EE was bugs. To be clear, I got to think about the concept<\/em> of bugs, but not about real live bugs. Bugs are a big byproduct of the testing process, and like it or not, they help dictate some part of the flow of software development. At the very least, they give us something to think about.<\/p>\n

So let me share what I\u2019ve been thinking about\u2026<\/p>\n

Every bug is more difficult to find than the previously found bug. <\/em><\/strong>There will be little hops off the axis, but in general, I believe this to be true. Imagine if the opposite were true \u2013 if every bug were easier to find than the previous bug, we\u2019d find more bugs every day and eventually have a product consisting entirely of bugs (insert joke about your least favorite product here). <\/p>\n

As bug discovery decreases, product quality improves<\/em><\/strong>. There are plenty of reasons where this may<\/em> not be true, but if few bugs are being discovered, it probably <\/em>means that product quality is improving (or that all of the testers went on vacation) \u2013 but \u2026<\/p>\n

Tester skill and knowledge increases with each found bug. <\/em><\/strong>When a tester finds a bug, they also acquire knowledge<\/em>. They learn something about the product, or a technique or behavior that they can use to find bugs in the future.<\/p>\n

Tester skill can\u2019t grow as fast as bug discovery difficulty decreases. <\/em><\/strong>If tester skill improved faster<\/em> than the difficulty of finding the next bug increased, we\u2019d find more <\/em>bugs over time until we had a product with more bugs than \u2026yes \u2013 you see where I\u2019m going.<\/p>\n

\n

<some made up line charts would do well here, but I\u2019m feeling lazy><\/p>\n<\/blockquote>\n

What I\u2019m struggling with is coming up with a better understanding of how these statements interact. In more concrete terms, if the number of bugs discovered in a product or feature area isn\u2019t tapering off at \u201can expected rate\u201d, does that mean that product quality isn\u2019t improving as much as expected, or that tester skill is improving more<\/em> than expected? Since the answer is (you guessed it) \u201cit depends\u201d, how do you dig deeper? How do you know the right<\/em> answer with some degree of confidence?<\/p>\n

I have answers for this that satisfy me (customer data<\/a> is a big part of this), as well correlating a bunch of other measurements, but I wonder if anyone else thinks about stuff like this and has other thoughts or ideas or experiences to share.<\/p>\n","protected":false},"excerpt":{"rendered":"

I probably haven\u2019t mentioned in a while how nice it is to be back on a product team. The complexity and dynamic nature of software development is something I missed more than I knew, and I\u2019m having fun being back in the flow of shipping a product for a million or so users. One thing…<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_kad_post_transparent":"","_kad_post_title":"","_kad_post_layout":"","_kad_post_sidebar_id":"","_kad_post_content_style":"","_kad_post_vertical_padding":"","_kad_post_feature":"","_kad_post_feature_position":"","_kad_post_header":false,"_kad_post_footer":false,"_kad_post_classname":"","_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[1],"tags":[],"class_list":["post-165","post","type-post","status-publish","format-standard","hentry","category-allposts"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_likes_enabled":true,"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/posts\/165","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/comments?post=165"}],"version-history":[{"count":0,"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/posts\/165\/revisions"}],"wp:attachment":[{"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/media?parent=165"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/categories?post=165"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/tags?post=165"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}