Here\u2019s a story I hear often. The names have been changed to prevent the guilty.<\/p>\n
Jake had barely taken a sip of his steaming coffee when he saw that thirty-two of the automated tests failed in last night\u2019s test pass. \u201cCrap, I\u2019m slammed today\u201d, thought Jake, \u201cI don\u2019t have time to look at thirty-blanking-two failures\u201d. Without a second thought, Jake clicked the \u2018re-run failures\u2019\u2019 button on the web page that displayed results and turned his attention back to his coffee. After finishing his coffee and filling a second cup, Jake was happy to see that twenty-five of the failing tests now passed. \u201cMust be a flaky environment\u201d, thought Jake as he took a big swig of coffee and got to work investigating the seven remaining failures.<\/i><\/p>\n
A few weeks later, Jake was sitting in in a meeting to go over a few of the top the live site failures reported by customers and the operations folks. Ellen, the development manager was walking through the issues and fixes, and throwing in a little lightweight root-cause analysis where appropriate. \u201cThese three\u201d, she began \u201ccaused a pretty bad customer experience. When we first looked at the errors, we figured it had to be an issue with the deployment environment, but we discovered that we could reproduce all of these in our internal test and development environments as well.\u201d Jake\u2019s stomach sunk a bit as Ellen continued. \u201cIt turns out that although the functionality is basically broken, it will work some of the time. I guess our tests were just lucky.\u201d <\/i><\/p><\/blockquote>\n
In some versions of this story, Jake steps up to the plate and takes responsibility. In other versions, he merely <\/em>learns a lesson. In a few versions of the story, Jake calls the whole thing a fluke and goes through the same thing later in his career.<\/p>\n