Debugging For Testers

I came across a few comments and statements recently that set off a slightly red (orange?) flag for me. The gist was that debugging was for coders, and that testers “just found the issues”. I get the context (or at least I think I get it) – I agree that all testers don’t have to be coders, and that all testers certainly don’t have to be debugging masters, but I
don’t think that any testers should run away from a debugger in fear either.

In fact, I’d say there’s a minimum amount of information that every tester should know about debuggers, regardless of whether they are monster-rock-star programmers, or part-time IT application validators. I’ll start a list here, and likely add to it when I’m annoyed.

So here goes:

Things Every Tester Should Know About Debuggers

  • Call Stacks – A “call stack” is a list of functions that led up to the application crash. Different debuggers display this differently, but it will look something like this:
    FunctionThatCrashed
    Function_Three
    Function_Two
    Function_One

    Read this from bottom to top – Function_One called Function_Two, which called Function_Three, which called FunctionThatCrashed (which crashed).

    Typically you’ll see parameters after the functions, and those will probably give you a clue. If you’re working with developers, and you see a crash in the debugger, the call stack is probably the most important contextual information you, as a tester, can provide.

  • Crash vs. Assert – When a program breaks into the debugger (meaning the program stops executing, and the debugger springs into action), the program may have crashed (e.g. attempted to read/write to/from invalid memory), or it could have broken at a developer-induced breakpoint (often in a macro called an ‘assert’). On x86 / amd64 platforms this shows up as an int 3
    or int 2C in the debugger. The important thing to note in the case of the assert type of error is that the reason for the break is obvious  – or at least obvious if you can view the code. In practice, asserts often look like this:
      bool bSuccess = CreateUserAccount();
      //break into the debugger of the call to CUA fails so we can debug
      ASSERT(bSuccess == true);

When a tester sees this break, instead of saying, “The program crashed during logon”, they can say, “I’m hitting an assert because CreateUserAccount is failing on today’s build”. While both statements have value, the latter statement is much more informative, and in many cases actually actionable.

  • Basic Lingo – Know a little about a few basic types of crashes – e.g. Access Violation (AV), Stack Overflow, or Buffer Overrun, and know a little about what causes these errors (e.g. reading from a bad memory address, calling too many functions – usually via recursion, or putting too long of a string into a buffer). You don’t have to be an expert, but a little knowledge here goes a long, long way in diagnosing bugs and getting them fixed.

Off the top of my head, that seems like enough, but not that much. There’s certainly more to learn, but I think knowing just this much should be a minimum for any professional tester.

More ideas – add ’em to the comments.

Comments

  1. Of course you are right – debugging is handy for testers as a way to add useful and important information to bug reports, as a way to narrow down the scope of a bug, etc. The next question is always “how far should the tester go?” The right answer is “go just far enough, then no more”, and the line is drawn differently in every company. Some companies encourage tester to debug and then fix the bug. Others just want a bug report written with sufficient information so that the developer can rapidly debug.

  2. Great post!
    Recently many people say “testers should have the knowledge of code”, but didn’t say “how it could be”. In this context, your post will be a piece of easy and comfortable first step to the testers willing to learning about code. To spread your knowledge and thought, May I translate this article into Korean and post on my blog (http://angel927.tistory.com)? It will be helpful to testers in Korea if you allow. Thanks!

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.