The Problem with TheApp
Last week, I posted a small testing challenge (in short, an application wouldn’t launch).Shmuel Gershon has a wonderful write up of how he found the problem here.
For those too lazy to read, the problem was that the application attempted to read a reg key on launch, and failed to launch (with no diagnostics) if the reg key didn’t exist.
Having done a massive amount of application compatibility testing over the years, this is a fairly common scenario I’ve run across (simplified quite a bit in this case). Typically, it works like this: a team makes an update to an application or operating system, and some application begins to fail in a bizarre way (including failing to launch, as well as functionality or stability changes). After some investigation, we usually discover that the application relied on a registry key, a file, or functionality that no longer exists, or has changed. Half of the time, the application vendor is out of business, or doesn’t have the bandwidth to make an update, so we have to figure out what went wrong and hack do whatever is necessary to ensure the application continues to work.
Some of the time, the investigation requires a bit of reverse engineering, but at least 9 times out of 10, I use tools like sysinternals procmon to see what’s going on. I was happy (but not surprised) to see that Shmuel ended up using this tool to find the issue (and was even happier to see how well the tool led him to the exact issue).
For my job, I do almost as much investigation into why things don’t work as I put into discovering what works (and doesn’t work). To me, the detective work is part of the testing role. I was surprised yesterday, when a tester told me that his job was problem detection only, and that the rest was for someone else. That world of testing would be boring to me – ymmv.
Congrats Shmuel for the great investigation, and for having this valuable tool in his toolbox.
Note: for simplicity, I put the key under HKCU so everyone could access it, and used an obvious key name (MagicValue) – in the real world, it’s not always this way.
It was great to read Shmuels write-up of his investigation, almost as good as pairing with him.
I’ve dabbled a bit with Procmon, I’ll have to dabble some more
thanks for the challenge, it could have fallen flat with no responses but turned out to be a good learning opportunity
I enjoyed very much the thrill of reading Shmuels investigation. This is a great way to improve my tests skills. Thank you Alan and Shmuel
Thanks Alan. Shmuel post was very insightful and lots of insights for myself.
Phil,
“almost as good as pairing with him” … That is praise for Rapid Reporter and the way it leads a tester to share his mind. 🙂
I’ve seen once and again sessions that are made intelligible by Rapid Reporter. Yay.
Alan,
“I do almost as much investigation into why things don’t work as I put into discovering what works (and doesn’t work).”. Nice point, and good approach.
However, while the extreme of “problem detection only” can be harmful (and boring), the other extreme of insisting too much on an investigation can also be harmful. I was once told by a (good) manager to consider investigating less, as when it is too much I could invest this time into finding more and new problems. I think often times this is a valid consideration, and the right answer of course follows rule #17.