Lessons from TapeOp

I had a short bit of travel this week, and got a bit of reading done on the flights. I love my kindle, and use it when in the air, but I also pack a magazine or two to read during takeoff and landing.

On the way home, I was catching up on a few issues of Tape Op. I haven’t done much with recording in years, but I still love to read about the process – both the creative and technical aspects are extremely interesting to me. Perhaps I was in a “testing mindset”, but a few of the articles screamed testing at me. I’m sort of tired of the “testing is like …” metaphors, so I won’t stretch this too far, but still felt the need to share.

The latest issue (#84) has an article by Garrett Haines on Building Troubleshooting Skills. Garrett founded Treelady Studios in Pittsburgh, and consistently has pragmatic advice for recording enthusiasts of all skill levels. As far as I know, the article isn’t available online, but I’ll summarize the main points with some testing comments and see where it goes.

Troubleshooting

One of the most important skills for a recording engineer is troubleshooting. If something goes wrong, you can either figure out what the problem is and fix it, or you can send the band home just when they’re hitting their groove. I’ve always enjoyed the troubleshooting aspect of testing. For me, finding the bug is secondary to isolating the bug. I don’t find the exact root cause (or causes), or offending line of code for every bug I find, but I do like to isolate the bug as much as possible.

Let’s say that I get to a login screen and it doesn’t work. I try again, ensuring that my test username and password are correct. I could enter a bug and be done with it, but there’s likely some more I can do (even from an entirely black box perspective). I’ll see if other username / password combinations work, I’ll try variances of case, and compare simple passwords with complex passwords** until I can get some more clues to what’s going on.

Get in the right frame of mind

When something goes wrong (all of a sudden the test code won’t run an half of the machines), don’t panic. I don’t often have days when everything goes perfectly, and I doubt you do to. One reaction is to freak out and claim the sky is falling, but it helps to expect that things may go wrong and just address those issues as they arise.

Recreate the problem

That’s weird – something isn’t working- the first thing to do is reproduce it. I love the example Garrett uses here. “Sometimes reproducing the problem  can lead to an immediate solution – e.g. lifting the mute on the ‘track that won’t play;”

I remember working on a product when a tester ran into a colleague’s office, panicked because none of the audio tests he was running were working. My colleague calmly walked to her office, and had her reproduce the problem. He looked at her computer for a moment and informed her that she didn’t have any speakers connected. This is an extreme (yet true) example, but sometimes looking at the problem – and thinking through how the success path should work is a huge help.

Isolate the root cause

Garrett has a few dozen examples of how to use this technique, and most relate to software testing. Tell me if some of the troubleshooting scenarios he lists are familiar to us in the testing world.

• Intermittent problems – these are my favorite to debug and isolate (in software, at least)
• Not understanding the studio configuration – as a tester, you have to have an idea of how the system works in order to isolate many (most?) issues.
• Invest in diagnostic tools – I couldn’t survive as a tester without debuggers, network sniffers, process monitors and other tools that let me know what’s going on
• Trace the thunder – this is the concept of thinking through the flow of how the scenario works. The path from microphone to mixer (which may go through one or more effects), is similar to the flow of logging into a system (keystrokes go to UI, which encrypts the credentials, then sends them to a server (perhaps checking a local cache first), etc.
• Make sure you’ve had enough coffee and sugar, but don’t have so much that you’re twitchy – ‘nuff said.
• Don’t be afraid to call for help if you’re in over your head – Good advice for anyone.

Repair the problem & test the solution

Not always necessary for a tester to do, but something you can try if it will confirm your hypothesis. Think your problem is due to a missing configuration file or reg key – add the file or setting and see if it fixes the problem. You’ll learn more about the system whether the solution works or not.

Learn from the experience

Garrett says, “Tell your studio mates and colleagues what you’ve learned. Post findings on some of the various discussion forums.” Can you think of better advice for testers?

There’s plenty more to learn, but I think you get the point. Software (like recording studios) are complex environments that break or malfunction – but a structured approach to troubleshooting will minimize downtime (and yak-shaving time) and you’ll make more progress in the long run.