Chris McMahon’s latest post (Just Fix It) proposes that as far as bug tracking goes, the best course of action is to skip the “tracking” part of the workflow and “Just Fix It.” I’m a huge fan of this approach and think that for the most part, tracking a large number of bugs in a big fat bug system (and often overemphasizing the church of the bug curve) pretty much encourage a test-last / test-quality-in workflow.

I see this concept come up frequently, and I’ve noticed a bit of a trend. Teams that follow Just Fix It love it. Teams that prefer to fix bugs later are sure that the concept won’t work for their team, and that they need the curve and tracking data in order to ship their (undeniably unique) product. As a side note, one fun thing I’ve talked a few teams at Microsoft into doing is to pair on bug fixes – when Joe-Tester finds a bug, he Tells Kathy-Developer about the issue, and then the two of them pair-program the fix. I could write a whole post on why this is so cool, but I’ll leave it at that for now.

In short, I’m a huge fan of Just Fix It, but as usual, the totality of overlap of agreement between Chris and I is about 93%.

For example, let’s say on Monday, Alex-Tester notices that the froobnozzle isn’t working. He tells the developer, who fixes the problem immediately. Tuesday, Beth-Tester notices that the froobnozzle doesn’t work when interacting with another part of the system. She tells the developer, who fixes the problem right away. Over the following days and weeks, a lot of problems are discovered with the froobnozzle. The froobnozzle, is, in fact, a piece of crap held together by the 1s and 0s version of spit and duct tape. A bug tracking system let’s you see (I hate to say this out lout…) trends of where errors are. Bugs don’t appear randomly sprinkled throughout a product – they tend to congregate in clumps. Knowing where the clumps are can guide further testing, or risk decisions.

Source control almost mitigates this concern, but unless you have a diligent comment policy for check-ins, you probably won’t be able to differentiate between a “I was adding new code or functionality” check-ins vs. “I was fixing shit I broke” check-ins.

But I won’t go as far as to say you need a bug tracking system. As Chris describes it, and as most people use them, a bug system is really a work-item tracking system anyway. If you track work items on post-its or notecards, bugs should work the same way. As far as trends go, I think a simple tic-mark system would work just as well. When you discover a problem with the froobnozzle, write it on the board and put a tic-mark next to it. When a component gets n number of tics, schedule refactoring time, or do a design review (or both). Alternately, look at the components with the highest number of tics during the retrospective or sprint planning and review those for potential re-work.

But other than that, yeah. Just Fix It!

Note: I would have posted this as a comment on Chris’s blog, but I, for one, find the comment forms on blogspot completely unusable.

I posted a few months ago regarding my move to using a virtual machine as my “mail” machine. I’m still a huge fan of that approach, but in my new job (the really cool one working on the Xbox team!), I’m running Win7 rather than server as my main machine. I looked into using an internal hosted VM server solution, but since the solutions I found are used primarily for testing, performance wasn’t quite what I wanted (or what I was used to).

Let me outline the problem. I don’t look at email or twitter or my rss feed all of the time – in fact, most of my electronic communication time occurs when I’m compiling / building (I could use that time to have a sword-fight, but I find it better to at least pretend I’m productive). I love to group the “non-essential” stuff together, and when I’m waiting on a build, it’s the perfect time to write email, documents, tweet (or sometimes blog).

The problem is, that when I’m compiling, my 12 cores are pretty busy. This makes reading email, or doing pretty much anything else that would like more than a few time slices of cpu power sort of suck.

I have two computers (a desktop and a laptop), so I have an option of using my laptop for those tasks while the compiler elves are taking over my computer, but given the choice of using a 24” monitor and a full size ergonomic keyboard over a 13” monitor and compressed keyboard, which would you choose? To be fair, I adore my laptop (Lenovo X200s) – but I adore it because it’s small and I can use it anywhere (including on an airline tray table while the monster man in front of me leans his seat all the way back for 8 hours straight). But it’s not a computer I like to write large documents or presentations on.

The solution, as many have already guessed, is to connect to my laptop via terminal server and use it from my desktop machine. The overhead of TS is small enough, that I barely notice a perf problem even at peak memory and cpu usage. The TS session even stretches my desktop to 1920×1080, so it’s a pretty sweet setup. All that is good (and somewhat obvious), but few things in life ever go without a hitch.

Despite how much I wanted to love the setup, I noticed that my connection would drop frequently – and then it would take a minute until I could reconnect. I powered through it for a day or two (I tell myself that I was subconsciously gathering clues). Then finally, I found the clue that led me in right direction. I listen to music on Zune quite a bit while I work (I configured terminal server to play audio on the server machine (my laptop), so I select songs from the TS session on my desktop, and listen through headphones connected to my laptop). So…I noticed that whenever my computer disconnected, I lost audio a few seconds later. The first few times this happened, I (ignorantly) assumed that because I lost the TS connection, I also lost the audio. Then it hit me that since the audio is playing on my laptop, I should never lose audio…unless the network connection was lost.

Losing the TS connection and losing audio were both symptoms of the same root cause (lost network connection) – the only thing left to figure out was why two computers on the same subnet were losing their connection.

I took a look at power management settings to ensure that my laptop wasn’t suspending, but all was good. My next step was to look at my adapter settings to see if the network adapter was powering down for some reason. Sure enough, there was a setting named “System Idle Power Saver” that was enabled. I disabled the setting, and I’ve only had one dropped connection all week.

This example may sound more like yak-shaving than testing, but I’ve always liked the trouble shooting aspect of testing. Going beyond “here’s a problem”, to “here’s what (probably) caused the problem” is, in my opinion) an important skill for testers.

Oops – my build is done – time to get this posted.

I’ve been thinking about testing activities and tester roles lately, or more lately, since it seems to be a recurring topic on this blog. I have to admit that I remain a bit surprised how often testers argue about what is and what isn’t testing.

So – let me throw a few examples of potential testing activities out there for discussion or comment. For the scenarios below, consider two questions:

• Does this describe a testing activity?
• What role typically performs this activity?

Alex practices Test Driven Development. He consistently writes unit tests before implementing the code those unit tests verify. Over time, he’s built up a large set of unit tests that verify the correctness of all of the modules he owns. His unit tests give him the confidence to refactor or add functionality at any time.

Beth works closely with another teammate to ensure that the code implementation is highly testable. This includes pairing on many of the coding tasks, as well as adding hooks or accessible interfaces to the implementation when necessary. Her work results in code that the team finds straightforward and easy to test.

Chuck spends a lot of time analyzing code. Mostly, he’s a rock star code reviewer, and has found dozens and dozens of critical security and reliability issues in his team’s code base during the review process. He often uses tools and techniques like static analysis and complexity measurements in order to target his efforts. The whole team looks to him for critical code reviews, and he’s a mentor for many of the new team members.

Denise painstakingly enters test cases into a test case management system, and then regularly runs subsets of those tests as dictated by her management. She likes the predictability and structure of this approach and his attention to detail makes her successful in her job.

Frank finds product bugs by the handful. He uses a variety of test techniques, approaches, and tools to find several high quality bugs every day. Everyone on the team values his efforts, and many people on the team look to him for coaching on test approaches. His approaches are unscripted, but nobody doubts the effectiveness of his approach.

Fiona spends most of her day writing automated regression tests, and is constantly learning new approaches to writing automation. She writes high quality, trustworthy code that has few false positive / false negatives, and has earned the respect of everyone on the team. She enjoys writing automated tests and is proud of her accomplishments.

Greg works closely with the user experience and project management teams to ensure that the customer experience of his team’s product is of high quality. He frequently analyzes reports from customers and makes sure that all relevant team members know what the most critical issues customers are facing. He uses his broad knowledge of customer data to be influential on both product and test design.

Helen is a bit of a tool-smith. Although her team uses a lot of open source tools, she writes the applications and utilities that stitches the tools into an end-to-end testing framework. She loves thinking “big picture” and ensuring that all of the components in the system work together seamlessly. She doesn’t perform any product testing, but her work enables many of the testing activities to be successful.

To me, I could make a case that each of the scenarios above describe a testing activity – but I could also make a case that many do not. I’ll leave my observations at that for now and see what happens in the comments.

As I’m writing this (or at least starting to write this), I’m on a plane from Frankfurt to JFK airport in New York (where I will catch another flight home to Seattle). I was in Frankfurt for the first ever German Testing Day event. The event is put on by the same folks that put on Swiss Testing Day, so I knew it would go well.

For those interested, here’s a quick recap of the trip.

November 6-7

I flew from Seattle to Detroit, then on to Frankfurt. I just reached “Silver Medallion” status on Delta, so I got a free bump to economy-comfort class and an extra few inches of room on the Detroit to Frankfurt leg of the trip. I’ve been to Frankfurt at least a dozen times, but this was the first time I’ve left the airport (travellers in the audience already know that Frankfurt is a busy airline hub).

I arrived at my hotel (~10 kilometers or so outside of the downtown area) around noon. In an attempt to fight off jet lag, I dumped my bags and then set off to explore. The train station was a 5-minute walk away, and within 20 minutes or so I was in downtown Frankfurt. I took a break at a Starbucks (who apparently have free wireless worldwide these days) to sync my mail and do a bit of work before grabbing some dinner and heading back to my hotel to sleep.

November 8

After a pretty good night of sleep, I had just 3 things on my agenda. 1) Explore more of Frankfurt; 2) Prep for my presentation; 3) Meet the conference board for dinner at 8:00pm.

I combined the first two by working at a coffee shop until I couldn’t stand the wooden seat anymore, then walking around Frankfurt until I couldn’t stand that anymore. Try as I might to park myself somewhere other than Starbucks, it was the only place I could find that had free Wi-Fi access. Coffee Fellows and O’Reilly’s pub both had free Wi-Fi, but they both required that I have a working mobile number (I asked about this at O’Reilly’s, and they said it was so they could track me down if I downloaded exceptionally large amounts of data).

One interesting discovery was a “Hammering Man” sculpture, since we have something almost exactly the same in Seattle (I’ve discovered these are two of three Hammering Men in the world).

                         

By 4:00 or so, I was feeling a little drained, so I went back to my hotel to dump my laptop and take a quick nap before heading off to dinner. It was great seeing Adrian Zwingli from Swiss-Q again and catching up with him. He’s such a passionate person and the perfect person to run these events. After a few beers and a plate of cheese Spätzle that was just a bit too rich to finish, we all headed back to the hotel.

November 9

After another fairly good night of sleep, I got up, prepped a bit more, then took a taxi to Jahrhunderthalle – the site of the conference. I gave the opening keynote of the conference (no pressure!) on Test Innovation at Microsoft. I haven’t given a talk on this topic before, and I had a huge number of insights and revelations during my preparation. I think this is a topic I will continue to develop (both test innovation in general, and sharing the amazing test innovations at Microsoft). The GTD folks got the first glimpse into some of the cool things MS is doing in testing – as well as some tips on how to make innovation happen (hint – you don’t start by saying, “I think I’ll innovate today!”). I felt like the talk went well, but also think that this is a talk I can repeat and improve on in the future as my own thoughts on the subject evolve.I did have quite a few people give me compliments on the talk during the day, but I also realize that people generally don’t track down a speaker to tell them that their talk sucked.

I spent the rest of the day listening in on some of the English talks at the event, and getting a bit of work done. A pair of presenters from eBay talked about test automation principles, and they had a good message (meaning that I agreed with what they had to say, and think they have a good approach). I talked to them a bit afterwards too, and they seem like great people too.

After the event, Sebastien Kernbach (Conference Head from Swiss-Q) one of his colleagues (a woman whose name I have, unfortunately, forgotten) and I had dinner together at the hotel before heading off to bed. Not a crazy night at all, but I was beat, and ready for sleep.

November 10

Time to fly home. I took the hotel shuttle to the airport, had some breakfast, and got on my flight. The connection at JFK went smoothly, and I was home, in my own bed by 9:00pm Seattle time.

Overall, I had a great time on the trip, and once again, the Swiss-Q folks showed me that they know testing and know how to create test events. I was honored to have been invited, and hope I have the opportunity to present at another of their events sometime in the future.

But for now, my travel schedule is completely empty, and I plan to keep it that way for as long as I can get away with it.

Chris Mcmahon once (I think after reading this post) described me and my test role at MS as being like a “tenured professor” – that I have the freedom to choose what I do and choose how I go about it (as long as I show value). I enjoy this role, and think I’ve been able to help a lot of my team members  become better testers.

But sometimes, even tenured professors are drawn by the lure of an insanely hard challenge. If bigger challenges weren’t available, I suppose tenured professors would stay in their jobs for decades (and I know that many of them do). However, I know of several of my own former professors who found new roles lured entirely by the challenge of the unknown. Even if Indiana Jones would have been tenured, I’d bet he’d have left his position if he had the opportunity for the right archaeological challenge.

So – even though I’ve only spent a relatively small amount of time with the Lync team at Microsoft, the lure of the insanely hard problem has pulled me away. About a week ago, I began a new position on the Xbox team where I will work on some of the interesting challenges in testing the xbox platform, including a deep-dive into testing the Kinect device.

The Lync team was (and still is) awesome, and will always be one of my best memories of Microsoft. They will continue to be wildly successful without me (and to tell you the truth, the Xbox team would be successful without me too), but it’s the right move for me to make right now.

Along with my new job, I have a new office, a new desk, a new computer, and a new Xbox – and testing challenges I have no idea how to solve…yet. And I’m pretty excited about all of it.

More to come.

I’m in Frankfurt for a few days, and tomorrow (Wednesday), I am giving a keynote at German Testing Day on Test Innovation at Microsoft.

At this point, I’ll give you a few moments to chuckle, laugh, or make snide remarks about the topic. I think I’ve heard it all before; “testers at MS just write automation all day”, “testers at Microsoft don’t care about quality”, or “testers at MS don’t get testing” (all quotes I’ve heard or read about MS testing – and all, interestingly, by people who have never worked at Microsoft). This post is, I suppose, inspired by my talk tomorrow, but is more of a general example than a reflection of the topics of tomorrow’s presentation.

I think our lack of open-ness in how we test and innovate in testing at Microsoft is (IMO) one of our biggest flaws as a company – or at least one of our biggest flaws in being a good member of the worldwide testing community. We tried to make a dent with testercenter, but that project is all but dead. A few of us who test at MS share what we do, but none of us do a great job sharing our innovations. I hope we can get better at that soon. The short and honest truth is that I can easily name at least fifty testers at MS whose testing skills and ability to innovate are equal or better than any tester you know. I suppose in a company of nearly 10k testers that’s not really a big deal, but it’s really hit home as I’ve prepared for this presentation. I have some ideas for getting some of these people and their ideas on the global radar, but for now, you’ll just have to trust me.

I’m probably not the right person to lecture (or write) about innovation, but I can tell you what kinds of innovative testing solutions tend to be successful (at MS at least). A lot of innovation starts with the “There’s got to be a better way” approach. The worst rut to get into in software development is the “we’ve always done it that way” syndrome. Innovative testers stop, look at the big picture, and ask, ‘”is there a better way to do this?” Some of the biggest test innovations at MS have been solutions that take something complex or difficult (or “impossible”), and making it brain dead simple. One tester created automatic filtering of code coverage reports based on a specific change set – in other words, he made it brain dead simple to know if you (or any tester) tested each changed line in a specific fix or feature addition. Some of our games testers have created a system that allow the bug triage team to navigate to the exact place in the game where an error occurs from the bug tracking system (and conversely, the ability to create a bug from the place in the game where the tester observed the error).

Other innovation happens when someone finds a way to implement a known idea into a testing process. Testing localization is a huge (and hugely expensive) task. Ross Smith (and a few of his cohorts) had the idea to optimize the task via crowdsourcing and gaming elements.The results have been phenomenal.

Eric Sevareid once said,

The chief cause of problems is solutions.

Many years ago, test systems at MS reached a level of maturity and scale that enabled teams to easily run millions of tests. These systems[1] were the culmination of several innovations, but eventually led to other problems. The prime example of a problem created by the test system innovation is the challenge of what to do with all of those failed tests. Think for a moment. If you run a million tests and have a 99% pass rate (I know, but this is just an example), you have ten-thousand failures to investigate. Automatic failure analysis (discussed in HWTSAM) is an innovation that greatly reduces the time testers need to investigate automation failures (my stance is that automation is worthless unless every aspect of automation beyond test authoring is not also automated). We’ve also made great strides in test selection (let’s say you only had time to run five thousand of those million tests – which would you run?).

Of course, innovation occurs in the testing itself (as well as in the systems surrounding the testing). We’ve done amazing things with test generation, test design, and run time analysis.

Some of which I’ll share with the audience tomorrow – and likely in this blog sometime in the future.

 

[1] I described a test automation system in Beautiful Testing.

This post isn’t really testing related, so skip it if that’s what you’re looking for.

Late Saturday night, I returned from my first trip to Israel. I was invited to present at the Intel Software Professional’s conference in Tel Aviv a few months back, and I gave an updated version of my STAR East 2010 keynote on improving methods of gathering customer usage data. I also spent some time at Microsoft’s office and did a bit of sightseeing. It was a quick, but fun trip that I would do again in a heartbeat.

The long version follows.

Monday & Tuesday, October 24 and 25.

My flight out of Seattle was mostly uneventful.As someone who hates to check bags, I was a bit bummed to find out that Lufthansa had a carry-on limit of 8 kilos (my non-full and size-appropriate bag was 10 kilos). I’ve had enough bad experiences with lost or delayed luggage that I prefer to carry-on for anything less than a week. In the end, it didn’t matter (more later).

I flew from Seattle into Frankfurt, and arrived with just enough time to run to a different terminal and catch my flight. I had 5 or so minutes to spare, so I suppose I could have just jogged, but after the 11 hour flight, I needed a bit of exercise anyway.

I landed in Tel Aviv mid-afternoon on Tuesday. Since I sat in row 54 of an HFP (huge plane), I was just about the last person on our flight to arrive in a packed passport control room. After spending some time in the second slowest line, I switched to the actual slowest line and eventually was allowed into the country proper. By this time, my bag was off the plane and was one of the few left on the carousel, so I grabbed it, found a taxi, and went to my hotel.

I stayed at the Dan Panorama in Tel Aviv. After admiring my beach view and unpacking, I found something to eat and went to bed.

Wednesday, October 26.

I used to never have a problem with jet lag. As yet another sign of my age, I didn’t get a great night of sleep, but at 5:30am I got out of bed and went for a run on the beach (or next to the beach). It was fantastic – perfect weather, the sound of the ocean, and a bit of blood pumping through my system gave me energy to face the day.

Microsoft Israel has offices in Haifa and Herzelia and I spent the day in Herzelia talking with testers in a variety of settings. Surprisingly, I had energy all day and had some good conversations (from my side at least – I hope the folks in Israel found the discussions helpful).

Then it was back to the hotel for dinner, a bit of work, and bed.

Thursday, October 27.

Once again, I didn’t sleep well (I can’t believe I claimed for years that jet lag was something I never worried about). The good news is that I finally finished reading Life –  the Keith Richards autobiography. It’s not the most well written book, but to me, it was a fascinating page turner. I’m not a huge fan of the entire Stones’ backlog, but to me, the 10-year period from Beggar’s Banquet through Some Girls is some of the best rock music ever made.

Since I’d given up on sleep, I loaded a dozen or so of the Rolling Stones singles from that period and went for another run on the beach. When I got back, I had breakfast, checked in with Shmuel (my host from Intel), and …went back to bed.

My presentation at Intel was in the afternoon, so I had time to get some rest, wake up, and prepare a bit more. From my point of view, the presentation went well – the audience seemed engaged and receptive, and I received good comments afterwards. I feel fortunate to have been invited and hope the folks at Intel found my presentation useful.

Shmuel and I went out to dinner, then we both called it a night.

Friday, October 28.

Shmuel picked me up Friday morning for some sight-seeing in Jerusalem. First stop was the fascinating (and somewhat depressing) Yad Vashem – the Holocaust Memorial. From there, we went to an open market, wandered a bit and had lunch. The final stop of the tour was Old Jerusalem and the Western Wall. I was fortunate enough to meet Shmuel’s wife and five children there as well. After a bit more wandering, I headed  back to Tel Aviv.

I had a plan for my Friday night, and I’m happy to say it worked out well. When I got back to my hotel, I took a 5-6 hour “nap”, and woke up (as I hoped), around 1:00am. After freshening up a bit, I took a 15 minute walk up to Mike’s Place to watch game 7 of the baseball world series. It’s a bit strange watching a baseball game that starts at 2:00am, but it was fun to be able to watch the game (go Phillies), and sort of surreal to walk back to the hotel along the beach at dawn after the game.

Saturday, October 29.

After showering at the hotel and packing, I left for the airport. Leaving Israel is an interesting experience. My “exit interview” was 20-30 minutes long and reminded me a lot of the police interviews you see on television and the movies. I pretty much told them what I did every minute I was in Israel and they doubled back once in a while to check my answers for consistency. I was tired, but had nothing to hide, so found the experience kind of fun. Once the interview was over, I checked in and made my way to the gate.

The flight home was long (made longer by delays in Newark due to the weekend snow storm), but I did eventually make it home and spent Sunday relaxing with my family. Monday would bring a whole new adventure (and stage) to my life, but I’ll save that for the next post.

 

I’ve been having one of those experiences where a lot of the random bits of flotsam on my radar are sort of converging. But I don’t know if it’s really convergence, or if I’m forcing it. I hope it’s the former, but time will tell.

In my last post, I reflected on a wonderful article from Phillip Armour. His breakdown of “tests that tell us what we know” (0OI tests), and tests that tell us what we don’t know we don’t know (2OI tests) aligns perfectly with something that’s been gnawing at me for a long time.

You see, I’m also bothered (and have been for years) to some extent by one of the more popular testing debates – the scripted vs. exploratory tests…and to top it off, I’m completely confused by the variety of reactions to the “test is dead” statement one of my old friends has been spouting about recently (I warned you about random thoughts floating in my head…)

But it’s kind of coming together for me – some of it as I write these paragraphs.

A part of the problem is that what I call testing is probably different than what you call testing. That’s also true if I turn it around – what you call testing is probably different than what I call testing. The difference is, that I don’t care what you call testing, and some of you care a lot get really pissed off when somebody calls something testing that doesn’t match your definition.

Testing, in some intersection of our combined testing definitions, contains a lot of 0OI tests. “When you enter 2 + 2 in the calculator, the display shall enter 4”, or “Save the file and ensure it is created”, or “When the user clicks the submit button without a user name, they shall receive an error message”. Many in the test community refer to these as “checks”, and I’m ok with that term, but to me, these are a type of test rather than something different than tests.

Whether you call these tests, checks, or 0OI tests (my current favorite), I don’t want to do them. It’s not just that 0OI tests bore me (because they bore me to tears), but because I think it’s fiscally irresponsible to push verification of mind-numbingly basic functionality so far downstream. In my opinion, developers should do this work, but I guess if any testers find 0OI tests entertaining, I’m sure there’s a lazy developer somewhere who’d like to hire you as his cabin boy.

Short story is that 0OI tests are dead to me.

And then there’s 2OI work – the kind of work good testers love to do. 2OI testing is the land of “I wonder what happens when…”, and “How can I get the program to…” tests. 2OI tests can be manual or leverage tools or automation. Any form of discovery (or new knowledge acquisition) about the software is 2OI testing.

Testers not infatuated with Armour the way I am typically call 2OI testing exploratory testing (true for both manual or automated 2OI tests). In Armour’s recent article on testing that I mentioned in my previous post, he suggests that the sweet spot for 2OI (exploratory) effectiveness is about 50% – meaning that about half of your tests should discover product issues. If you’re finding less, you could vary your techniques, and if you’re finding more, you should back off the effort. I haven’t applied that concept yet when performing testing, but I plan to the next time I conduct a testing session.

But, the way I see it, exploratory testing is dead too. OK – that statement was overly controversial. What’s dead to me, is the term exploratory testing. To me, it’s just testing. It’s all just testing. I wrote a blog post once about this subject (see last three paragraphs to avoid most of my snarkiness) – then one of the top practitioners of ET wrote this post with a similar stance – but it seems every tester in the world is still compelled to proclaim they are “exploratory testers”, because they like to use their brain while they’re testing. All of this is fine, of course, one can use whatever terms they like, but to me, at least, it’s all just testing.

Testers do some of it. Others do some more of it. Some of it’s dead to me, and perhaps none of it is dead to you. It doesn’t matter.

It’s all just testing.

…the paradox of testing: We want to find defects in the software under test, but we do not really want to find them".”Phillip Armour

I often speak of Phillip Amour’s Five Orders of Ignoranceas they relate to software engineering (note; as of this writing, that link doesn’t work, but I don’t have another link to his paper). I’ve mentioned the orders of ignorance on this blog, and also in this article for Better Software.

Although I respect my readers, I know you won’t read those links, so let me briefly cover the orders of ignorance.

0OI – Zero-Oh-I is lack of ignorance. It’s when you know something. I know, for example, that the first track on the Rolling Stone’s Sticky Fingers album is Brown Sugar.

1OI – lack of knowledge. I don’t know (or remember) what the track is following Brown Sugar, but I could find the answer quickly.

2OI – lack of awareness. You have 2OI when you don’t know what you don’t know. I know that there are Stones tunes that I’ve never heard before, but it would be impossible for me to make a list of Stones songs I’ve never heard before.

30I – lack of process. You have 3OI when you don’t have a suitable method for discovering 2OI (for discovering what you don’t know you don’t know).

4OI – meta-ignorance. You have 4OI when you don’t know about the five levels of ignorance.

In the Better Software article, I attempted to stress how important the levels of ignorance are to software testers and software testing, and I frequently bring them up in conversations about testing. Recently, my mind was blown when Phillip Armour wrote an article specifically about the application of the levels of ignorance to software testing. A direct link (for those who have access to the communications of the ACM) is here. If not, I found a link here which should get you to the article.

And for those of you who don’t click through, here’s a summary.

When we test to ensure requirements were implemented, or the stories function as expected, we are testing for 0OI. We know what pass and fail looks like, and we can create tests accordingly. The 0OI testing set may be large, but it’s bounded. A passing 0OI test doesn’t expose any new knowledge – it just proves what we thought we already knew.

Testers also perform a lot of 2OI testing. We explore or run dynamic tests on a system to help discover what we don’t know we don’t know. Testing for 2OI is an unbounded test set – the things that a system might do is infinite. This is where we need to devise tests when we don’t know what we’re looking for. (Note – we don’t test for 1OI because if we truly knew in advance what we didn’t’ know, we’d resolve the ignorance and conduct 0OI testing.

Another (shorter) way to think about this is that 0OI tests are knowledge-proving tests, while 2OI tests are knowledge-acquiring tests.

Armour goes on to discuss the theoretical information content for a test and optimal failure rates (which I’ll leave to you to read, as I’ve probably already butchered the content already). It’s good stuff full of great quotes, and I hope you check it out.

Last Tuesday, I went to a pub near my house to watch Sounders FC win their third straight US Open Cup. For most of the game, there were just three of us watching the match, but it was still a lot of fun to watch.

With about five minutes left in the match (the Sounders were ahead 1-nil, and closing in on their second goal), a group of men came into the bar and filled 10-12 seats at a neighboring table. One of the men – I’ll call him Frank looked at the three of us watching the match, looked at the screen and claimed loudly, “That’s not a game!”. Apparently, that’s all of the commentary vocabulary Frank had, because he repeated the claim to his table-mates several more times during the remaining minutes of the match.

A part of me (perhaps the soccer hooligan in me?) wanted to confront Frank and ask him why he didn’t consider the most popular sport in the world to be a game, but I held my tongue and put my focus on watching the sounders score late to seal the game, and on their subsequent celebration.

But then I thought about Frank.I wondered what circumstances would lead to his statement. First off, soccer isn’t that popular in the US, so he’s certainly not the only one around who doesn’t value (what I see as) the beauty of the game. It also looked as if Frank and his friends came from a sporting event (it’s not softball season, but it could have been flag football). Perhaps Frank’s team lost, or he was frustrated with his performance. I don’t know many people who would purposely act as he did without some sort of other motivation. In short, there’s a lot going on with Frank I don’t know about, so I gave him the benefit of the doubt. Over time, I’m sure I’ll see Frank again, and as I get to know him (even solely via observation), I’ll get more insight into his motivations and know what drives his statements and social behavior.

This is how I try to behave at work. One of Microsoft’s core employee competencies is something called Interpersonal Awareness (which I prefer to call IPA). Most people think they’re pretty good at IPA because they get along with others, but that’s only a tiny scratch in the surface of IPA. In fact, I’ve stated on a few occasions that IPA is one of those skills that you’re not any good at until you realize that you’re not any good at it. When you think you have it down, you probably suck at it.

IPA is more than playing nice with others. It’s more than paying attention to how you come across. It’s reading people (including yourself), and adjusting to how you’re coming across based on reactions and words, while taking into account whatever may be motivating the people you’re interacting with – in real time. Yeah – I suck at it too, but that doesn’t mean I don’t work on it.

Let me try to bring it home. Think of your favorite “stupid” manager, “dumb” tester, or “egotistical” freak. Chances are, that those people are neither, stupid, dumb, nor egotistical. They have, however, acted that way and it caused you to react. There are two things going on here – you’re reaction and their behavior. Take a moment to think about why you had a negative reaction to whatever they did or said. Does it create more work for you? Does it violate your values? There’s a reason, and it’s worth it for you to take the time to understand why you reacted the way you did.

Now dig into the “stupid” manager’s statement. Perhaps he asked you to increase your code coverage percentage to some arbitrary number, and you think (rightfully) that increasing code coverage just to hit a number is s dumb idea. But first, take some time to think about why Mr. Pointy-Hair would have asked you that. Do you think his manager asked him? Do you think he thinks the number means something. Do you think he’s under any pressure himself that may have triggered him to ask, or is it something you think he’s valued in the past? Chances are, that Mr. PH isn’t stupid (despite his request). But until you know what motivated him to make the request, you can’t actually give him what he needs vs. what he (thinks) he wants. You probably won’t guess motivations right away, but without practice, you’ll never figure it out.

Think about this the next time you read (or say) that something is dumb, stupid, idiotic, or even careless. I may be overly optimistic, but I don’t think there are nearly as many idiots in the world as some would lead you to believe. But smart people say dumb things all the time – and without understanding the motivation behind their statements, we can’t grow – as professionals, or as humans.