What am I really saying here?

I ran into a former colleague yesterday  – she mentioned that she had read several of my recent blog posts and that it seemed that I had something else to say – that I was hinting at something that I never quite said. I wasn’t aware of anything I had said (or not said), but I do know that I can be careful with my words sometimes, so I went through the last dozen or so of my blog posts, to see if I could find any loose ends or hints.

I didn’t see anything – but I recognize that I may note be able to see it in my own writing. So, if you have time, let me know what you think – do you think there’s something I’m not saying? Something between the lines? Something you want to hear?

Flashes of (near) fame

I had a burst of traffic on my blog last week when 10’s of eager internet users stumbled across my little rant fest. I try to avoid traffic for traffic’s sake (past experiences tell me that I could merely comment half-heartedly on any of the sacred subjects of the testing world and generate a significant amount of comments and traffic (and hate mail) – but I’m too busy to deal with that). For the most part, I just like to (carefully) write what’s on my mind and see if anyone else can relate.

But I digress…

Raymond Chen included me in his quarterly linkfest last week. For those of you not familiar with Raymond, you may think “big whoop – a link”, but Raymond gets thousands of readers – dozens of which read the article on my site then left :}. A colleague also pointed out to me that a few folks on mini-msft’s blog were talking about me – I popped on, added some context and popped off. I had 60-ish referrals from that link, so apparently there are still a lot of people still reading mini’s stuff (which squashes my theory that the mini-msft commenters were comprised of 15 or so whiners who all think the world is out to get them).

And now things are back to normal (at least as far as my visitors per day trend goes). Welcome to any new regular readers, buy my book, leave a comment, blah, blah, blah.

Getting it right the second time

I’ve been working on integrating a set of code scanning tools into our build system. Test code needs to be trustworthy – when test code fails, it should indicate a product failure. More importantly, when tests pass, they better indicate that the functionality under test works. One part of increasing quality is getting rid of the coding errors that are most prone to causing test failures. This includes a basic set of fxcop rules for managed code, similar rules for native code, and specific checks for hard coded paths, hardcoded passwords, and other easy to spot things that will eventually cause false positives.

The framework we use requires “hints” to find the reference libraries at compile time (basically, we run fxcop against every managed assembly at build time and track a full list of errors). The hints are just a pointer to the library in the project xml file. It’s a bit of a pain, but completely worth it in the end. I spent a few hours last week building a portion of the tree, reviewing the errors for the references the framework couldn’t find, adding them, then verifying the additions. Lather, rinse, repeat.

I made a ton of progress in those few hours, but stopped because it just didn’t seem efficient. I was doing other work during the builds, so I felt productive, but it just didn’t seem right. So, I stopped and did other work. As I look back on it, it wasn’t really even a conscious decision – I just sort of mentally reprioritized that task.

Today, for some reason, I decided to take another look. I thought about it for about two minutes and had a solution. It was so dirt simple that I’m embarrassed it wasn’t my first choice. Rather than build part of the source code, review the errors, find the binary, and add the hint, I took care of it all at once. When the framework couldn’t find a reference, it put an error in the log file something like this:

Could not find reference – Microsoft.Internal.Foo

It printed one of these for every reference it couldn’t find, so I just built everything – and while it was building, I wrote a script that would parse out the reference name, look for the binary on disk, normalize the paths, remove the duplicates (multiple projects will share references), then add the appropriate xml to the project file. Total time on task was about 20 minutes and it worked perfectly.

The point of all this is that I realize that I do stuff like this all the time. I often dig into a problem, pause (or leave), and then come up with an optimal solution. Part of me thinks that I should recognize the optimal solutions the first time, but another part of me wonders if I’d be able to come up with good solutions if I didn’t get my hands dirty first. I’ll pay more attention and let you know how it goes.

Trivial Pursuits

I think a true passion for learning is one of the things that makes testers great. The best testers I know are always looking for concepts or viewpoints they haven’t heard before and looking for opportunities to learn anytime they can.But it’s usually not enough to just to want to learn – you need some skills on how to learn – on how to learn what you don’t know you don’t know. You have to be able to do a lot more than type things into a search engine. If you want to learn about something, search engine results are just the first clue of many that will take you on the learning journey.

I’ve taken two classes in my life that I will forever be grateful for taking. The first was typing in 9th grade. I was awful in the class, but it’s amazing how the basics have helped my productivity throughout the years (just ask anyone who’s ever been annoyed with me for continuing to type while I carry on a conversation with them). The other class was a Methods of Research class in grad school. To be specific, the class was Mus 510: Methods of Musical Research. The class was entirely about how to find information…and it was intense. My favorite part of the class (and the most hated for many others in the class) was the weekly “trivial pursuit” exercise. Each week we were given a page of questions, and asked to find the answers (and note where we found them in a specified bibliographic style). This was 1992, so the internet wasn’t really cranking along enough to help. Instead, the university library was our playground.

So you can get an idea of what we went through, here are a sampling of some of the questions. There is no additional context – this is exactly – and only, what was written on our assignment sheet.

  • Whose death was “bewailed” in Thomas Campion’s Songs of mourning in 1613? Where in this country will you find a complete copy?
  • Who made two snare drums dated 1839 and 1841 now owned by the New York Historical Society?
  • In what book will you find biographical notes on 53 song composers of India?
  • Who first decided that A should be 440 vibrations per second?
  • (and, my favorite…) What does “Gabriel” have between his teeth?

Every week we’d have ten or so of these, and every week I’d arrive at the library at 8:00am on Saturday when they opened, and wrap up just in time to leave at 5:00 when they closed. It was a long day, and it was mentally hard work, but I loved it. Little did I know that I was studying to be a software tester.

What I learned was how to dig deep. Start with whatever you think is the keyword and look there – then look for citations or references. If there aren’t any, look for unique phrases and search on those. During my eight hour Saturdays in the library, I learned that the first place you look for an answer is almost always only a clue for where you need to look next.

Let me put it another way – if you think you found the answer on your first try – or think you know what something is truly about because you read the wikipedia article or someone’s blog post, you will almost always be wrong. If you want to be a great tester, don’t settle for surface area answers – dig deep and pursue!

A Tour of My Office

I’ve been in my new office for nearly a month, so I thought I’d share a tour.

For the most part, it’s a typical Microsoft office – about 100 square feet with a window, desk, door and …stuff. Here’s what you see when you walk in the door.

Front Door View

I have three computers. A HP Z400 drives the two monitors in the foreground, and a Dell 670 drives the remaining LCD. Both are dual proc Xeons (which appear to windows as 4 processors), the the HP is younger by about 5 years (it’s brand new) and builds about twice as fast. I use the Dell for mail, charging my phone, writing, and most everything else that isn’t build, analysis, or test related (although it can cover those tasks if the Z400 is in the middle of something. I also have my trusty Lenovo x200s as another backup. Most of the time it also functions as my primary “home” work machine.

I have a lot of books – here’s one of my bookshelves.


I could do a whole other post explaining where some of these came from. On the top shelf (next to several copies of hwtsam), are my “commemorative” copies of Windows 95, Windows 95 Plus Pack, and some other software I worked on. The other day I ran across some “gold” copies I have of Win95, and the Win95 SDK/DDK that were handed out to team members when we shipped (it’s just a regular CD printed on a gold colored disk in a jewel case).

Here’s a reverse view of my desk – pretty much the view I have now while I type on the Dell.


Note that the phone on the right doesn’t have any keys – it connects via USB cable and works with communicator. It’s the phone that rings when you call my office (assuming I’m logged into communicator on that machine).

Here’s a view of my other bookshelf.

another shelf

Again, lot’s of stuff that I’ll explain if you ask.

Finally, here’s some crap piled near my window – at Microsoft, you tend to accumulate stuff that is important enough to keep, yet there’s no place to keep it at home.


In this picture are two “Ship-It” plaques (one is filled), my 10 year award plaque, a Windows 95 team commemorative clock, and a plaque containing the one patent I have filed (the block next to it is a patent “cube” for the same thing – you get that for successfully filing for the patent.

The one thing I probably miss the most from my old office is the lack of writable surface. Just about every vertical space (and one of our tables) in the old TestEx team room was a writable surface, and I got used to writing diagrams, notes, etc. everywhere. For now, I just have a little whiteboard, but I’ll try to see if I can get it replaced with something much larger at some point..

So – there you go. If you want something explained or have questions, let’s chat in the comment section (or on twitter if you prefer – @alanpage)

Checking in on the new gig

Today was my 14th (business) day on the communicator team. I’m into sort of a groove and having a good time. My job today is so drastically different than what I was doing just a few months ago that I thought I’d share a few things so everyone would know what I’m up to.

I’ve managed to initiate a few cool projects and get myself injected into a few more. One thing I really love about this team is the amount of trust my manager and teammates have in me and my ideas. Last week, for example, I was pointing out some perceived flaws in one of our processes – along with my ideas of how the processes should work.Thinking out loud, I said, “it would be cool to do a case study and gather some data on this.” I already have the details and participants lined up and should have the study done in the next week or so. It’s really exciting to see things more so fast.

Another principal sdet on the team and I are starting a program that will enable testers on the team to step away from the bulk of their day jobs for up to 4 weeks. This will enable testers to work on projects they may not get a chance to otherwise – and, of course, some work gets done that wouldn’t get done otherwise.

I’m also giving a series of brown bag talks on various testing topics. You all know that I’m a fan of the five orders of ignorance and this is a great opportunity for me to help people learn what they don’t know. I’m also heavily involved in the interviews for new team members – it’s fun to interview people for testing positions (in EE, I interviewed people to be teachers and advocates for test).

And finally, as a big proponent of test code quality, I’m unleashing a boatload of static analysis tools on the test tree. This is a nice project because it gives me a chance to do a cursory review of all of the test code in the tree – the end result will be higher quality tests and tools – tests and tools that can be trusted to provide accurate results. The topic of static analysis is worth another post someday – but I’ll wait until I’m done with this little project first.

I think that covers it – so far so good!

Testing for Brown M&Ms

One thing I’ve learned in my testing career is that where there are bugs, there are more bugs. Some of my colleague Nachi’s research shows that components with a high number of pre-release bugs has a corresponding share of the post-release bugs. The heuristic works well on a smaller scale too – if I find annoyances or other slop 10 minutes into a testing session, odds are that there are many, many more bugs in the feature.

This post by Ajay demonstrates a good example of “slop”. There are certain things anyone designing a kiosk application should do to ensure that the application remains in kiosk mode (sidenote: as a tester, I love kiosk machines because they’re a great target to demonstrate testing ideas). When I see flaws like this, I know there are more – perhaps dozens or even hundreds of other flaws in the application, and I wonder how many of those flaws are exploitable for more serious damage.

And now, as I read the comments, I see that JB said basically the same thing in the first comment.

And then I was reminded of this article I read last week about business advice from Van Halen. Van Halen were famous for the rider in their contract stating that they should have a bowl of M&Ms with all of the brown candies removed. It seems like an arrogant rock star tactic, but there was something significant in the request.David Lee Roth knew that if the M&M request wasn’t handled correctly that there were likely other problems in the venue adhering to the contract. Brown M&Ms in the bowl indicated that a deeper look at the logistics of the event were in order.

In other words, Roth was no diva. He was an operations expert. He couldn’t spend hours every night checking the amperage of each socket. He needed a way to assess quickly whether the stagehands at each venue were paying attention — whether they had read every word of the contract and taken it seriously. In Roth’s world, a brown M&M was the canary in the coal mine.

I’m sure there are hundreds of examples of the “brown M&M heuristic” – now I just have to see if I can remember to call it that.


*I have no idea why I decided to use the word “slop” today to describe annoying behavior, so don’t ask.

Automation & Test Cases

I’ve come to realize that my answers to life and the universe may vary from others. Sometimes I fall in love with my answers, but I’m nearly always quite open to ideas that differ from mine. In fact, I love the conflict in thought because it makes me re-think the original problem.

One such example is in my mind. Slightly mentioned in HWTSAM and described in detail in my chapter of  Beautiful Testing is my approach to automation. When I write automation, I don’t think of test cases first. I may use a spec, notes from a spec, or just lot’s of notes in general to help me design test automation. I explore the component I’m testing– either manually, or in the debugger and create the tests as I develop the automation (skipping lot’s of design stuff because that’s not the point of this post). When I’m done, I prefer an approach where I extract the test cases from the test automation (either via c# attributes or export names or some other form of tagging meta data) and import theses test cases into the test case management system.

An alternate approach is to write the test cases first, import them into the TCM, then write the automation (optionally creating stubs automatically from the test cases). At first, I thought I didn’t like the idea at all, but I suppose it does force you to think about the details of testing up front (if that’s a good thing). One con that I haven’t resolved is what to do when the automation changes. With my approach, you just re-import the metadata. If you change the automation using the alternate approach, you probably need to update the TCM manually – but I suppose you could find a way to make that easy.

How about you – do you prefer one approach over the other or have an alternate approach? 

The truth about meetings

I’ve been talking about meetings recently at work (I’m still giddy that have far fewer meetings to attend these days). I’ve had discussions about meetings for at least a decade, and they’re all about the same.People grumble about ineffective meetings, while others grumble about too many meetings. Some grumble about who should be in the meeting, while others grumble about the meeting they should have been invited to.

imageHere’s the deal (according to me at least). Meetings are necessary and a good use of time (as long as you don’t screw them up). Try as you might, you can’t get a team aligned as well or as efficiently over email. Some problems (and solutions) won’t be put on the table in email or over IM. As introverted as many software people are, sometimes you need to talk to other humans to make the most efficient progress.

Think, for a moment, what a good meeting looks (and feels like). Good meetings are engaging – you want to listen and participate (rather than type on your laptop or phone). In good meetings, you learn something new (or help someone else learn something) or you reach agreement and have a plan of action by the time you walk out the door). After a good meeting you should have a feeling that progress has been made.

Then, there are the other kinds. We’ve all been there – half the room is typing on laptops, looking up once in a while to ask the presenter to repeat the last five minutes of their presentation. People talk, but no decisions are made, and most people are more confused about what’s going on than when they walked in the door. When you leave, you don’t know what to do, or at best, you are assigned to go back to your office and come up with a plan of action on your own.

We can all recognize a good meeting vs. a bad meeting, but how do you change one into the other? You have few options. One is to cancel all of the “bad” meetings. Then, you’ll only have good meetings left and life will be great. The problem is that you may actually need most of those meetings – you just need them to be more effective. Another option is to ensure that each meeting has a flow (I didn’t say “agenda” on purpose – I believe that meetings with out agendas can be effective if you have a consistent way to let the agenda emerge (scrum daily stand-ups are a great example of this (unless you consider an “everyone talks about their stuff” an agenda)). Also make sure that every meeting has a facilitator (who may or may not be the owner of the meeting). The facilitator’s job is to keep the meeting on track, stop “rat hole” conversations, and to assign action items as they come up.

Another suggestion is to reduce the meeting length. My hunch is that most 60 minute meetings can be done in 30 minutes, and 90 minute meetings can be completed in 45 (or less)). Meetings are gaseous entities that expand to fit the available space, and I believe that if you reduce the meeting time, that within a few sessions you will be getting just as much done (in half the time). If it works for you, please send your checks to me – I accept paypal.

More collateral for you on the subject

Making Time

Yesterday, a colleague asked me where I find the time to blog, twitter, etc. This is something I get asked often, but the only answer I have is that I just make time. I put blogging, presentations, sasqag work, and other community stuff right alongside my core work on my todo list. I’m heavily driven by my todo list (I use the tasklist in outlook), and anything I think is important makes it onto the list. I don’t work exceptionally long hours – I just make time for what I think is important. I don’t work exceptionally long hours either – my home life and my work life sort of blend together (I take care of personal stuff during the work day, and work most mornings and evenings from home. I’d estimate that I typically work about 50 hours a week at the most (although that number is a bit higher now as I ramp up on my new job).

I suppose it also helps that I think about work a lot. In fact, I write most of my blog posts (including this one) in my head before I sit down to type them out. I use my commute (20 minutes) to think through blogs, papers, projects, and whatever else I’m doing that requires a bit of thought before execution.I don’t have a lot of dead time during my day, but I think that helps keep me sharp.

I have one other bit of related advice to share (something I should have included in my gravy train presentation). There is always enough time. I’ve talked to people around the world (mostly testers), and I’m surprised how many tell me that there is some cool thing they want to do…but they just don’t have enough time. “I’d like to learn a new language,,,but I don’t have time”, or “I’d like to look ahead and build a strategy for our team…but we’re just too busy with execution right now”. When people tell me this, I ask- “What would happen if you were sick for a day? So sick you couldn’t even get out of bed?” “Would you be able to “catch up?” Sometimes they try to convince me the world would end, but the conversation always ends with a statement like “it would be inconvenient, but we’d survive”. “Fine”, I say, “if this is important, block some time off of your calendar and make it happen”. You will never get off of the treadmill if you don’t make that time, and if you don’t get off the treadmill, you’re never going to make great things happen.

And everyone should get a chance to make great things happen.

ed. Adam Goucher pointed me to this article on making time. A fantastic read – thanks Adam.