Author: Alan Page

For anyone who recently attended STAR East and saw my presentation, thank you for attending, and for reading my blog. Some of you have viewed the slides on the proceedings CD and noticed that it’s not really like what I presented. I tend to change quite a bit of material between the date the slides are due and the actual presentation (In fact, I added a new slide about an hour before the conference).

Here are the slides I presented last week. Stop Guessing How Customers Use Your Software

And here is the video I showed about what happens with Windows error reports.

It’s Thursday night, and I have just returned home from STAR East. Lee Copeland said that he thought it was the best STAR conference ever, and I know many who agreed with him. As for me, I had a great time, but was a bit too distracted from trying not to suck at my keynote that I couldn’t take it in as well as I hoped. I give a lot of presentations, and can’t remember the last time I was nervous before a presentation – but I was a bit nervous before my Wednesday afternoon keynote. I thought I had reasonable material, and I’m usually pretty good at keeping thoughts straight in my head, but I would be talking to a room filled with peers who I really respect. I’ve given keynotes before (not at STAR), and presented to large groups before, but if I messed up (or worse, if I was boring), I knew that there were people in the audience who would talk about me on the internet! I once gave a keynote a few years ago and I had the distinction of following Joel Spolsky (I’m fairly certain he was a lot better than me, but at least nobody tweeted aboutit).On top of that, Elisabeth Hendrickson gave what was probably the best presentation I’d ever seen on Wednesday morning in her keynote session, so I was feeling sort of intimidated,So, I tweaked a little more, and went through my slides a bit more than I usually do. Then I paced. I never pace… – but I did more laps than I’d like to admit around that ballroom while I got my act together.

Then, for the next fifty minutes, I told some stories and shared a few ideas with several hundred of my best friends in testing.

And no one booed!

But it’s better than that. Some of those same respected colleagues ( those responsible for my nervousness) told me that they liked the presentation – that was very nice to hear, and each of those comments were probably the highlight of my week.

But there were other highpoints from the week – I met at least half a dozen people that I knew from twitter and blogs, but had never met in person before – each and every one of these meetings was a memorable moment for me and more good memories. Joey McAllister and I had dinner Wednesday and talked about music (a part of my life that I don’t get to talk about enough), and I got to talk to Lanette and Matt a bit more about details of my recent job change. I signed a few copies of hwtsam, but also signed Beautiful Testing for the first time (and second, third, and fourth times too).

And now it’s back to reality of my day job. For the first time I can remember, I have no conference plans on my calendar (or any business travel for that matter). I finished a chapter for Dot Graham’s upcoming compilation last week, and have no other (external) writing projects lined up (except this blog). This is good, because I have a lot of really cool work on my plate, a great team and a pretty awesome product to ship. Fun stuff.

Some thoughts from me over on Inder Singh’s blog if you’re interested (in the article as well as in the comments).

More from STAR East later this week.

For some reason, I have a lot of friends from the UK (could be soccer, could be Microsoft – I don’t know). I hang around them enough to know not to look down when someone says “Pants”, and that “sixes and sevens” have nothing to do with math. Learning the subtleties took time, and I misinterpreted conversations often enough to be embarrassed (or at least confused). We speak the same (roughly) language, but important meaning is sometimes lost.

Back in my day job, I asked four testers I know what terms they would use to describe some terms for the following scenario:

Say you have a simple test (or check) that verifies that the xyz driver is loaded when a USB XYZ device is plugged into the system. And – you want to run this test on 32bit and 64bit Windows, and with at least four different USB firmware implementations.

So – you have something you want to verify – let’s call it an “idea”, and you want to confirm your idea on some different configurations – seems pretty straightforward.

1. What do you call the “idea” (plug in the device and make sure the driver loads)? Is that a test case?
2. What do you call the permutations? Do they have a different name or are they just more of #1 – i.e. do you still have one test case, or do you have eight?

I have my own ideas (and I wrote about them in hwtsam), but when I asked four testers this question, I came up with five answers (adding mine to the mix).

I worry (perhaps too much) that lack of common vocabulary causes problems in testing. My colleagues who are in favor of certification (or at least aren’t violently opposed to certification) say that the common vocabulary encouraged in the various Bodies of Knowledge are the best things about the certification. But – since there are a variety of certifications (and a larger variety of interpretations), I can’t say that I expect certification will solve the vocabulary problem. But I can’t help but wish that we could dissolve the language gap.

Then again, perhaps it’s not that big of a problem. We do find ways to talk, and we do make (some) progress. For example, I was in a twitter conversation last week (or as much of a conversation as you can have in 140 characters) – I said “The problem with blah is that x, y, and sometimes z” – and the response was “In my world, I call that funklestein”. OK – great – that’s not my word, but now we can have a conversation, and it wasn’t that hard to get us there. I wish all testing conversations were that easy.

 

I’m wrapping up a ton of stuff this week in order to prepare for being away at star east next week. I hope to get a chance to meet some of you there. Come find me.

I ran into a former colleague yesterday  – she mentioned that she had read several of my recent blog posts and that it seemed that I had something else to say – that I was hinting at something that I never quite said. I wasn’t aware of anything I had said (or not said), but I do know that I can be careful with my words sometimes, so I went through the last dozen or so of my blog posts, to see if I could find any loose ends or hints.

I didn’t see anything – but I recognize that I may note be able to see it in my own writing. So, if you have time, let me know what you think – do you think there’s something I’m not saying? Something between the lines? Something you want to hear?

I had a burst of traffic on my blog last week when 10’s of eager internet users stumbled across my little rant fest. I try to avoid traffic for traffic’s sake (past experiences tell me that I could merely comment half-heartedly on any of the sacred subjects of the testing world and generate a significant amount of comments and traffic (and hate mail) – but I’m too busy to deal with that). For the most part, I just like to (carefully) write what’s on my mind and see if anyone else can relate.

But I digress…

Raymond Chen included me in his quarterly linkfest last week. For those of you not familiar with Raymond, you may think “big whoop – a link”, but Raymond gets thousands of readers – dozens of which read the article on my site then left :}. A colleague also pointed out to me that a few folks on mini-msft’s blog were talking about me – I popped on, added some context and popped off. I had 60-ish referrals from that link, so apparently there are still a lot of people still reading mini’s stuff (which squashes my theory that the mini-msft commenters were comprised of 15 or so whiners who all think the world is out to get them).

And now things are back to normal (at least as far as my visitors per day trend goes). Welcome to any new regular readers, buy my book, leave a comment, blah, blah, blah.

I’ve been working on integrating a set of code scanning tools into our build system. Test code needs to be trustworthy – when test code fails, it should indicate a product failure. More importantly, when tests pass, they better indicate that the functionality under test works. One part of increasing quality is getting rid of the coding errors that are most prone to causing test failures. This includes a basic set of fxcop rules for managed code, similar rules for native code, and specific checks for hard coded paths, hardcoded passwords, and other easy to spot things that will eventually cause false positives.

The framework we use requires “hints” to find the reference libraries at compile time (basically, we run fxcop against every managed assembly at build time and track a full list of errors). The hints are just a pointer to the library in the project xml file. It’s a bit of a pain, but completely worth it in the end. I spent a few hours last week building a portion of the tree, reviewing the errors for the references the framework couldn’t find, adding them, then verifying the additions. Lather, rinse, repeat.

I made a ton of progress in those few hours, but stopped because it just didn’t seem efficient. I was doing other work during the builds, so I felt productive, but it just didn’t seem right. So, I stopped and did other work. As I look back on it, it wasn’t really even a conscious decision – I just sort of mentally reprioritized that task.

Today, for some reason, I decided to take another look. I thought about it for about two minutes and had a solution. It was so dirt simple that I’m embarrassed it wasn’t my first choice. Rather than build part of the source code, review the errors, find the binary, and add the hint, I took care of it all at once. When the framework couldn’t find a reference, it put an error in the log file something like this:

Could not find reference – Microsoft.Internal.Foo

It printed one of these for every reference it couldn’t find, so I just built everything – and while it was building, I wrote a script that would parse out the reference name, look for the binary on disk, normalize the paths, remove the duplicates (multiple projects will share references), then add the appropriate xml to the project file. Total time on task was about 20 minutes and it worked perfectly.

The point of all this is that I realize that I do stuff like this all the time. I often dig into a problem, pause (or leave), and then come up with an optimal solution. Part of me thinks that I should recognize the optimal solutions the first time, but another part of me wonders if I’d be able to come up with good solutions if I didn’t get my hands dirty first. I’ll pay more attention and let you know how it goes.

I think a true passion for learning is one of the things that makes testers great. The best testers I know are always looking for concepts or viewpoints they haven’t heard before and looking for opportunities to learn anytime they can.But it’s usually not enough to just to want to learn – you need some skills on how to learn – on how to learn what you don’t know you don’t know. You have to be able to do a lot more than type things into a search engine. If you want to learn about something, search engine results are just the first clue of many that will take you on the learning journey.

I’ve taken two classes in my life that I will forever be grateful for taking. The first was typing in 9th grade. I was awful in the class, but it’s amazing how the basics have helped my productivity throughout the years (just ask anyone who’s ever been annoyed with me for continuing to type while I carry on a conversation with them). The other class was a Methods of Research class in grad school. To be specific, the class was Mus 510: Methods of Musical Research. The class was entirely about how to find information…and it was intense. My favorite part of the class (and the most hated for many others in the class) was the weekly “trivial pursuit” exercise. Each week we were given a page of questions, and asked to find the answers (and note where we found them in a specified bibliographic style). This was 1992, so the internet wasn’t really cranking along enough to help. Instead, the university library was our playground.

So you can get an idea of what we went through, here are a sampling of some of the questions. There is no additional context – this is exactly – and only, what was written on our assignment sheet.

• Whose death was “bewailed” in Thomas Campion’s Songs of mourning in 1613? Where in this country will you find a complete copy?
• Who made two snare drums dated 1839 and 1841 now owned by the New York Historical Society?
• In what book will you find biographical notes on 53 song composers of India?
• Who first decided that A should be 440 vibrations per second?
• (and, my favorite…) What does “Gabriel” have between his teeth?

Every week we’d have ten or so of these, and every week I’d arrive at the library at 8:00am on Saturday when they opened, and wrap up just in time to leave at 5:00 when they closed. It was a long day, and it was mentally hard work, but I loved it. Little did I know that I was studying to be a software tester.

What I learned was how to dig deep. Start with whatever you think is the keyword and look there – then look for citations or references. If there aren’t any, look for unique phrases and search on those. During my eight hour Saturdays in the library, I learned that the first place you look for an answer is almost always only a clue for where you need to look next.

Let me put it another way – if you think you found the answer on your first try – or think you know what something is truly about because you read the wikipedia article or someone’s blog post, you will almost always be wrong. If you want to be a great tester, don’t settle for surface area answers – dig deep and pursue!

I’ve been in my new office for nearly a month, so I thought I’d share a tour.

For the most part, it’s a typical Microsoft office – about 100 square feet with a window, desk, door and …stuff. Here’s what you see when you walk in the door.

I have three computers. A HP Z400 drives the two monitors in the foreground, and a Dell 670 drives the remaining LCD. Both are dual proc Xeons (which appear to windows as 4 processors), the the HP is younger by about 5 years (it’s brand new) and builds about twice as fast. I use the Dell for mail, charging my phone, writing, and most everything else that isn’t build, analysis, or test related (although it can cover those tasks if the Z400 is in the middle of something. I also have my trusty Lenovo x200s as another backup. Most of the time it also functions as my primary “home” work machine.

I have a lot of books – here’s one of my bookshelves.

I could do a whole other post explaining where some of these came from. On the top shelf (next to several copies of hwtsam), are my “commemorative” copies of Windows 95, Windows 95 Plus Pack, and some other software I worked on. The other day I ran across some “gold” copies I have of Win95, and the Win95 SDK/DDK that were handed out to team members when we shipped (it’s just a regular CD printed on a gold colored disk in a jewel case).

Here’s a reverse view of my desk – pretty much the view I have now while I type on the Dell.

Note that the phone on the right doesn’t have any keys – it connects via USB cable and works with communicator. It’s the phone that rings when you call my office (assuming I’m logged into communicator on that machine).

Here’s a view of my other bookshelf.

Again, lot’s of stuff that I’ll explain if you ask.

Finally, here’s some crap piled near my window – at Microsoft, you tend to accumulate stuff that is important enough to keep, yet there’s no place to keep it at home.

In this picture are two “Ship-It” plaques (one is filled), my 10 year award plaque, a Windows 95 team commemorative clock, and a plaque containing the one patent I have filed (the block next to it is a patent “cube” for the same thing – you get that for successfully filing for the patent.

The one thing I probably miss the most from my old office is the lack of writable surface. Just about every vertical space (and one of our tables) in the old TestEx team room was a writable surface, and I got used to writing diagrams, notes, etc. everywhere. For now, I just have a little whiteboard, but I’ll try to see if I can get it replaced with something much larger at some point..

So – there you go. If you want something explained or have questions, let’s chat in the comment section (or on twitter if you prefer – @alanpage)

Today was my 14th (business) day on the communicator team. I’m into sort of a groove and having a good time. My job today is so drastically different than what I was doing just a few months ago that I thought I’d share a few things so everyone would know what I’m up to.

I’ve managed to initiate a few cool projects and get myself injected into a few more. One thing I really love about this team is the amount of trust my manager and teammates have in me and my ideas. Last week, for example, I was pointing out some perceived flaws in one of our processes – along with my ideas of how the processes should work.Thinking out loud, I said, “it would be cool to do a case study and gather some data on this.” I already have the details and participants lined up and should have the study done in the next week or so. It’s really exciting to see things more so fast.

Another principal sdet on the team and I are starting a program that will enable testers on the team to step away from the bulk of their day jobs for up to 4 weeks. This will enable testers to work on projects they may not get a chance to otherwise – and, of course, some work gets done that wouldn’t get done otherwise.

I’m also giving a series of brown bag talks on various testing topics. You all know that I’m a fan of the five orders of ignorance and this is a great opportunity for me to help people learn what they don’t know. I’m also heavily involved in the interviews for new team members – it’s fun to interview people for testing positions (in EE, I interviewed people to be teachers and advocates for test).

And finally, as a big proponent of test code quality, I’m unleashing a boatload of static analysis tools on the test tree. This is a nice project because it gives me a chance to do a cursory review of all of the test code in the tree – the end result will be higher quality tests and tools – tests and tools that can be trusted to provide accurate results. The topic of static analysis is worth another post someday – but I’ll wait until I’m done with this little project first.

I think that covers it – so far so good!