Some random stuff

I have some random stuff to share – most of it not worthy of a post, so I’m consolidating.

  • I’ll be heading to Toulouse in a few weeks for a bit of a vacation. Looking forward to the time off, as well as food, wine, and warm weather. At one point in my life, I was almost adequate in speaking a very broken form of French (one that only included present tense verbs and lot’s of food terms) – but I’m pretty sure I’ve forgotten even that little bit of the language these days. Hopefully enough will come back before I embarrass myself too much.
  • My “new” job (I’ve been here 4 months already) is going great. I’ve had some pretty good success with exploratory testing on the team lately, and it’s an effort that a lot of people are seeing value with. I’m also working on some stuff with code coverage, test code quality and code reviews that are keeping me busy.
  • Buried in this post from just over 4 months ago was a comment about my rising blood pressure. I’m happy to say that as of this morning (after a visit to my doc), my bp has returned to it’s historic norms (actually, a bit better – 110/66). Better yet, I mentally and physically feel better these days (and this is before my vacation).

I have a follow up on my SDET Pendulum post drafted, and will try to get it up by the end of the weekend (for those of you who like my longer posts).


I’d like to announce my candidacy for…no – not that kind of politics, I thought I’d drop a few comments on office politics.

Too often, people see “politics” as the evil underbelly of the corporate world, and that competency in office politics requires that you can backstab your coworkers with no remorse, find a variety of ways to undermine your colleagues, and take credit for the work of anyone who won’t fight your claims. This definition from covers it well:

to deal with people in an opportunistic, manipulative, or devious way, as for job advancement.

But to me, that’s a cynical, shallow, and completely unhelpful view of what politics really are in the workplace. However, I do like the first four words – “To Deal With People” – in my happy rose colored view, that’s the part of the politic game that everyone has to play. I think that long term success in any workplace requires that you can deal with people. If you want influence, you’re going to need to have allies. If you attempt a difficult change, you may also annoy or disappoint some people. That’s all fine, as long as you find a way to talk, listen, or do whatever it takes to keep the wheels moving. I’m not saying that you have to kiss everyone’s ass, but if you want to be a leader – someone who can influence people to change, improve, or flat out take a chance on something that you want to do, you’re going to need to find a way to deal with them.

There may be people in the world that you don’t like or respect. That’s ok –we all know those people. Those of you who are politically savvy know that you probably still need to be able to communicate with them occasionally, so you don’t burn bridges. Who knows – you may need those people as an ally someday, and if you flipped the bit on them (and told them that you flipped the bit as well), all I can say is good luck getting that next “big idea” to fly.

More importantly, you need to be able to influence the people who can help you. Chances are good that you have more great ideas than you can take care of yourself. That’s ok, because great leaders take pride in making those around them better, and your great ideas are probably the thing that will make all of your colleagues better.

As long as they’ll listen to you.

And they will listen to you, as long as you are worth listening to. So build credibility, help them in their own projects, give them credit, and raise them up any way you can. In other words, use politics (the good kind) to get your way…and become a leader.

The SDET Pendulum

At a recent internal forum, I hosted a panel discussion comprised of “senior” level testers at Microsoft. The panel was evenly split between managers and non-managers and we took random questions from the audience on career paths in test. The testers in the panel had experience ranging from 11 to 24 years in test. Some of the questions strayed slightly from the topic (e.g. “What does the future of test look like to you?”), but for the most part went well. The session was one hour, but could have held it’s own for at least another 30 minutes.

That’s pretty much it, so I suppose I can end this post here…

But one question came up that I want to dig into a bit, and I know that some of you who occasionally read this blog have some opinions on the subject, so fire them away please.

The question in mind may as well have been seeded by me, because it’s a subject I’ve brought up before. The question was: “Has Microsoft swung the pendulum too far in hiring only programmers as testers?” I can’t remember if my answer was “No, but Yes”, or “Yes, but No”, or “It depends”, but I’ll give a (much) longer version here for your benefit.

BTW – I would bet that if I asked most non-microsoftees the response they would give for this question, they’d say, “of course – my {cousin | dad | sister | friend } who is the best tester in the {company | city | country | world } can’t get a job there, and they’re {fantastic | awesome | superman }. They just can’t write code.”

And, in many cases, they may be right – but maybe not – or it depends. Some context is in order. I’m not the company expert on this, but my opinion and view of history is as good as any.

Our software is big and complex. We deliver platforms, operating systems, server products and put them into huge deployments. We need testers who can program to help solve the big testing problems that our systems present. Say, for example, you’re a tester on the Exchange team, and you need to verify that messages of varying lengths travel from one endpoint to another successfully. No problem – send mail to a test account and make sure it gets there… well…that solves one happy path. The reality is that the tester needs to send multiple mails through the system with a variety of headers and sizes to ensure that anytime the message is broken up that it still ends up at point B correctly. And, going through outlook is probably not the most efficient way to test this, so it’s probably better to construct the messages programmatically then send them through the system. Oh yeah – there are an almost infinite number of ways the server topology can be constructed, and those settings make a difference too (in other words, we need to deploy any arbitrary topology automatically on demand for these tests). As you can see, there are some obvious benefits to having someone who can program as a tester.

Microsoft has always had testers that can write code. Always. For much of our history, we’ve also employed testers who don’t write code (but who are still good testers). We’ve found over the years that our best testers and best test leaders had a programming or CS background. I’m guessing we also found that defining the career path (MS is really, really big on career paths) for testers was a bit easer to map out when we took programming background and application into account. We still use a lot of non-programmer testers in contract and vendor roles, and I’m guessing we’ll continue to do so. There are many great testers who have learned how to program, but over time we found (or I should say, my personal experience was) that when we fished in the pool of programmers for good testers, we found a much higher percentage of good people than when we fished in the pool of good testers looking for people who could write programs. So, for full-time “blue-badge” employees, we fish for testers where programmers live; we get what we want, and everyone is happy.


Note that in the above I referred to testers who can program. Where I fear we make the mistake in hiring is when we (or anyone else who hires “SDETs”) hire programmers to do testing. The difference is subtle, but important. A tester who can program is a tester first – but they are a tester who can rely on computer knowledge or programming skills to do their work better. A programmer who tests writes tools and automation first – hoping that it will help them be a better tester (hint – it doesn’t work very often). For the most part, Microsoft has testers who can program – and they are freakin’ awesome. The problems they solve and the way they go about it makes me proud, excited, happy, (and a bit jealous) all at once. It’s what SDETs should do.

The industry perception of the SDET appears to be a role of writing automation and tools all day. When I hear this, I think of all the great testers I work with and tell people “No, No – just because our testers can program doesn’t mean that’s all they do all day – they are testers first”. Then, whether it’s in 5 minutes or 5 days (but sometime soon after), I’ll see a blog or a tweet or some other sort of message from someone saying “I’m an SDET – I write automation and tools all day”, and I realize that there’s still some work to do. It doesn’t mean that most of our SDETs write tools and automation all day, but there are certainly pockets. My thoughts are that if you want to write tools and code all day, get out of my business – go be a developer. You’re not helping me (and you’re probably slowing me down), so leave me alone. I want to test, and I want to work along side people who feel the same way.

In the panel discussion I asked every hiring manager in the room to ensure that in the next interview they conducted that they looked for testers who used “technical” skills to solve their problems. Instead of making them write code on the whiteboard, give them hypothetical testing tasks and examine how and when they used the power of the computer to help them. For example, if I ask an interview candidate how he would know if a line drawn on the screen was straight and he started writing up an algorithm to scan the pixels on the screen and determine if the offsets were consistent with a “straight” line, I wouldn’t hire him (“Look at it”, or “use a straight edge while looking at it” are both acceptable answers). However, if I describe the message transport scenario I used a few paragraphs above and their answer is to ask everyone in the company to send a bunch of mail, I probably won’t hire you either. (I’m working on an internal presentation on hiring testers – I’ll come up with better examples :})

Every business is different, and your need for testers or “SDETs” will depend on your business. My call to action is this: If you wan to hire an SDET – hire a tester who can program – not the other way around”. You and your product will be much better off.

It’s a big world (of testing) out there

A few weeks ago, I talked about some presentations I was involved with for internal audiences at Microsoft. Joe Strazzere asked if I could share slides or elaborate, so here goes (on one topic at least).

I’ve never met Joe – but I feel like I know him a little. I interact with testers regularly that I’ve never met. I take time every day to see what testers are doing, what they’re trying, and what they’re thinking. The problem is, that if you’re reading this, you’re probably the same as me. You look beyond the walls of your job to discover new ideas about testing. The talk I helped with (Joshua Williams actually ran the talk) was about looking outside the walls of Microsoft (or wherever you work) to discover what other testers were doing and what they cared about. The best testers I know have a passion for learning – but more importantly, they seek out new ways of learning, rather than just reading one source, or following the work of one test blogger or consultant.

So – I gave them options. I talked about the testing stack exchange, software testing club, test republic, and sqaforums. I talked about Software Test & Performance…I mean Software Test & Quality Assurance magazine and Better Software. We have a license for the IEEE Explore site, and I unveiled the world of academic papers on testing (and above all stressed the importance of critical thinking when attempting to suck down this info). I tell a story sometimes about the first testing book I read. It opened my eyes – after reading it, I felt like I knew everything (I was too naive to challenge it). I read a second book on testing and noticed that it contradicted the first book sometimes. It wasn’t until I read my third (and fourth, fifth, sixth and more) that I began to form my own opinions about testing. Forming your own opinions – rather than just repeating what others say, is something I value tremendously in testers I work with. I also gave a nice plug to weekend testing, as the movement is all about learning and testing.

We also talked about open source. We’re not big adopters of oss at Microsoft, so a lot of people don’t know much about what’s out there. I said what I could without pissing off our lawyers too much.

That was pretty much it. There were some good tangents in the discussion section. Somebody was disappointed that we just didn’t tell them what was happening in the industry (you’re missing the point), but I hope we at least opened a few people’s eyes to the world of testing.

If you’re reading this, this post probably isn’t for you – but get your coworkers (just the one’s you care about should be fine) to look beyond their little part of the world to learn. It’s a big world of testing out there.

Why I Write and Speak

I’m planning to give an internal presentation this week on what I do. I’ve been in my new job for three months, and although I’m still learning, I have enough of a routine that I’d like to share it with a few peers across the company to get feedback and ideas. Most of what I do is my job (that’s probably important, right?). I also still do a lot of cross company stuff (that’s probably pretty important too!).

With my remaining hours, I do “other stuff about testing” (I’ve been thinking about calling this “Project OSAT”, but it’s probably better to just call it “other stuff”). This includes writing books, contributing to other books, writing articles and blog posts, interacting on twitter, and participating in the overall testing community as much as I can find time for.

Sometimes, colleagues ask why I write, or why I speak at conferences. It’s not fame, and it’s certainly not for the money. It’s for ONE reason, and probably because of my lack of sleep due to 4:30am world cup matches, I’m going to share my motivation with everyone.

I write and speak because I’m lazy and cheap.

I suppose I should explain…

How We Test Software At Microsoft was written for one main reason. I talk to a LOT of people and companies about testing – many want to know Microsoft’s general approach to testing. I gave many of these companies the same answers. Over, and over and over again. Finally, I decided (with the help of a few colleagues) to write it all down and sell it for 25 bucks a pop (street price). There was just no way I could keep my sanity and continue to tell people about what testers do at Microsoft. Most of the articles and blog posts I’ve written have been for the same reason – I’m too lazy to give the same answer to multiple people, so when I start hearing the same question too many times, I write it down somewhere (NOTE: this is often also a good heuristic for automation).

Let’s not forget cheap. I hate paying to attend conferences (even if it’s technically my employer’s money). I do like to meet with other testers and talk shop (aka Project OSAT), but if I can get into the conference for free, I’m all over it (sometimes I even manage to get flights or more covered). So I submit proposals to a few conferences a year, and I typically get asked to speak at a few more. I do try to do a good job when I’m there (despite my laziness, I have a high bar for personal quality), but I go to conferences mostly because I get in for free.

Just because I’m lazy and cheap doesn’t mean I don’t care. I love writing, and I love speaking about testing and sharing what I’ve learned. I have worked, and continue to work extremely hard at both of these endeavors (see the note about my high bar for personal quality above). In fact, I’ve never taken money for any writing or speaking (some of probably think this is dumb, but it’s true). Way back when I first started writing for Better Software, I did this because not getting paid was easier than trying to decipher the company moonlighting policy. These days, I’ve found that I actually can get paid for most of this stuff, but I still don’t bother with it. When I consider how much I enjoy writing and speaking, doing it for free seems like the right thing to do (as long as you give me something free too – I’m still cheap).

This week’s conference

Every year, Microsoft’s Engineering Excellence group holds an internal conference – five days of talks, panels, booths and receptions (in other words, it’s just like any other conference, except only Microsoftees are invited). This is the first year since 2004 where I’ve been at the conference and not in the EE group. You’d think that it would be a new experience, but somehow I managed to get myself signed up to present in more sessions than I’ve ever been involved in at this event.

But first, on Monday morning, as the opening keynotes were just ending, I was talking to a group of brand new Microsoft employees. I speak at Microsoft’s new employee orientation fairly often, and it’s something I really enjoy. In the afternoon, I was involved in two sessions. The first was called something like “what testers at MS should know about testing outside of MS”. I think a lot of testers in the industry think that all there is to know about testing and software development exists in their itty bitty tiny corner of the world. Because they never look outside of their little corner, they never realize the truth (if you’re reading this, this probably isn’t referring to you). A colleague of mine kicked off the discussion, and I handled the middle section on communities. I talked about many of the forum and discussion sites currently available, including a shout-out to weekend testers. The session closed with a discussion of non-MS testing tools, specifically in the open-source area.

The second talk was a panel discussion on “The Reality of Careers in Test”. I thought the talk had a good setup, when the “baby” of the panel announced that he had been testing for only 11 years (the range went to 24 years). We had just worked the q&a into a groove when it was time to wrap up – it would have been nice to have a longer session, but it went well.

I hosted a two hour session on Tuesday. We took a deep look at one class of tools used by testers at Microsoft and had panels of owners discuss features and answer questions. One sort of cool thing we tried was to display a feed of questions and comments from (an internal twitter-like tool) during the discussion section. I thought it worked out pretty well and will have to think about how I can do something similar at an external conference someday.

In a session today, I hosted a sixty minute roundtable discussion on test careers. The discussions went well (from my point of view at least), and it was fun. Tomorrow I’m invited to a “special leadership” session (no presenting, just listening). I’m going to try to attend the whole session, but I’m probably going to have to break to spend some time on my “day job”.

I’d love to say that next year I won’t give any talks, but I’m pretty sure that’s not going to happen. I like to take what I know, combine it with my experiences, and try to help people learn something new or get better at what they’re already doing, so I guess there’s no avoiding it for me.

But that’s ok!.

Time Flies

I wrote a bit of a career retrospective just a few months before I joined the Communicator team at Microsoft (the series is captured in these posts: (part 1 is here, part 2 is here, part 3 is here, part 4 is here, and part 5 is here).

June 6, 1995 – exactly 15 years ago today – was my very first day at Microsoft. No need to retrospect again, but wow – what a variety of experiences I’ve had. I never thought I’d work at Microsoft this long, but I have no regrets.

Here are some non-traditional stats I thought I’d share.

  • Number of offices:15
  • Number of buildings I’ve worked in: 7 (Buildings 4, 10, 27, 118, 21, “Westpark”, and 30)
  • Most number of offices in one building: 5 (building 27)
  • Number of managers: 12 (including one manager for nearly 6 years)
    • Note: two of those managers lasted only a week
  • Number of projects I’ve worked on that were cancelled by BillG: 1 (in 1999 or so, I was part of a small virtual team working on some digitial tv stuff)
  • Number of programming and script languages I’ve learned 9 (in mostly chronological order: Visual Test, C, C++, Perl, VBScript, x86 Assembly, C#, Python (I’ve used all of these for testing except Python)
  • Number of shipping products I’ve worked on: 11 (not bad, considering that I spent 5 years in EE not working on products)


  • Number of employees at MS when I started: ~7500 (compared with ~90k today)
  • Number of those 7500 still at MS: ~4500
  • Number of those 4500 with the same start date as mine: 30

There’s probably more, but that’s all that comes to the top of my mind. Not bad for only 15 years.

It must be simple

Many of you know that I’m a big fan of the five orders of ignorance (link is to my interpretation – the author’s original paper is here). I especially like the concept of “unknown unknowns” those bits of knowledge that are critical – despite the fact that we didn’t recognize those bits until they fell onto our laps.

I’ve been pondering over an interesting fuzzy line between Armour’s second level of ignorance (you know you don’t know something), and his third level of ignorance (you don’t know you don’t know). I’ve noticed that often, when you know nothing about a subject, you perceive it as simple. Then, as you learn more, it seems impossibly difficult; and then, as you master it, it become’s easier to manage (but probably never as simple as you imagined in the first place). The second and third levels of ignorance seem to play with each other as you try to move knowledge into the first level of ignorance (you know what you know)

An example would probably help here. Many years ago, I knew nothing about recording music, but I wanted to be able to record my own stuff, overdub, mix, edit, etc. Since I had never done it before, I was convinced it was easy, and that anyone with a brain and a halfway working set of ears should be able to do it.Within a few weeks of attempting to pick up this “easy task”, I realized it was extremely complicated and required a lot of meticulous work (this was pre-digital, so within a few weeks I was fixing timing issues with a razor blade). I didn’t think I’d ever get a handle on it, but eventually I figured enough stuff out for my own purposes, and to this day I still uncover areas of complexity I never imagined.

There’s another great example of this from my list of favorite movies that came out when I was 19 – Better off Dead. There’s a scene where John Cusack’s character is on top of a mountain ready to ski down and his friend gives him this advice:

Go that way, really fast. If something gets in your way, turn

That’s exactly the “expert” advice someone who doesn’t know the subject would give. Steve Martin used to tell a similar joke about skiing – I don’t remember it exactly, but it went something like this:

Skiing – you go to the top of a mountain with slippery things on each foot and go down. Try not to – that would be a sport!

Perhaps a better example for testers is Jerry Weinberg’s Perfect Software book. To me, it’s not so much a testing book, but it’s a wonderful book to give to people who don’t know anything about testing – mainly because most people who don’t know about testing often think it’s a simple activity that has something to do with banging on keys or pushing buttons. Once you learn a bit about software testing, you of course realize that it’s far more complex than you ever could imagine. Eventually (if you’re lucky), you fall in love with the challenge and never look back.

I see at least a few examples of this phenomenon every day. I have lost track of the times I’ve heard people critiquing the efforts to plug the oil leak in the Gulf and offering their own advice. However, the fact that they haven’t been able to solve it, and have tried to fix it several ways must mean it’s probably a more complex problem than most of us understand (or that the efforts are run by aliens from the planet Stupid).

I suppose the moral of all this is that nothing is as simple as it appears – especially with knowledge acquisition. Keep it in mind the next time you think you know something …

Are you a game changer?

I took an internal training course last week where we heard lectures from a bunch of MS execs (I don’t do it justice – it was far more interesting than I anticipated and something I’d do again in a heartbeat).

Anyway, one of the lecturers was talking about software markets and showed a diagram that looked something like this:

Picture1The basic concept was that there’s a base set of functionality or features that a product has to have to compete, as well as areas where it needs to be better than (or at least as good as) competitors. “Hot” products also have something unique – a “game changer” that helps drive adoption.

With the iPhone for example, the foundation was that it had to make phone calls and play music.The touch controls, the camera and photo viewing and some of the other similar features made the iPhone competitive.

The apps, the appstore, and the “there’s an app for that” wildness is huge – I know dozens of people who use an iPhone only for the apps. That is a game changer.


Of course, this is a lot like Kano – same idea, different names.

imageKano has “must-haves” – the features you, well – must have. The “one-dimensional” properties in Kano are kind of like the differentiators in the chart above, and the “attractive” needs in Kano are sort of the game changers.

The thing I like about Kano is the way you plot the data. For each feature you’re considering, you ask customers (a focus group works great for this) the functional and dysfunctional form of their thoughts. For example, “How would you think of shimmery blue menus”, and “How would you feel if we didn’t have shimmery blue menus”. Their responses help you plot out where on the chart the particular feature lives.

Then it’s up to the decision makers to use the data and come up with the right mix of features that will satisfy customer needs – and give them enough “oomph” that they’ll want to give you money.


That’s all sort of interesting (or not), but I sat down here to write about something else. I’ve known about Kano for years, but when I saw the triangle version last week I had an insight.

The triangle also describes your career

If you want to be successful, at the very least you have to be able to do your job – that’s the foundation. Lot’s of people do just this and get by just fine. The next level (differentiation) is where you get good at your job – possibly even excel. There are a lot of people here too – these are your typical “great” performers. In many companies, these are the cream of the crop – and they should be.

Then, you have the game changers. There aren’t many of these, but they kick ass. They always have …something – something that only they bring to the table. These are the people that innovate and come up with the ideas that the differentiators use to get ahead (building, of course, on their execution and foundation of knowledge). Many people want to be game changers – some even proclaim they are game changers, but this level is reserved for people who earn it through great ideas and insanely hard work.

So how about you – do you want to be a game changer?

Network and influence

I gave a short talk to an internal MS community this week. The topic of the day was “influence”, and I thought it was appropriate to talk about the value of building (and maintaining!) an informal network – and the impact of that network on influence. The advice to have an informal network isn’t new – I bet it’s mentioned in the majority of books on leadership, but I don’t think enough people who want to be leaders (or be influential) take the point seriously enough. Your network is a resource for discovering new information as much sounding board,for ideas you want to share.

Your network is also an opportunity for you to build credibility. I know people who claim to be leaders. Sometimes they claim to be a leader because they’re in a position of authority. Other times, their “leadership” is just a self proclaimed act. In my world, you aren’t a leader until other people say you’re a leader. You don’t get it any other way than building credibility and trust among those you want to lead – and your network is a great place to build that credibility and trust across people who don’t have to listen to you if they don’t want to.

There are other values of a network. In HWTSAM, I talked about the Test Architect Group (TAG) at Microsoft. TAG is a collection of senior testers who meet regularly to talk about a variety of testing topics.

The value of having Microsoft’s most senior testers regularly review, brainstorm, and dissect solutions for complex test problems is immeasurable. In recent years, TAG has become something of a sounding board for new thinking, new methods, or new tools in testing. Presentations and demonstrations of ideas and implementations from test groups spanning every Microsoft division fill many of the meeting agendas. The value and depth of the feedback that the TAG provides is respected and sought after. A few meetings a year are reserved for “TAG business,” which includes discussions about company-wide initiatives driven by TAG
and other projects where TAG is a significant contributor (such as the MSDN Tester Center).

Perhaps the largest benefit of the regular meetings is in the value of networking. The extensive peer discussions and the view into the variety of work done across the company that is presented give TAG members much of the knowledge and information they need to make strategic decisions that affect the entire company.

I’ll say it again because it’s huge – the value of this particular tester community is the network. The fact that we talk to each other about what we’re working on gives us a small-world network – a huge reduction in degrees of separation between just about any testing knowledge in the company.

Yet – many people don’t take advantage of their opportunities to build their network. They don’t have the time – they think their day job is more important, or have something else to do they think is more important. It’s hard – last week I was in a great “hallway conversation”, but I found my mind wandering as I wondered if I should really be back in my office doing “real work” (I didn’t, now I’m behind, but it was worth it). Alas, some people who do consistently prioritize their commitment to building their network find that their superiors don’t share the priority and look down on work spent “outside of their core job” (my advice for these people is to find a new job).

I was asked once whether it was leaders who built great networks, or if people who built great networks became leaders. My old answer was that it was a toss up. Today, as I think about this more, I realize that there are a great number of leaders who got there because of their effort to build, maintain, and nurture a huge network of informal relationships. I’ve also seen leaders (including exec level leaders) fail because they thought that their position was enough, and didn’t work to build their network, credibility, and trust.