Archive for the ‘HWTSAM’ category

Tester DNA

August 28th, 2010

In HWTSAM, we (Ken, actually) talked a bit about tester DNA – that bit of mental goo that makes some people better (or at least more prone to being) testers. As I’ve been talking to (potential) testers lately, I’ve had a chance to dwell on this a bit more. What is it that makes a good tester? Given that the answer to that question requires context, I won’t answer that exactly – instead, I thought I’d share some of my thoughts on what I look for in testers.

Testing is broad and evolving – nobody knows everything about it, so the ability to learn quickly is critical (this aids in problem solving as well). If you’re the type that takes a long time to ramp up in new technologies or concepts, you may struggle as a tester. Probably more critical is a passion for learning. Good testers don’t wait for ideas to come to them, instead, they seek out knowledge  – they not only learn what they know they need to learn, they find ways to learn what they don’t know (i.e. they strive to resolve second level ignorance). I believe that the big innovations in testing will come from applying knowledge from outside the field of software and software testing. In order to advance the state of the art in testing, we need testers who seek knowledge – and who are able to apply those abstract concepts to solve some of our big problems in test.

But – to take care of that last sentence, you’re going to need people who can see the big picture – systems thinkers. There are numerous people who claim to be systems thinkers, but systems thinking takes practice as well as some innate ability (or DNA) to be beneficial – and it’s much harder than many people think. Often when I interview testers, I ask a “testing” question that has two parts to solve. The first (the question I actually ask them) is obvious and has a solution that is difficult enough that they solve it as they would any other question. However – there’s a hidden problem in the question. The good testers quickly see the secondary problem as the far more difficult problem to solve and focus their answer on solving the underlying problem. These are the systems thinkers – they know to look at the whole rather than the parts and know that understanding interactions and patterns are keys to good problem solving. The great testers – and there are only a few of these – can actually solve the problem reasonably well (frankly, I worry about testers recognizing the problem more than solving it, but I’m frequently impressed by testers who nail every aspect of this question).

nitpickers moment – for those of you who will take this opportunity to gripe about SDETs, no part of solving this question relies on programming skills. It does require that you can think and see beyond the obvious. I’m not going to put the question on my blog, because I still want to use it. I would be happy to discuss it with you privately (or via an IM session) if you’re insanely curious.

There are numerous other skills I look for in testers, but I consider those to be supportive and of the “more is better” category. For instance, if you are completely disorganized, you may not be successful, but you don’t have to be the most anal note taker either. You need some degree of organization, self motivation, confidence, and trust to be successful, but those really only show up on my radar if you truly suck at them.

As I re-read this post, I realize that the things I mentioned above are also the things that make testers successful in the long term – so it makes sense that’s what I look for when hiring testers. And I think that’s good!

3 comments »

Posted in HWTSAM, Software Testing, Test Careers

Why I Write and Speak

June 13th, 2010

I’m planning to give an internal presentation this week on what I do. I’ve been in my new job for three months, and although I’m still learning, I have enough of a routine that I’d like to share it with a few peers across the company to get feedback and ideas. Most of what I do is my job (that’s probably important, right?). I also still do a lot of cross company stuff (that’s probably pretty important too!).

With my remaining hours, I do “other stuff about testing” (I’ve been thinking about calling this “Project OSAT”, but it’s probably better to just call it “other stuff”). This includes writing books, contributing to other books, writing articles and blog posts, interacting on twitter, and participating in the overall testing community as much as I can find time for.

Sometimes, colleagues ask why I write, or why I speak at conferences. It’s not fame, and it’s certainly not for the money. It’s for ONE reason, and probably because of my lack of sleep due to 4:30am world cup matches, I’m going to share my motivation with everyone.

I write and speak because I’m lazy and cheap.

I suppose I should explain…

How We Test Software At Microsoft was written for one main reason. I talk to a LOT of people and companies about testing – many want to know Microsoft’s general approach to testing. I gave many of these companies the same answers. Over, and over and over again. Finally, I decided (with the help of a few colleagues) to write it all down and sell it for 25 bucks a pop (street price). There was just no way I could keep my sanity and continue to tell people about what testers do at Microsoft. Most of the articles and blog posts I’ve written have been for the same reason – I’m too lazy to give the same answer to multiple people, so when I start hearing the same question too many times, I write it down somewhere (NOTE: this is often also a good heuristic for automation).

Let’s not forget cheap. I hate paying to attend conferences (even if it’s technically my employer’s money). I do like to meet with other testers and talk shop (aka Project OSAT), but if I can get into the conference for free, I’m all over it (sometimes I even manage to get flights or more covered). So I submit proposals to a few conferences a year, and I typically get asked to speak at a few more. I do try to do a good job when I’m there (despite my laziness, I have a high bar for personal quality), but I go to conferences mostly because I get in for free.

Just because I’m lazy and cheap doesn’t mean I don’t care. I love writing, and I love speaking about testing and sharing what I’ve learned. I have worked, and continue to work extremely hard at both of these endeavors (see the note about my high bar for personal quality above). In fact, I’ve never taken money for any writing or speaking (some of probably think this is dumb, but it’s true). Way back when I first started writing for Better Software, I did this because not getting paid was easier than trying to decipher the company moonlighting policy. These days, I’ve found that I actually can get paid for most of this stuff, but I still don’t bother with it. When I consider how much I enjoy writing and speaking, doing it for free seems like the right thing to do (as long as you give me something free too – I’m still cheap).

1 comment »

Posted in General, HWTSAM, Me, Writing

Network and influence

May 13th, 2010

I gave a short talk to an internal MS community this week. The topic of the day was “influence”, and I thought it was appropriate to talk about the value of building (and maintaining!) an informal network – and the impact of that network on influence. The advice to have an informal network isn’t new – I bet it’s mentioned in the majority of books on leadership, but I don’t think enough people who want to be leaders (or be influential) take the point seriously enough. Your network is a resource for discovering new information as much sounding board,for ideas you want to share.

Your network is also an opportunity for you to build credibility. I know people who claim to be leaders. Sometimes they claim to be a leader because they’re in a position of authority. Other times, their “leadership” is just a self proclaimed act. In my world, you aren’t a leader until other people say you’re a leader. You don’t get it any other way than building credibility and trust among those you want to lead – and your network is a great place to build that credibility and trust across people who don’t have to listen to you if they don’t want to.

There are other values of a network. In HWTSAM, I talked about the Test Architect Group (TAG) at Microsoft. TAG is a collection of senior testers who meet regularly to talk about a variety of testing topics.

The value of having Microsoft’s most senior testers regularly review, brainstorm, and dissect solutions for complex test problems is immeasurable. In recent years, TAG has become something of a sounding board for new thinking, new methods, or new tools in testing. Presentations and demonstrations of ideas and implementations from test groups spanning every Microsoft division fill many of the meeting agendas. The value and depth of the feedback that the TAG provides is respected and sought after. A few meetings a year are reserved for “TAG business,” which includes discussions about company-wide initiatives driven by TAG
and other projects where TAG is a significant contributor (such as the MSDN Tester Center).

Perhaps the largest benefit of the regular meetings is in the value of networking. The extensive peer discussions and the view into the variety of work done across the company that is presented give TAG members much of the knowledge and information they need to make strategic decisions that affect the entire company.

I’ll say it again because it’s huge – the value of this particular tester community is the network. The fact that we talk to each other about what we’re working on gives us a small-world network – a huge reduction in degrees of separation between just about any testing knowledge in the company.

Yet – many people don’t take advantage of their opportunities to build their network. They don’t have the time – they think their day job is more important, or have something else to do they think is more important. It’s hard – last week I was in a great “hallway conversation”, but I found my mind wandering as I wondered if I should really be back in my office doing “real work” (I didn’t, now I’m behind, but it was worth it). Alas, some people who do consistently prioritize their commitment to building their network find that their superiors don’t share the priority and look down on work spent “outside of their core job” (my advice for these people is to find a new job).

I was asked once whether it was leaders who built great networks, or if people who built great networks became leaders. My old answer was that it was a toss up. Today, as I think about this more, I realize that there are a great number of leaders who got there because of their effort to build, maintain, and nurture a huge network of informal relationships. I’ve also seen leaders (including exec level leaders) fail because they thought that their position was enough, and didn’t work to build their network, credibility, and trust.

 

-

No comments »

Posted in HWTSAM, Leadership

Meaning

April 21st, 2010

For some reason, I have a lot of friends from the UK (could be soccer, could be Microsoft – I don’t know). I hang around them enough to know not to look down when someone says “Pants”, and that “sixes and sevens” have nothing to do with math. Learning the subtleties took time, and I misinterpreted conversations often enough to be embarrassed (or at least confused). We speak the same (roughly) language, but important meaning is sometimes lost.

Back in my day job, I asked four testers I know what terms they would use to describe some terms for the following scenario:

Say you have a simple test (or check) that verifies that the xyz driver is loaded when a USB XYZ device is plugged into the system. And – you want to run this test on 32bit and 64bit Windows, and with at least four different USB firmware implementations.

So – you have something you want to verify – let’s call it an “idea”, and you want to confirm your idea on some different configurations – seems pretty straightforward.

  1. What do you call the “idea” (plug in the device and make sure the driver loads)? Is that a test case?
  2. What do you call the permutations? Do they have a different name or are they just more of #1 – i.e. do you still have one test case, or do you have eight?

I have my own ideas (and I wrote about them in hwtsam), but when I asked four testers this question, I came up with five answers (adding mine to the mix).

I worry (perhaps too much) that lack of common vocabulary causes problems in testing. My colleagues who are in favor of certification (or at least aren’t violently opposed to certification) say that the common vocabulary encouraged in the various Bodies of Knowledge are the best things about the certification. But – since there are a variety of certifications (and a larger variety of interpretations), I can’t say that I expect certification will solve the vocabulary problem. But I can’t help but wish that we could dissolve the language gap.

Then again, perhaps it’s not that big of a problem. We do find ways to talk, and we do make (some) progress. For example, I was in a twitter conversation last week (or as much of a conversation as you can have in 140 characters) – I said “The problem with blah is that x, y, and sometimes z” – and the response was “In my world, I call that funklestein”. OK – great – that’s not my word, but now we can have a conversation, and it wasn’t that hard to get us there. I wish all testing conversations were that easy.

 

I’m wrapping up a ton of stuff this week in order to prepare for being away at star east next week. I hope to get a chance to meet some of you there. Come find me.

No comments »

Posted in HWTSAM, Software Testing

HWTSAM – One Year Later

December 7th, 2009

I think it’s been a year since How We Test Software at Microsoft made its way to store shelves (and amazon). For the first few months, I watched the amazon sales ranking multiple times a day. I took a screenshot last December 18th that shows one of the few times we hit the #1 testing book. The book actually made up in the 7k range overall once, but apparently I didn’t take a snapshot.

image

Since then, the Chinese version was released, and the Korean version is imminent, and I’ve traded writing on weekends and evenings for more time with my family (and occasionally, more time for work). When I finished writing the book, writing another was the farthest thing from my mind, but since then, I wrote a chapter for Beautiful Testing, and have at least entertained the idea of writing something else…someday.

In hindsight, there are many things I’d like to redo with the book, but it is what it is, and I can live with that.It’s a book full of information and stories about how testers at Microsoft do their job. It’s a book about people, approaches, and some tools. It talks about when and why we automate tests, but covers a wide range of other topics as well, and I’m happy with the story it tells.

I think the book has sold somewhere around 5-6k copies (I haven’t looked at numbers in 6 months, but I’ll update this post if I do). That’s certainly not a huge number as far as books go, but it’s still amazing to me. My thanks go out to everyone who bought a copy (and more thanks to those who actually read it).

6 comments »

Posted in HWTSAM, Me, Software Testing, Writing