At a recent internal forum, I hosted a panel discussion comprised of “senior” level testers at Microsoft. The panel was evenly split between managers and non-managers and we took random questions from the audience on career paths in test. The testers in the panel had experience ranging from 11 to 24 years in test. Some of the questions strayed slightly from the topic (e.g. “What does the future of test look like to you?”), but for the most part went well. The session was one hour, but could have held it’s own for at least another 30 minutes.
That’s pretty much it, so I suppose I can end this post here…
But one question came up that I want to dig into a bit, and I know that some of you who occasionally read this blog have some opinions on the subject, so fire them away please.
The question in mind may as well have been seeded by me, because it’s a subject I’ve brought up before. The question was: “Has Microsoft swung the pendulum too far in hiring only programmers as testers?” I can’t remember if my answer was “No, but Yes”, or “Yes, but No”, or “It depends”, but I’ll give a (much) longer version here for your benefit.
BTW – I would bet that if I asked most non-microsoftees the response they would give for this question, they’d say, “of course – my {cousin | dad | sister | friend } who is the best tester in the {company | city | country | world } can’t get a job there, and they’re {fantastic | awesome | superman }. They just can’t write code.”
And, in many cases, they may be right – but maybe not – or it depends. Some context is in order. I’m not the company expert on this, but my opinion and view of history is as good as any.
Our software is big and complex. We deliver platforms, operating systems, server products and put them into huge deployments. We need testers who can program to help solve the big testing problems that our systems present. Say, for example, you’re a tester on the Exchange team, and you need to verify that messages of varying lengths travel from one endpoint to another successfully. No problem – send mail to a test account and make sure it gets there… well…that solves one happy path. The reality is that the tester needs to send multiple mails through the system with a variety of headers and sizes to ensure that anytime the message is broken up that it still ends up at point B correctly. And, going through outlook is probably not the most efficient way to test this, so it’s probably better to construct the messages programmatically then send them through the system. Oh yeah – there are an almost infinite number of ways the server topology can be constructed, and those settings make a difference too (in other words, we need to deploy any arbitrary topology automatically on demand for these tests). As you can see, there are some obvious benefits to having someone who can program as a tester.
Microsoft has always had testers that can write code. Always. For much of our history, we’ve also employed testers who don’t write code (but who are still good testers). We’ve found over the years that our best testers and best test leaders had a programming or CS background. I’m guessing we also found that defining the career path (MS is really, really big on career paths) for testers was a bit easer to map out when we took programming background and application into account. We still use a lot of non-programmer testers in contract and vendor roles, and I’m guessing we’ll continue to do so. There are many great testers who have learned how to program, but over time we found (or I should say, my personal experience was) that when we fished in the pool of programmers for good testers, we found a much higher percentage of good people than when we fished in the pool of good testers looking for people who could write programs. So, for full-time “blue-badge” employees, we fish for testers where programmers live; we get what we want, and everyone is happy.
Hardly.
Note that in the above I referred to testers who can program. Where I fear we make the mistake in hiring is when we (or anyone else who hires “SDETs”) hire programmers to do testing. The difference is subtle, but important. A tester who can program is a tester first – but they are a tester who can rely on computer knowledge or programming skills to do their work better. A programmer who tests writes tools and automation first – hoping that it will help them be a better tester (hint – it doesn’t work very often). For the most part, Microsoft has testers who can program – and they are freakin’ awesome. The problems they solve and the way they go about it makes me proud, excited, happy, (and a bit jealous) all at once. It’s what SDETs should do.
The industry perception of the SDET appears to be a role of writing automation and tools all day. When I hear this, I think of all the great testers I work with and tell people “No, No – just because our testers can program doesn’t mean that’s all they do all day – they are testers first”. Then, whether it’s in 5 minutes or 5 days (but sometime soon after), I’ll see a blog or a tweet or some other sort of message from someone saying “I’m an SDET – I write automation and tools all day”, and I realize that there’s still some work to do. It doesn’t mean that most of our SDETs write tools and automation all day, but there are certainly pockets. My thoughts are that if you want to write tools and code all day, get out of my business – go be a developer. You’re not helping me (and you’re probably slowing me down), so leave me alone. I want to test, and I want to work along side people who feel the same way.
In the panel discussion I asked every hiring manager in the room to ensure that in the next interview they conducted that they looked for testers who used “technical” skills to solve their problems. Instead of making them write code on the whiteboard, give them hypothetical testing tasks and examine how and when they used the power of the computer to help them. For example, if I ask an interview candidate how he would know if a line drawn on the screen was straight and he started writing up an algorithm to scan the pixels on the screen and determine if the offsets were consistent with a “straight” line, I wouldn’t hire him (“Look at it”, or “use a straight edge while looking at it” are both acceptable answers). However, if I describe the message transport scenario I used a few paragraphs above and their answer is to ask everyone in the company to send a bunch of mail, I probably won’t hire you either. (I’m working on an internal presentation on hiring testers – I’ll come up with better examples :})
Every business is different, and your need for testers or “SDETs” will depend on your business. My call to action is this: If you wan to hire an SDET – hire a tester who can program – not the other way around”. You and your product will be much better off.
