HWTSAM Ten Year Anniversary

What a difference a decade makes. Amazon tells me that Dec. 10, 2008, is the release day of How We Test Software at Microsoft.

Before I reflect on the content, take a look at this photo from the back cover:

Or this one from a few months later at Barnes & Noble:

I’ve aged (at least) ten years since then – but I don’t think it’s too bold of me to claim that I’ve aged better than the contents of the book. Again, this isn’t the first time I’ve stated that a lot of the material isn’t relevant anymore…but I re-skimmed the book (I still own one copy), and thought it would be fun to reflect on the book and talk about what I’d do differently if I was writing it today.

Part 1

The first three chapters – Software Engineering at Microsoft, Software Test Engineers at Microsoft, and Engineering Life Cycles don’t represent anything happening at Microsoft (or most other places) anywhere else.

At the time, Microsoft had nine-thousand testers and eventually grew to almost 10,000. Today (or at least when I left), there were almost no full-time test positions. I wrote about how a few teams at Microsoft were using Agile practices – that’s something more popular today – as it is everywhere.

A 2018 version of HWTSAM would require a absolute complete rewrite of this section of the book.

Part II

This section of the book covers test design – a bit. For a current-day rewrite, I’d probably just write a lot about test ideas and where they come from – I discovered mind-maps after writing the book, so I think they’d get a lot of attention. I’d also emphasize that these testing ideas could be developed and explored by anyone on the team (not just dedicated testers) – and I’d definitely include a huge shout out to Elisabeth Hendrickson’s Explore It! for further reading.

In hindsight, BJ’s chapters on functional and structural testing dove a bit too deep into the details, and there were other areas of test design that deserved more coverage.

Part III

This section of the book covered tools and systems. Ten years ago, it included nothing on open source tools, nothing on mobile testing, and is probably the most irrelevant of the sections in the book.

Most notably is a huge part of Chapter 9 (Managing Bugs and Test Cases) where I discussed how to write a good bug report. While still possibly relevant to some people, a discussion on the subject today would definitely include a section on zero-bug backlogs and working without a bug database.

I wrote a chapter on Customer Feedback Systems that seems to barely scratch the surface of what Microsoft (and most companies) do today.

Ken Johnston wrote the final chapter of this section, and it is probably the first chapter to lose relevance – possibly even losing relevance between the time Ken wrote the chapter, and the time the book was released. Testing services and micro-services has matured massively since 2008 – possibly enough for a whole new book on the subject while I attempt to retroactively delete this snapshot of history.

Part IV

The final section of the book was about Building The Future. At the time, I don’t think I realized exactly why this section was important. It covered Quality Culture, Quality Ownership, and the various Test Communities at Microsoft.

Readers who pay attention to my recent ranting will probably note that community is something I believe in strongly as a method for discovering new information; a place where ideas can meet; and a place to vet existing ideas. At the time, I spoke of the groups at Microsoft, but not the value of community. While none of the communities I wrote about exist anymore, I now recognize some of my earliest thoughts on the power of community.


Writing the book was fun, and I know that benefitted Microsoft a lot (recruiters handed out the book to nearly every SDET candidate). It was also a lot of hard work, and due to a long story of effects, Ken and BJ and I received exactly $0 from the sales.

Even so, I’m still glad we wrote it, and in a way, I’m actually glad that so much of it became dated so quickly.

Unless I’m still blogging for some reason ten years from now, this will probably be my last ever post about the big book about testing at Microsoft.


  1. Great summary. I’m going to write a proper post here but Windows Update just completed and It’s
    asking me to reboot Windows 10….what could go wrong? brb!

  2. It’s stunning how much testing has changed in the last 10 years. I’ve not worked somewhere as giant as Microsoft but ten years ago I was hiring a tester for about every two engineers, and today I’m hiring one for every six to twelve. And I’m not doing it as Manager/Director of QA, but as Manager/Director of Engineering, because QA leadership roles have shriveled up and blown away.


Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.