Five years ago this week, How We Test Software at Microsoft hit the shelves. The book has done well \u2013 both in sales, and in telling the story of how Microsoft approaches software testing.<\/p>\n
An unfortunate aspect of writing a book like this is that after five years, most of the book is obsolete. Sure, there are some nuggets that stand the test of time, but so much has changed \u2013 and is changing, that it\u2019s hard for me to recommend much of it as practicum. I don\u2019t know if a new version is warranted \u2013 or necessary, but you never know what will happen. <\/p>\n
To be clear, I\u2019m still proud of the book, and think it has a lot of great ideas that will help many companies \u2013 it\u2019s just not an accurate reflection of how Microsoft tests software. Here\u2019s a section by section recap \/ overview of what\u2019s changed\u2026.<\/p>\n
Now that we\u2019re a Devices and Services company, the description of our product groups is flat out wrong. Beyond that, our organizations are much different \u2013 our test dev ratios are changing (from nearly 1:1 at the time of writing to nearly 2:1 dev:test today). <\/p>\n
Five years ago, a few teams used adaptive (agile) engineering approaches, whereas now, nearly every team in the company is using or experimenting with some flavor of adaptive development. We ship almost everything faster. Testers are moving to analysis of end-to-end scenarios and data analysis, while \u201ctraditional\u201d functional testing is moving (where it belongs), to the programmers. The career path examples we showed in the book have all changed \u2013 and will likely change again as the company continues to adapt and grow.<\/p>\n
The techniques described in the about testing section are all still relevant (although these days, I could easily increase the chapter on test design by about 50 pages and keep it full of good ideas). What\u2019s changed most (and what will continue to change) is how much of the activities in this section are carried out by developers (and how much more<\/em> of will be carried out by the developers in the coming years. <\/p>\n I\u2019m still particularly happy with the chapter on Model-Based Testing (and thanks to Michael Corning and Harry Robinson for their help with the ideas for this chapter).<\/p>\n The section on bug reports is good \u2013 as long as you have the need to track bugs formally. Many teams can (and do) get by with fixing bugs as they find them \u2013 with no need to document the bug. I\u2019ve also become even less of a fan of tracking test cases, but the examples are probably still valuable for those who need to do this.<\/p>\nSection 3 \u2013 Test Tools and Systems<\/h5>\n