Building Quality

I mentioned in my last post that I have a new job at Microsoft (and I discussed it a bit more on the last AB Testing). During the interviews for the job, I talked a lot about quality. I used the agile quadrants as one example of how a team builds quality software (including my roles in each of the quadrants), but I also talked about quality software coming from a pyramid of activities and processes. I’ve been dwelling on the model for the last week or so, and wanted to share it for comments and feedback…or to just brain-dump the idea.

Processes / Practice / Culture

The base of software quality (and my pyramid) is in the craftsmanship and approach of the team. Do they care about good unit testing and code reviews, or do they check in code willy nilly? Do they take pride in having a working product every day, or does the build fall on the floor for days on end? The base of the pyramid is critical for making quality software – but on established teams can be the most difficult thing to change.

Code Quality (correctness)

An extension of PP&C above is code correctness. This is a more granular look specifically at code quality and code correctness. This includes attention to architecture, code design, use of analysis tools, programming tools, and overall attention to detail on writing great code.

Functional Quality

Unit tests, functional tests, integration  / acceptance tests, etc. are all part of product quality. I italicize, because for some reason, some folks think that quality ends here – that if the tests pass, the product is ready for consumers. (un?)Fortunately, readers of this blog know better, so I’ll save the soapbox rant for another day. However, a robust and trustworthy set of tests that range from the unit level to integration and acceptance tests is a critical part of building software quality.

Automate Everything

There are some folks in software in the “Automate Everything” camp. A lot of testers don’t like this camp, because they think it will take away their job. Whatever.

As far as I can tell from my limited research on this camp, Automate Everything means automate all of the unit functional and integration tests…and maybe a chunk of the performance and reliability tests. For some definitions of “Everything”, I agree. Absolutely automate all of this stuff, and let (make) the developer of the code under test do it. The testers’ mind is much better put to use higher up the pyramid.

-Ilities

Performance, reliability, usability, I18N, and other non-functional requirements / ilities are what begins to take your product from something that is functionally correct to something that people may just want to use. Often, the ilities are ignored or postponed until late in the product cycle, but good software teams will pay a lot of attention to this part of the pyramid throughout the product cycle.

Customer Quality

It doesn’t matter how much you kick ass everywhere else in the pyramid. If the customers don’t like your product, you made a shitty product. It may be a functionally correct masterpiece that passes every test you wrote, but it doesn’t matter if it doesn’t provide value for your customers. Team members can “act like the customer”, be an “advocate for the customer”, or flat out, “be the customer”, but I’ll tell you (for likely the twentieth time on this blog), as a member of the product team, you are not the customer! That said, this is the part of the pyramid where good testers can shine in finding the fit and finish bugs that cause a lot of software to die the death of a thousand paper cuts.

Now, if you do everything else in the pyramid well, you have a better shot at getting lucky at the top, but your best shot at creating a product that customers like crave is to get quantitative and qualitative feedback directly from your users. Use data collection to discover how they’re using the product and what errors they’re seeing, ask them questions (in person, or via surveys), monitor twitter, forums, uservoice, etc. to see what’s working (and not working), and use the feedback to adapt your product. Get it in their hands, listen to them, and make it better.

More to come as I continue to ponder.