I come across this frequently enough that I’m sure I’ve blogged about it before, but context dictates that I do it again. The story I hear goes pretty much like this:

My team really needs to improve X, but nobody is taking responsibility for it. The fix is obvious – we need our exec / manager / project owner to mandate that we do X and measure Y & Z to make sure people are doing the right thing.

Here’s a concrete example. I was on a team once where quality was in the toilet. We couldn’t even get a build to run for over a week. The solution was require everyone on the team to perform a list of actions before every check-in. Of course, the list including using flaky tools, and getting sign off on every step, even if the step wasn’t applicable to a particular check-in. Given the extra work required for check-in, developers queued up weeks worth of check-ins for one big check-in rather than go through the steps in the checklist multiple times. Day to day build quality was slightly better, but velocity was way down. While slower velocity is not necessarily a bad thing, the big problem was culture. Someone honestly thought the mandates would create a culture of quality – but the only culture they created was a culture of finding ways to skirt the damn checklist. Team morale sucked, and the product we built sucked too.

To be fair, I’ve seen some successful mandates in my career. The push for improvement in every developer to write secure software at MS worked. But that culture change was also pushed by huge monetary losses.

In general (and in every other case I can think of), mandates don’t work – especially if you are using the mandate to change culture. Yet I see mandates suggested as a solution for changing culture time and time again. I just don’t get it.

The folks at 37 Signals say it best, “You don’t create a culture. Culture Happens.”

So help let it happen. Instead of a mandate, help your team see where you want to go. Make it personal. Appeal to their own pride and values. Show them how change will help them. Find allies who think like you do. Experiment.

Let it happen. Make it happen.

Recently, I was helping another part of the team with a project. Or at least it ended up that way. There’s a particular bit of what they’re doing where I have some expertise, so I volunteered to take care of a chunk of the work where I thought I could help out.

One thing I’ve learned through experience (and through testers eyes) is to take a look at the big picture of the problem I’m solving before diving in. For example, let’s say someone asked me to change the text on a button from “Query DB” to “Query Database”. What could be a 30 second task is far from it. First, I need to make sure the button is big enough, I need to check to see if there are other buttons in the application that need similar renaming. I need to make sure the documentation (including screen shots) are updated. I probably need to make sure the localization team knows what to do with the update. Of course, I’ll see if there are any tests that look for this particular string on a button, and after I punch them in the face for testing against a hard coded UI string, I’ll make sure they fix it.

In this case, I needed to add functionality ‘A’ to a system. I know functionality ‘A’ pretty well, but in this case, in order to add ‘A’ correctly, I needed to update ‘B’ – and for ‘B’ to work, I needed to refactor huge chunks of ‘C’. I went to the team and told them that I knew what needed to be done, but it was complex (due to A, B, and C), and that while I was willing to do the work, it would take me a few days to a week to implement and test.

Then they asked me my new favorite estimation question. “How long will it take you to do the easy part.” My answer, of course**, was, “It depends. Which part is the easy part?” To be fair, they meant, how long will ‘A’ take (because they had some insight into B & C), but it was still a fun quote.

 

** Alan-ism #17: The answer to any sufficiently complex question is, “It depends”.

Joey McAllister (favorite hashtag: #expectpants) recently talked with me (electronically) about critical thinking and learning.

The interview is here in case you’re curious.

After my last post, I thought I was done commenting on stack ranking (for at least another year), but two things inspired me to write just a bit more. First, a few people flat-out asked about my opinions for alternatives; and secondly, a consultant added some ignorance to the discussion that gave me just enough motivation to write a bit more.

In this article, Dick Grote talks about the purpose of the stack rank, and for the most part, he’s spot on. Yes, a lot of companies stack rank, and yes, it’s a reasonable way in some aspects to identify both top and low performers. And then, he says this:

That’s because forced ranking makes it impossible for managers to declare that all of their employees are above average, he says, and that makes it easier to find and retain the top guns

So, according to this, one basis for stack ranking is bad managers – or lack of trust of managers. Either choice seems to me like stack ranking is going after the wrong problem. It’s not impossible to have a team of above average performers relative to other groups (statistically, of course, on a team of 11 people, five are above average, and five are below average, but all eleven may be better than fifty people on a different team. To our credit, most teams at Microsoft try to find enough peer groups so they can stack at least 50 people together, where the curve is a bit more fair.

The notion that stack ranking is a cure for bad or mistrusted managers grates on me, because I know there has to be some truth to it. Once up on a time, managers at MS had more control over budgets and rankings. The curve has always been there, but there was flexibility in rewards (and a bit more in rankings), but that’s gone.

As far as alternatives go, there is no one-size fits all model. If I got to be king for a day, I’d leave it up to organizations within the company to come up with their own methods of reviewing employees. I’d probably have to do it at the division level, but I suppose divisions could decide if they wanted flexibility at lower levels in the org. Budgets would remain the same, but it would be up to the division to decide how to distribute the rewards. The challenges of re-recruiting top talent and managing out poor performers would remain, and I’d make sure all performance plans included stipulations for both.

With my grand new scheme, Division A could decide that they wanted to continue with the traditional stack rank. Division B could decide that everyone gets the same flat bonus with extra rewards for top performers. Division C could abolish review ratings and give smaller “spot” bonuses throughout the year. Divisions D, E, and F could do something completely different if they wanted. I (as King) would offer feedback where needed, but I would trust division leaders to make decisions that would advance their business.

One of the great things about Microsoft is that you can move from division to division, completely changing what you work on, without leaving the company. An advantage of a decentralized system, is that employees could choose groups based on their review system as well as the technology or people. Some people (believe it or not!) like the stack rank review system – and those people could seek out a group that used that system. Others, may prefer collaboration above all else, and could find a group where the review model favored collaboration.

One worry of this system is what happens when someone moves from a group with one review system to a group with a different review system. For example, let’s say I’m about to get a ‘bad’ number in a group with a stack rank – so I change groups to a group with equal rewards for everyone. My simple solution (I always prefer simple) is to prorate the rewards based on time in group before the review cut off. For those that worry about dishonesty from the group I left, remember, that I always prefer trust over conspiracy, and I believe that the right thing nearly always happens in the long run.

Another worry along the same lines is how to evaluate an employee potentially coming to your group. If another divisions review system is different, how do you know how they performed. My answer to this is the same as it is today with the company wide system. Stop looking at the damn review score. Today, the ‘5’ rating is a ball and chain. You get no rewards – maybe because you’re in the wrong team, and you can’t change jobs. I don’t think we actually fire many 5’s – we just lock them into a job they don’t like until they quit…but I’m digressing… The point is that we hire smart people at Microsoft. A poor performer in one group is often a star performer in another. Talk to them, talk to co-workers and former managers and use that information to make a choice on whether they’re a fit for your group rather than a number alone.

And with that, I’m done commenting on stack rankings (besides answering questions in the comment section, of course). As I’ve said before, and as others have repeated, I don’t mind the system, and the parts I don’t like are a reasonable tax to get to play in the Microsoft sandbox.

I just wish it were better.

In my last post, I tried really hard to focus on the message of employee reviews, and how that message shouldn’t be a surprise. I said things like: “I’m not a fan of the system…”, and “It doesn’t matter how messed up the system is…” – yet the mention of the dreaded-curve of the MS review system was still the focal point of some of the comments, and of many private emails. I had always planned to follow the Three Surprises post with my view of the curve-based review system, but I’ve gone back and forth several times in the last day. If you’ve skimmed ahead, you probably figured out that I decided to go ahead and share my thoughts. There’s probably nothing new here, but I think there’s probably something here that both MS and non-MS employees will be able to learn from.

As I also mentioned in my last post, I’m not particularly fond of the curve , but consider it the tax I pay in order to get paid extremely well to do a job I enjoy. The most recent version of the review curve is the third variation I’ve experienced at Microsoft and is only 18 months old. All versions have their pros and cons, and over beers, I will gladly share the full story and dozens of anecdotes. For this forum, however, I’ll keep things (relatively) short and focus on the latest revision.

The most positive aspects about the curve are the simplicity and the transparency. There’s little mystery to the review system – peer groups (people in the same level band) are stack ranked from 1-n, and then the rank is broken into ratings groups based on known percentages. The bonus percentages for each score and level band are known, so there’s not that much mystery in the system (other than the surprises I mentioned in the last post). It’s easy to apply and easy to manage at all levels of the organization.

On the negative side (from my view at least), there are just a few things I think are worth pointing out. Microsoft values differentiation of employees. What this means in practice is that once the lines are drawn to determine review rankings, everyone in each peer group receives the exact same reward. In a ranking system of 1-5, this means that the person who just missed getting a “1” ranking gets the exact same rewards as the person who barely squeaked out of a “3” rating to get a “2”. As I mentioned above, this makes the system simple, but also can send some difficult messages. Despite this, since the goal is to differentiate employee rewards, this aspect of the curve works as promised.

It’s also a big challenge for a ‘superstar’ team. The curve applies to the team (or, for higher levels, across a division), but it’s possible to get a lower review score just by being on a good team. Even worse, the opposite is true. One can guarantee a good review score by finding a dysfunctional team and stepping in as the superstar.

And that leads to the part I really worry about – the side effects of the curve and competition. Behaviors visibly change as calibration season approaches. Some people begin to suck up in attempt to play the “review game”, while others refuse to take risks they would take at any other part of the year. A co-worker was commenting recently on strange behavior in a meeting when I asked him, “would the conversation have been different at a different time of the year?” (the answer was a resounding yes). I am a huge believer in collaboration (I once had a manager who said, collaboration was my super-power). The problem is that collaboration and competition don’t work well together. It’s in my best interest (as far as rewards go) to do everything I can to make sure all of my peers fail – and they, likewise, should feel the same. Given the system, that case could indeed happen, but fortunately most teams are able to rise above it – despite the potential hit on our own rewards and career growth. Alfie Kohn has written some fantastic books on this subject – including twoI keep on my desk this time of year: Punished by Rewards: The Trouble with Gold Stars, Incentive Plans, A’s, Praise, and Other Bribes, and No Contest: The Case Against Competition. To me, Kohn’s work says a lot about review systems in general. Combine these with Dan Pinks work on motivation, and you can form a pretty good picture for yourself with review systems.

Psychologically, as you can imagine, it’s a mess. Once the posturing is over and the calibration happens, news of the actual review scores slowly leak out – causing a zombie-like depression over a fair share of employees. After a few months, the fog shakes off, and it’s back to business as usual – for another 8-10 months or so.

I realize, that by sharing these thoughts, I’m opening the floodgates for criticism and complaints about the system. I live with this system in order to get to do what I do. If I ever get to a point where my job isn’t the best possible job I can imagine for myself, or if a review system conflicts with what I get to do, my story will change. Until then, it’s definitely a price I can pay.

I’m two months into my 18th year at Microsoft, and I still really enjoy it – most of the time at least. My job is great, I work on amazing technology, and with people smarter than you can imagine. But, for two months or so every year, the dreaded MS review monster comes out from the magic curtain of HR and freaks out employees across the company. I’m not a fan of the system, but to me, it’s the tax I pay to get to state the second sentence of this paragraph. I won’t rehash the forced curve stack rank story, but I will share one bit of advice.

It doesn’t matter how messed up the system is – review ratings should never be a surprise.

Managers may say, “but I didn’t know they were getting this rating until review time” – to which I’d say, “baloney” (actually, something stronger, but this blog is rated PG). As a manager, if you work in a stack rank system, you always need an idea of where your employees line up. When I was a manager, I made sure my team leads were stack ranked monthly. If you do it that often, it doesn’t take that long, and you can work on setting expectations throughout the year. It’s common sense, and common courtesy.

I thought I’d share (all true experiences) a few‘surprises’ from my history at Microsoft as examples. I’ve generalized the rankings for consistency where appropriate. The first surprise goes like this:

“Alex” worked his butt off all year. His team enjoyed working with him, and he finished several projects that many of his peers admired. He was quiet, but determined and proud of the work he accomplished.

At his review, he received a rating of ‘underperformed’. Alex was in shock and took the next week off to make sense of it. His manager told him that he accomplished a lot, but that he did a lot of the “wrong” work, and that his work quality was poor. This was the first time he had heard this feedback.

In this case, both Alex and his manager blew it (yes, I’m giving Alex some blame too). His manager bears the bigger portion of blame for not giving him feedback throughout the year, but Alex needed to ‘manage up’ as well and ask his boss for feedback. Alex left the company soon after and his now near the executive level at another software company.

“Beth” was almost a celebrity on her team. She had a lot of visibility around the company, and her team was excited to have her as a team member. She was one of those people who made everyone around her a strong performer, and her teammates heaped praise on her as part of the review process. She put a ton of effort into crafting performance goals that would show her value on the team, and even presented them to the Vice President of her division for feedback and approval. Her manager gave her feedback throughout the year that she was doing well…and then she got an ‘underperformed’ on her review. Both Beth and her manager were shocked! You see, at Beth’s level, her skip level manager expected something different from her. There was no warning or feedback before this.

Beth quit the team immediately and found another job at Microsoft.

In this case, I put the majority of the blame on Beth’s manager (for not looping in his manager earlier), but one thing I’ve learned at MS, at least, is that you always want to put at least a little effort into knowing your management chain and knowing what they expect of you.

The last surprise is a first hand experience – it’s the only time I’ve ever surprised an employee at review time. At this time, we had a review system that went in half-point increments from 2.5-5.0. Most employees received 3.0 or 3.5 ratings. A 2.5 meant your job was at risk, and 4.5 and 5.0 ratings were reserved for (roughly) the top 1% and .01% respectively. The conversation went like this…

“Evan – you’ve done a great job this year. You’ve built a great team, done some fantastic technical work, and helped build a strategy for the team…but as I look at what your peers have done, and what the rating systems dictate, I can’t justify giving you a 4.0 rating…

<I watch Evan’s eyes as they go from confused, to hurt…and then, just has his eyes get a twinge of Hulk rage in them, I continue…>

So, based on what you accomplished and how you got it done, I have no option other than to give you a 4.5 rating.

Yes – it was a bit mean, but given the nature of the surprise, Evan was extremely happy. He’s still at MS, and has grown into a major leader in his division.

As far as I’m concerned, that’s the only type of surprise that should ever happen with an employee review (just be careful of the timing on your delivery :} ).

As I write this, I’m waiting (literally, waiting on hold) to give a webinar for Swiss Testing Night. It’s a twenty minute presentation – which I love (see my last post for another twenty minute presentation from me. I would love to see a test conference filled with nothing but 20-30 minute presentations someday (and would probably even help organize if someone were willing to really do it).

But this post isn’t about conferences, it’s about learning. These days, I’m a bit obsessed with ideas and learning, and can’t help look for patterns of learning in nearly everything I do. It’s intern season at Microsoft, and we have a few dozen eager young faces floating around the hallways making great things happen. As part of their experience here, the interns are invited to a weekly lecture series on a variety of topics – including a talk from me later this summer. I haven’t settled on a topic yet, but I expect it will be something along the lines of how what you learn at university probably won’t help you in the real world – the point being that the bits of facts and knowledge will help, but knowing how to learn is the most critical knowledge one can gain at university. I’ll find some ways to make sure the message works without sounding too much like a crazy old man, but we’ll see what happens.

Yesterday, I had a great visit with a computer science student at a university in Georgia, and I had a wonderful conversation on this same topic I’m on the alumni board for the school of Arts and Humanities at my alma mater (Central Washington University). Last  week, the chair of the music department emailed me (we’ve met once before) and asked me if I could talk to his son (the CS major mentioned at the beginning of this paragraph). They came over yesterday for a quick visit and chat. Over lunch, and during our tour, we talked a lot about the power of learning, systems thinking, and critical thinking. I’ve interviewed MIT grads who could practically recite a textbook, but couldn’t think – so it was great to talk to this kid studying computer science at a liberal arts school who seemed to already have a grasp of how he fits into the real-world.

I’ve said this before, but it’s worth stating again. Programming is easy – especially with the languages available today. What’s really hard is implementing the right program, or writing code to solve the problem in the right way. This is true for applications as much as for test automation – and you need to use your brain and learning skills to have a chance for success.

If you read my blog, you’re probably already sick of me, but I’ll share a few links anyway.

I gave a webinar a few weeks ago for the fine folks at EuroSTAR. Embedded recording is below. It’s a twenty minute ramble on ideas in testing. My laptop with the presentation was in front of me – and low on a table – so don’t be too distracted by my creepy eyes looking down all the time.

I took part in my first “Twinterview” (twitter interview) with the extremely nice people that run the Fusion and STANZ conferences in Australia and New Zealand. Here’s the link to the twinterview if you want to get a little more insight into what’s on my mind these days.

More details on those conference connections coming in a future post.

When serendipity strikes, I know it’s something I should try to write about. At least three times in the last week I’ve had discussions about vision, tactics, and the necessary balance needed between the two. Without vision, your daily work is just work – a grind that takes you in no particular direction at all. There’s nothing wrong with this approach for many roles. For example, when I was a bicycle messenger, my entire job was to pick up stuff from point A and bring it to point B. It was different every day, and I had fun – but there was no vision. The only goal I was working toward was paying my rent. On the other hand, you can certainly have vision (e.g. “to be the world’s greatest tight-rope walker”), but without a tactical plan to get you there, it’s probably not going to happen.

Here’s another example. During the summer, I usually have a chance to run more. Usually, I just run when I can, and run whatever distance I have time for on that day. I could do the same thing this summer and achieve all of the benefits one gets from exercise. I don’t need a vision – I can just run.

This summer, I have a goal – I’d like to run 10k in less than 65 minutes by the end of August (I’d love to go for an even 60, but not sure I can reach that goal). I have no reason for the goal other than personal motivation, but the presence of the goal changes my tactics. In order to reach the goal, I can’t just run. The goal dictates that I need to work on pace, and distance, so I need a strategy for both. I know my 5k pace ~33 minutes), and know I can run 10k (I don’t have a current pace, but know it’s slower than my 5k pace).

When coming up with a strategy, I’m fond of the Current State / Desired State format – e.g.:

There are a lot of details hidden in the arrow pointing from Current State to Desired State. I need to work on increasing my pace and distance, so I’ll have to make sure I have longer runs every week, as well as interval training to help increase both my pace and strength. And, because I’ll be running more (and in some cases harder), I’ll need to take precautions to avoid injury (and at my age, do my best to avoid general aches and pains). Even with those details, aligning the tactics necessary to achieve this goal aren’t that complicated. More or less, I’ll still just run – I’ll just have a few different variations this year that will help me achieve my goal.

Of course, goals and visions on software teams are a bit more complicated. Team-wide changes require technical changes as well as people changes. If, for example, you want to move your team to using more agile practices, you may choose to deal with tdd/bdd frameworks and new project management, but you also have to deal with a web ov concerns and hiccups as you guide team members through change. In a situation like this, your Current State->Desired state diagram may look more like this.

It’s ugly, but expected. I’ve seen many people with “visions” fail to succeed because they spent their time looking for shortcuts through the system rather than understanding that navigating the system is the key to achieving a vision (conversely, I’ve seen many others fail because they focused on navigating the system rather than their initial goals).

Bottom line is that both tactics and vision are easy. The challenge facing leaders is balancing execution and the vision and showing results.

Our family spent a bunch of time this weekend cleaning out the garage and taking care of a variety of long neglected household tasks. One thing I’d been meaning to do for over a year now is to get my Ducati up and running again. Between picking kids up (need a car for that), and riding my bike to work most of last summer, it’s probably been 18 months since I started the thing. Keep in mind, that Ducati’s are fickle machines to start with, but I figured I’d work on it for a while before calling someone to load it on a truck and haul it to the shop.

The first thing I did was drain the gas tank. I couldn’t recall if I added fuel stabilizer, but after that long, I can pretty much guarantee that the fuel was bad. I took the bad gas to the hazardous waste site (open on Sunday from 9-5!), picked up some new fuel, headed home, and gassed the Duc up.

I had the battery on a battery tender, but was still slightly surprised that it still had some starting power in it. Unfortunately, the engine just wouldn’t turn over. I double checked the fuel line (clear) and then pulled the spark plugs. The plugs were a little dirty, so I swapped them with a spare set from the toolbox.

Still nothing.

Sitting on the bike, I took a moment to think through how the engine worked. The starter was working, gas was flowing, but the engine wasn’t starting. Fortunately, I have a carbureted engine, and know what all (or most) of the engine workflow. There could still be bad gas still in the system, or there could be a problem with the carburetor. But neither of those seemed likely. I tried starting it one more time, and the engine just wouldn’t kick in.

While thinking through it some more, I noticed that I forgot to reattach one of the spark plug caps. I reattached it and…still nothing.

But – the behavior (i.e. engine sound) was identical with and without the spark plug cap attached – which pretty much guarantees that the spark plugs weren’t firing. I took them out one more time, cleaned them, and this time, checked the gap. For some reason, my spares were gapped really narrow (hint – always check the gap – even on brand new spark plugs selected for your vehicle). I widened the gap a few millimeters to spec, put them in and…

Vroom!

I immediately grabbed my helmet and gloves and went for a spin, and the bike ran great. No stalls, backfires or stutters. It probably still needs some more air in the tires and an oil change, but it sounds and runs just like a Ducati should.

In the end, this was just another debugging and diagnostic problem – much like the problems I face almost every day. The key points to remember are:

• Know the system, and think about the system. When software (or Ducatis) fail, think through the entire system to note where failures may be occurring
• Observe what’s going on – Products (and engines) fail for a reason. Chances are that there are unnoticed clues to the behavior you are seeing, so remember that anything you see may be helpful.