The Myth of the Myth of 10x

I first heard of “10x” software development through the writings of Steve McConnell. Code Complete remains one of my favorite books about writing good software (The Pragmatic Programmer, Writing Solid Code, and The Clean Coder are also on that list). Steve McConnell’s rarely updated blog (titled 10x Software Development, contains the following tagline:

Numerous studies have found 10:1 differences in productivity and quality among individuals and even among teams. This blog contains Steve McConnell’s thoughts about how to move toward the "10" side of that 10:1 ratio.

This article, and this one, are probably the best views from the McConnell side on 10x. They cover a lot of the studies and anecdotal information about the 10x concept. Although the original study cited a 10-fold difference in productivity and quality between different programmers with the same levels of experience, I use the 10x label to describe those software folks who “merely” differentiate themselves in terms of output and quality significantly more than their peers. While I will use the term “10x” throughout this post for conformance with the topic, it’s not an exact multiplication. For me, the term means, significantly higher output and quality.

The Myth?

Recently, there was a bit of buzz about this article about the Myth of the 10x Engineer. The meat of the article includes some valid criticism of the studies citing 10x Engineers, and as someone who is generally a skeptic, I acknowledge that the studies have holes. I think there are a lot of misconceptions about what a 10x engineer actually is, so the controversy doesn’t surprise me.

The article begins with the following statement:

I was a 10x engineer for 7 months and then I was a 0x engineer for a year and a half. You burn the candle at both ends. You end up with alcoholism and depression. You’re talking about a very small subset of people. And they might end up in divorce and financial ruin. When people think you’re a 10x engineer, they think you have skills that you don’t. You invariably let people down.

The full article (which, by the way, is very nicely written and worth the read) makes a very good case against the validity of the studies about 10x, and includes several statements from the author’s twitter followers supporting the quote above.

Not a Myth?

But I interpret 10x much differently, and think a lot of commenters miss the point. This, in my opinion, is at least partially due to the “10x” label.

But more importantly, 10x isn’t about working 10x as hard. It’s not even about working twice as hard. It’s not about effort. Being 10x (or significantly more impactful, or whatever you want to call it) is about making smart choices and taking care in your work. I’m sure if I were to put fifty similarly proficient developers in a room and ask them each to implement a known set of algorithms or implement well defined functionality into an existing app, or any other task where the problem and solution are largely understood, that there will be some variance in the time it takes each of them to finish the tasks, but the best in the group would maybe be twice as fast as the average – if that.

Where 10x software engineers excel is in adaptive challenges – where the problem may be understood, but the solution is an unknown (and where the problem may change as the solution emerges). 10x engineers recognize patterns, they know when they’ve hit a dead end, and they can zoom from the 10,000 foot big picture down to minutiae and back again in an instant. They aren’t significantly more productive than their peers because they work eighteen hours a day or even because they type at a whirlwind pace. They are more productive because they solve problems 10x faster than their average peer. They are more productive because there’s less waste in their work – they are careful enough to do things right the first time, but have enough breadth of knowledge that they find the correct choice sooner. You can’t make yourself into a 10x engineer by working harder or longer – you make yourself a 10x engineer by studying and practicing your craft, by investing in learning, and by challenging yourself to continuously make better decisions about how you work.

All that said, not everyone needs to be – or can be a 10x – and that’s ok. But I, for one, want (and love) to work with 10x developers so I can learn from them, learn how they learn, and perhaps discover a few tips to improve my own output and value.

14 Comments

  1. It’s really sad, and funny, how programming is still in the dark ages.

    I don’t think there are 10x programmers, so much as there are 1/10thX programmers.

    People who waste time on unit tests, daily meetings (scrum), and spend their productive time writing really crappy, bug-ridden code are the 1/10th X programmers.

    Being a 1/10th X programmer has become almost “industry practice.” You see it whenever you see people talking about “commit code to production on your first day!”

    It’s the culture of writing code before you ever think about, let alone understand, the problem, that produces the 1/10th X programmer.

    It’s not that they are stupid or less capable, it’s that they’ve been taught nonsense by people who propagate these myths because they don’t understand programming — whether’s its’ bosses who see them as interchangeable cogs and expect incompetence and so they force SCRUM and test-first on them, or other programmers who can’t be b others to do a good job and who instead think they’re “hackers” because they squirt out a bunch of code right away that is full of subtle bugs (that will never be caught by unit testing.)

    Ironically it’s the people who believe n the “10x programmer” and the “a people hire a people, while b people hire c people” myths that tend to be the 1/10th X programmers (and generally C people.)

    Reply
    • “commit code to production on your first day!”

      The journalism world has done an incredibly shallow job of covering this. Invariably, at the teams that do this well, the code-to-be-committed is something like:

      “Modify the HTML on the about page to include your image and bio. Add an image to the repository to pull from.”

      Now in order to do that, you need to drive a spike all the way through the DEPLOYMENT architecture – to “get” the version control system, the build/deploy pipeline, and a tiny part of the systems architecture. You’re writing HTML here, not even PHP, or java, or messing with classes, N-tiers, not touching the database.

      It’s a huge difference.

      The “commit to prod on day one!” thing is, when done well, a checklist that walks you through the process.

      Then again, I said “when done well” like three times above. I’m sure there are people that don’t do it well, in which case, the commentor’s comment holds. 🙂

      Reply
  2. nice post, Alan.

    I’m with DeMarco/Lister on this from PeopleWare on nX staff – the best staff do work the poor staff can’t do /AT ALL/. so there is no X to measure. I think I get the spirit of your point, and it is a point well made. Thank you.

    Reply
  3. There are lots of issues with the 10x myth.

    10x compared to what? On what scale? Considering what factors? Ignoring what factors? Who did the studies? Were those studies any good? Whom did the researchers observe–college students or professional programmers? Working internally, in a bank, or working on mass-market commercial software? Are the conclusions of the research conclusions generalizable? Relevant? If some programmer produces 20 times more code than another (but the code doesn’t work), is that programmer 20x more efficient, no more efficient, or -20x as efficient as a programmer whose code DOES work? Suppose that a programmer codes a saleable feature in two weeks of work but, in the course of doing so, creates a poisonous climate for the rest of the team, such that the rest of the team resigns. Is she 10x more productive, or 0x more productive, or -1000x as productive as a programmer who produces the same code in three weeks?

    The point here is that “productivity” is multivariate, multidimensional When you focus on one, or two, or three of those dimensions when hiring or managing, you invite distortion and dysfunction.

    There’s no question that some people are more productive than others when you base productivity on a given metric, based on a given model or theory. I’ve seen them. Yet they’re not “more productive” across the board. Some are more productive in some dimensions, not in others, and actively toxic in others. Yet it’s important to ask what you’re ignoring, what you miss, what you give up when you choose to pay attention to one (or even several) attributes while ignoring others.

    Finally: why should we, as testers, sanction bullshit measurement models based on bullshit citiations of bullshit studes?

    —Michael B.

    I’m now bracing myself for accusations of being intolerant and unfair of “imperfect” models. There’s a BIG difference between imperfect and industrial-grade bullshit such as the 10x claim. If you think that some programmers are good and others aren’t so good, why use “10x” instead of “good” or “better (in some dimensons)”?

    Reply
    • The missing summary bullet points are:

      1) 10x is a hypothesis. Like many hypotheses, the proposition is far more important than the name, but 10x is the hypothesis I’m discussing

      2) Some have proposed that 10x comes purely from effort. I think this stance is wrong. Decisions, systems thinking, etc. set these people apart

      3) I propose that some people are SIGNIFICANTLY more productive than their peers.

      I agree that 10x isn’t measurable (and also the notion that perhaps most developers are 1/10x). In summary, I believe that some people far outshine their peers, and they shine via smart choices & systems thinking rather than pure effort. I use the 10x number as convention, and as a discussion point for #2 above. I think it would be silly to try to measure it (same as I feel about most people measurements – I just propose that the really, utterly fantastic people we work with get to be that way through many means other than effort.

      In my experience the “Significantly More Productive” people I’ve worked with aren’t measured at all. In fact, my measurements are purely anecdotal, yet they are shared independently by others I’ve worked with. If you try to measure SMP (or 10x), you’re on a fools errand (or you’re a fool).

      I think we’re largely in agreement – I think you’ve just latched onto the 10x statement from a different view than I intended (which isn’t out of line given that “10x” as read, is highly tied to something that can be specifically measured.

      Reply
      • Yes, I didn’t take the 10x as being a real thing anyone really measured, which I thought was the spirit of your post. I believe /one/ among Michael’s concerns is that even if we use the term “10x” but mean “significantly more productive”, some chowderhead is going misunderstand us and make the kind of generalizations for which Michael’s “Wall of Questions” might lead to helpful systems thinking.

        That’s what my comment about ‘there is no X to measure’ was supposed to it on, but I didn’t want to get into a big fight on the interwebs. I’ve had one already this week, and it was no fun.

        Reply
        • Thanks Matt – I like this tangent, because it’s (**) a real problem that I don’t have a solution for.

          (**) The problem I’m referring to is what to do when a concept is known by “Name A”, but most people who pay attention know that the name is flawed and can lead to confusion – yet, if you don’t start by referring to the concept as Name A and expanding from there, you won’t have a point of reference to start from.

          “10x” is one example of that. “Automated Testing” is another.

          I can’t write this post without referring to the 10x hypothesis by name, but, by using the name there’s an automatic bias by those who realize the name is misleading or wrong.

          Reply
          • Some programmer quickly wrote a routine that doesn’t save the state of this page (and this comment field) periodically. If I accidentally click the mouse outside of this comment field, and hit backspace, I go back a page. When I’ve come back, I’ve lost the entire comment. I wonder whether that programmer was measured as being 10x as productive as the programmer of other blog software that retains the content.

            —Michael B.

          • I don’t know what else to say – I understand your concern with the term 10x, but I don’t understand your fixation on the term over what I actually wrote about.

            Not once in this post did I ever talk about the measurement of humans. I don’t believe in that. However, I do know a handful of people in software who are time and time again make the decisions that enable them to be much more productive than their average, “talented” peer. I’m sorry I used a term from the post I was replying to when speaking about those folks. I tried adding a disclaimer too, but that too fell on deaf ears.

            (thanks, by the way for viewing and commenting – I think we actually agree on the meaninglessness of the 10x term as a measurement despite the confusion in interpretation)

          • This is a recurring theme with Michael, right?

            Example:

            When people say “best practices”, they don’t actual mean the practices which are best – they mean something like “the practices we have found in our projects lead to improved outcomes, often enough that you might want to think about using them.”

            So using the term “best practices” is LIEING. STOP DOING IT.

            I understand this argument; I’ve used it myself.

            I see a very similar argument here:

            When people say “10x programmers, they don’t actually mean the programmer will produce the same software in 1/10th the time as a regular programmers – they mean something like “programmers who seem to produce software much faster that their peers, given the team is working on the same general technology. They also might be able to learn new technologies much more quickly, going from zero to sixty, or nothing to in production, must faster. Or perhaps both. The point is they are much more productive.”

            So using the term 10x programmers is LIEING. STOP DOING IT.

            I believe this is the argument michael is advancing. Put this way, I understand it much better — I just didn’t react to well to his initial style.

            There are a lot of people who don’t react well to my style. It takes a village.

            Hopefully, put three ways (Alan, Michael, Myself) the reader can decide for themselves how they feel. 🙂

  4. I have been studying design thinking for a year or so and what that teaches me is that there are two kinds of problems puzzles and mysteries. The former has context and data and easy to work with that most people can solve. The latter has either no data or fewer data or excess data and the solutions are uncertain. This is where design thinkers fit in and according to your 10x are those people, people who can see patterns and match them to find innovative solutions. There are tools out there that can help others to be one of 10x. Creativity can be learnt and we all can do if we learn to adopt uncertainty and take small steps quickly and don’t go for big steps slowly.

    Reply
  5. Great post. I agree with both the letter and spirit of your comments (including my “rarely updated” blog, sadly). In particular, I agree with the idea that “10x” is not meant to be taken to mean “10.00000x” — it’s more notional than that. However, it is intended to convey that there are significant differences — all things considered — between the best programmers and the worst.

    As I’ve written before, the research on this point is voluminous, clear, and all points in the same direction. If there’s a study that says, “All programmers are about the same,” I’ve never seen it, and no one has ever called my attention to it. The idea that the differences are insignificant certainly doesn’t match my personal experience.

    I find it interesting that some of the objections to the 10x claim purport to be objecting to the 10x claim but really point to potential problems with measuring 10x differences.

    * “Programmer energy ebbs and flows over time” — yes! So someone who can’t sustain a 10x level of output is not a 10x programmer, despite the fact that if you measured them during a “flow” period they might be measured as 10x when they really aren’t.

    * “Someone could very quickly produce low quality code” — yes! Those people are out there, and I agree that they are not 10x. I have written about that issue extensively. But I have also written about the fact that in real life, the people who produce the code with the lowest defect rates also tend to be the fastest, though of course there are exceptions.

    * “Productivity is a multi-dimensional attribute; it isn’t just about coding” — yes! Of course it is multi-dimensional attribute. I’ve written about that too.

    The biggest issue with 10x, which so far only Larry O’Brien has really talked about, is that the “n” in nX is a function of the size of the group you’re measuring. If I have a 2-person group, it’s unlikely I’ll find a 10x difference. If I have a 10,000 person group, it’s likely the difference will be far greater than 10x.

    The sensible way to address this issue is to decide how many standard deviations we want to talk about when we evaluate the “10x” claim. I don’t think it makes sense to talk about differences between ends of the range if the range is +/- 3 standard deviations of difference. (+/- standard deviations would include 99.73% of programmers.) I agree with the idea that people on one end of that distribution will be “negativity productivity units.” I’ve written about that too.

    I suppose someone could argue that really we just want to look at the middle two standard deviations (+/- 1 standard deviation, 68.3% of programmers), but the idea that we’re going to exclude 32% of programmers from the analysis doesn’t make sense to me.

    I think it makes the most sense to talk about differences in ends of the range with +/- 2 standard deviations (which would include 95.4% of programmers).

    Most of the research has been conducted on small groups, 5-25 people. Those groups are small enough to not show real +/- 3 standard deviation ranges, and in many cases they are too small to show even +/-2 standard deviation ranges.

    If you apply the statistics to adjust for small group sizes, you end up close to the 10x claim in most cases. Cartwright & Sheppard 1998 is an interesting case in point. They had a group size of 5 and found differences of 1.9x and 2.2x. If you approximate what their data suggests would have been observed in a larger group (i.e., by calculating what the differences would be at +/- 2 standard deviations), those factors increase to 3.8x and 6.9x.

    I focus on Cartwright and Sheppard 1998 because they show the smallest difference in productivity of any study I’ve seen. That puts the very bottom end of the range in the 5x range for +/- 2 standard deviations. Every other study would be significantly higher than that. And if you wanted to include more than +/- 2 standard deviations of course that would make the ranges even wider.

    If anyone wants to read more of my thoughts about programmer productivity, you can see several blog entries here:
    http://www.construx.com/Blog_Main/?searchtag=productivity&TagId=149962&TagLanguageId=1033&TagCount=6&TagType=Content

    Reply
  6. Your reasoning, as far as I can tell, appears to be “the 10x concept is not a myth because I define it differently from the way it was defined in the studies that are claimed to support the 10x concept”.

    I don’t think this works. What *would* work for me: your describing someone you’ve actually met (or have reliable information about), who fits your definition of “someone whose aptitudes allow them to deliver significantly higher output and quality”, and some explanation of why they are that way. That would still be anecdotal evidence, but better than no evidence at all.

    In a blog post a few years back (http://www.construx.com/10x_Software_Development/Chief_Programmer_Team_Update/) Steve has described what some might call “the original 10x programmer”, Harlan Mills. According to Steve, “Harlan Mills personally wrote 83,000 lines of production code in one year.”

    I think this qualifies Mills as a “very prolific” programmer. One issue with that descriptor is that, as you’ve aptly remarked, “prolific” isn’t the same as “productive” (and it’s one of the tragedies of our profession that we consistently fail to distinguish the two). We all know people who churn out reams of code that turns out to be worthless.

    Interestingly, it turns out Mills was one of those people.

    At least he was on the particular project Steve describes as “one of the most successful projects of its time”. You don’t have to claim that “all programmers are about the same” to make a counter claim to the 10x concept; you can for instance merely point out that if programmers are extremely inconsistent in their performance, that would explain the data in the 10x studies just as well. Maybe Mills was a 10x on some other project, but my research suggests he wasn’t a 10x in the above sense (“significantly higher output and quality”) on the Times project.

    Stuart Shapiro, in his 1997 article “Splitting the Difference”, described the same project somewhat differently:

    “As evidence, the authors pointed to the development of an information bank for the New York Times, a project characterized by high productivity and very low error rates. Questions were raised, however, concerning the extent to which the circumstances surrounding the project were in fact typical. Moreover, it seems the system eventually proved unsatisfactory and was replaced some years later by a less ambitious system.”

    Source: http://sunnyday.mit.edu/16.355/shapiro-history.pdf

    Shapiro is quoting from a much, much older article that appeared in Datamation in May 1977, “Data for Rent” by Laton McCartney:

    “Unfortunately for The Times, the IBM designed system didn’t prove to be the answer either. ‘They touted us on top down structured programming’, says Gordon H. Runner, a VP with The Information Bank, ‘but what they delivered was not what they promised.’ When the FSD system proved unsatisfactory, the TImes got rid of its IBM 370/148 and brought in a 360/67 and a DEC PDP-11/70. Further, Runner and his staff designed a system that was less ambitious than its predecessor but feasible and less costly. […] ‘With the new approach we’re not trying to bite off the state of the art,’ Runner explains. ‘We’re trying to deliver a product.'”

    (The PDF for the Datamation article isn’t available online, but I’m happy to provide it upon request.)

    I find it ironic and funny that “the original 10x programmer” left behind such a bitter taste in his customer’s mouth. It reminds me of the ultimate fate of the Chrysler C3 project that was the poster boy for Extreme Programming.

    Our profession has long been driven by fad and fashion, with its history written not by the beneficiaries or victims of the projects on which we try new approaches, but by the people most biased to paint those projects and approaches in a good light. Our only way out of this rut is to cultivate a habit of critical thinking.

    (I’ve written a lot more about the 10x myth, and my reasoning for branding it a myth, in my book: https://leanpub.com/leprechauns – if you found the above informative, check it out for more of that.)

    Cross-posted to my G+ feed.

    Reply
    • @Laurent, If everything you say about the New York Times project is true, that still doesn’t dispute the idea that Mills was a 10x programmer. He was prolific, and he was also productive. Yes, the system was replaced “some years later,” at which point the NYT replaced both the hardware and the software. If we discounted programmers’ work on every system that was replaced “some years later,” there would be no software projects anywhere that could be considered successful.

      For sake of argument, if you say that the customer replacing the hardware and software years later *does* reflect badly on the project, the bad reflection is on the *requirements* work on the project; it doesn’t reflect badly on Mill’s technical work.

      The 10x research is about differences in productivity in technical work. Mills did in fact write 83,000 lines of code in one year. That would be close to a 10x performance even today — and it was done in the late 1960s in a batch mode operating environment and on punch cards.

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.