I posted slides from my Eurostar Mobile Deep Dive presentation on slideshare.
I had a great time, and hope you find them useful. Let me know if you have questions.
I posted slides from my Eurostar Mobile Deep Dive presentation on slideshare.
I had a great time, and hope you find them useful. Let me know if you have questions.
Last night I dreamt about the worst presentation ever. Sometimes I was presenting, sometimes I was watching, but it was frighteningly bad. Fortunately, my keynote this morning went well – and now that it has, I’ll share what happened (including some conscious editing to make sure I cover all of the bases).
Moderator: I’d like to introduce Mr. Baad Prezenter. (resume follows)
Speaker: (taps on microphone – “is this on”). “Good Morning!” (when speaker doesn’t get the proper volume of answer, he repeats louder, “Good Morning!”. The audience answers louder and he thinks he’s engaged them now. “
Speaker then re-introduces himself repeating everything the moderator just told the room. After all, it’s important to establish credibility.
Speaker: “I’d like to thank Paul Blatt for telling me about this conference, Suzie Q for providing mentorship…” The list of thank-you’s goes on for a few minutes, before thanking the audience for attending his talk…even though many of them wish they hadn’t). Finally, the speaker moves on from the title slide to the agenda slide. He reads each bullet out loud for the audience members who are unable to read. He notices that one of the bullet points is no longer in the presentation and chooses that moment to talk about it anyway.
The next slide shows the phonetic pronunciation of the presenters main topic along with a dictionary definition. The presenter reads this slide, making sure to emphasize the syllables in the topic. It’s important that the audience know what words mean.
15 minutes in, and finally, the content begins.
The speaker looks surprised by the content on the next slide.
Speaker: “I actually wasn’t going to talk about this, but since the slide is up, I’ll share some thoughts.” The speaker’s thoughts consist of him reading the bullet points on the slide to the audience. His next slide contains a picture of random art.
Speaker: “This is a picture I found on the internet. If you squint at it while I talk about my next topic you may find that it relates to the topic, but probably not. But I read the presentations need pictures, so I chose this one! “
Speaker spends about 15 minutes rambling. It seems like he’s telling a story, but there’s no story. It’s just random thoughts and opinions. Some audience members wonder what language he’s speaking.
The moderator flashes a card telling him there’s 10 minutes left in his presentation
Speaker: “I’m a little behind, so let’s get going”. Finally, on the next slide are some graphics that look interesting to the audience and information that seems like it would support the topic. But the speaker skips this slide, and several more.
Speaker: “Ooh – that would have been a good one to cover – maybe next time” Finally the speaker stops on a slide that looks similar to one of the earlier slides.
Speaker (noticing that this slide is a duplicate): “I think I already talked about this, but it’s important, so I want to cover it.” Speaker reads the bullet points on the slide. At this point he hasn’t turned to face the audience in several minutes.
The next slide has a video of puppies chasing a raccoon that apparently has a lot to do with the topic. Unfortunately, the audio isn’t working, so the speaker stops the presentation and fiddles with the cable for a minute. Finally, he has audio, and restarts the presentation.
From the beginning.
He quickly advances to the video, stopping only once to talk about a slide he almost added that included an Invisible Gorilla and plays it for the audience. The audience stares blankly at the screen and wonders what drew them to this presentation in the first place.
Finally, the speaker gets to his last slide. It’s the same as the agenda slide, but…the bullet points are in ITALICS. He reads each bullet point again so he can tell them what they learned…or could have learned, thanks them, and sends them to lunch.
Twenty minutes late.
The audience are too dazed, and too hungry to fill out evaluation forms, so the speaker fills them out for them.
They loved him.
I’ve watched a lot of teams try to be more agile or more adaptive, or just move to faster shipping cadence. It has taken me a while, but I think I see a pattern, and the hard stuff boils down to two things.
Scope and Silos
Scope, in this context, is everything that goes into a feature / user story. For many developers in the previous century, this meant getting the thing to compile and sort-of work, and then letting test pound quality into it. That worked fine if you were going to spend months finding and fixing bugs, but if you want to ship every week, you need to understand scope, and figure out a way to deliver smaller pieces that don’t lower customer value.
Scope includes architecture, performance, tests, telemetry, review, analysis, interoperability, compatibility, and many, many other things beyond “sort-of works”. There may not be work to do for all of these items, but if you don’t consider all of them for all stories, you end up with an application where half of the features are incomplete in some way. If half of your application is incomplete, are you ready to ship?
The second “problem” I see – mostly in teams transitioning from predictive development models are silos (or strict adherence to the team ownership). You can find these teams by asking people what they do. They’ll say, “my team owns shimmery blue menus”, or “I own the front page”. When you have a team full of people who all own an isolated piece of the product, you very likely will have a whole lot of stories “in progress” at once, and you end up with the first problem above.
I’ve frequently told the story of a team I was on that scheduled (what I thought was) a disproportionate number of features for a release an a specific area. When I asked why we were investing so much in that area, I was told that due to attrition and hiring, that the team that owned the area was much larger than the other teams.
If you’re shipping frequently – let’s say once a week, you want every release to have more value to the customer than the previous release. Value can come from stability or performance improvements, or from new features or functionality in the product. On a team of twenty people, delivering twenty new stories is probably not a great idea. Failing to finish any of those stories and delivering no new functionality is worse.
So pick the top 3 (ish) stories, and let the team deliver them together. Forget about who reports to who, and who owns what. Figure out what’s most important for the customer, and enable the team to deliver value. Everyone may not be involved in delivering a story each release (there’s bound to be be fundamental, infrastructure, and other similar work that needs to be done). That’s ok – let the team self-organize and they’ll do the right thing.
In other words, I think a lot of improvement to be discovered by defining work better, and limiting work in progress. Not rocket science, but often forgotten.
This post is completely inspired by Trish Khoo’s post on Preparing for Your Presentation. I was going to add a comment, but it got too long, so it’s becoming a blog post. Go ahead and read that first – it covers way more than I’m covering here, and it’s a well written article.
Trish suggests starting early, and I can’t stress that enough – but there’s some flavor to the timeline that has worked consistently well for me. As soon as I know I’m doing the presentation, I make an outline. Sometimes I make the outline in powerpoint, but usually I start with Word (or notepad, or onenote). This helps me get my story together and give me an idea of what I want to say. I’ll add notes on what I want to research. I’ll read through it dozen times or so over a few days and add or edit as needed.
And then I’ll ignore it for at least a few weeks.
I don’t know if I can recommend this for everyone, but (assuming I’ve started early enough), during the few weeks away, my brain has subconsciously worked out a lot of the details. Whenever I come back to the outline, I immediately see obvious edits and areas to clean up. Usually this is the time I shove the outline into powerpoint and make a skeleton slide deck.
Trish also suggests nailing your intro and having one big message. For me, these are the same thing. At this point, I spend some time thinking about “the one thing” I want to get across. I not only figure out how I’ll work the message into my intro, but I’ll figure out how I repeat the message throughout the presentation. This also means that I usually find really “cool” material that I remove from the presentation because I can’t make a strong connection from the material to the message. It’s a tough decision, but it helps make the presentation clear.
The last thing I do is turn the text / bullet points from my slides into speaking notes and make the slides more about the concepts and ideas I’m talking about. I may use screen shots or stock photos – it all just depends. One word of warning though – I see a lot of people pull keywords from their slides into a search engine and grab whatever photo shows up. Beyond potential copyright issues, often the picture has nothing to do with the actual subject (e.g. if you’re talking about working with Red Hat Linux, by all means, don’t show a picture of a random red hat as your bullet-point-replacement).
From there, I tweak, tweak, and then tweak a little bit more. I know it drives conference organizers crazy, but the “draft” I deliver to them a month or two ahead of the conference is rarely what I present at the conference. Sometimes parts of the presentation just don’t “click” until late. Of course, it’s possible to over-tweak, but I’d much rather give the best presentation possible for the audience than match what I temporarily thought was complete a month or two ago.
The only thing not on Trish’s list that I want to add is that it’s really important to check out the room first. Try to watch at least one talk in the room you’re going to present in to get an idea of size (if it’s a long narrow room, take time to increase font size), or noises (so you won’t be as surprised if the kitchen is next door). Figure out in advance if you can put your laptop where you want, how you’ll pull off interactions, etc. As a last resort, if you can’t see another talk in the room, get there early, get set up, and get as much of a feel for the room as you can.
I’ll be fair. For keynote presentations, tutorials, and the like, I will always apply the above steps. It works for me, and I see no reason to change it. I think (hope?) it’s a reason I’m invited back to many conferences.
However, I give a lot of smaller talks (meetups, q&a sessions, etc.), and for those I prepare on a much lighter level – usually because I’m speaking on experiences or I’m confident I can wing it on the subject matter. It took a long time before I could pull this off, but I’m ok doing it now for some types of events.
I gave a talk at a meetup hosted by GoDaddy last Friday. I had a good time, and the comments afterwards were positive. More than one of the folks I talked with after the event asked why I don’t blog anymore. I have been a bit sparse in my blogging over the last few years, but I haven’t technically stopped….
Truth is that I spent a chunk of that time either on vacations, or in a job that wasn’t particularly interesting to me (work inspires blog posts). Of course, that excuse doesn’t hold up over the last three months where I’ve been in an extremely interesting role – just one that’s kept me extremely busy as well.
But, I was reminded on Friday, that even a few stories can strike a lot of inspiration and conversation with others, so I expect things will pick up – even if it’s just very short blogs posts.
Like this one.
I feel like today’s a good day to share a few stories about my first few months at Microsoft, and the (very) small part I played in shipping Windows 95.
My start at Microsoft is a story on its own, and probably worth recapping here in an abbreviated form. I started at Microsoft in January 1995 as a contractor testing networking components for the Japanese , Chinese, and Korean versions of Windows 95. I knew some programming and even a bit of Japanese (I later became almost proficient, but have forgotten a lot of it now). I also knew, for better or for worse, a lot about Netware and about hardware troubleshooting, and that got me in the door (and got me hired full time 5 months later).
Other than confirming that mainline functionality (including upgrade paths) were correct, there were two big parts of my job that were unique to testing CKJ (Chinese, Korean, Japanese) versions of Windows. The first was that at the time, there were a dozen or so LAN cards (this was long before networking was integrated onto a motherboard) that were unique to Japan, and I was (solely) responsible for ensuring these cards worked across a variety of scenarios (upgrades from Windows, upgrades from LanMan, clean installs, NetWare support, protocol support, etc.). One interesting anecdote from this work was that I found that one of the cards had a bug in its configuration file causing it to not work in one of the upgrade scenarios. Given the time it typically took to go to the manufacturer to make a fix and get it back we decided to make the fix on our end. Because I knew the fix (a one liner), I made the change, checked it in, and that one liner became the first line of “code” I wrote for a shipping product at Microsoft.
The other interesting part of testing CKJ Windows was that Windows 95 was not Unicode; it was a mixed byte system where some (most) characters were made up of two bytes. Each language had a reserved set of bytes specified as Lead Bytes, that indicated that that byte, along with the subsequent byte were part of a single double-byte character. Programs that parsed strings had to parse the string using functions aware of this mechanism, or they would fail. Often, we found UI where we could put the cursor in the middle of a character. The interesting twist for networking was that the second byte could be 0x7c (‘|’), or 0x5c (‘\’). As you can imagine, these characters caused a lot of havoc when used in computer names, network shares, paths, and files, and I found many bugs testing with these characters (more explanation on double-byte characters, along with one of my favorite related bugs is described here).
While I didn’t do nearly as much for the product as many people on the team who had worked on the product for years, I think I made an impact, and I learned so many things and learned from so many different people.
Readers of my blog know my stance on UI automation. But, as I’ve forgotten my StickyMinds password, and the answer is longer than 140 characters, so I’m responding here.
This article from Justin Rohrman talks about the coolness of Selenium for UI testing. In a paragraph called, “Why the UI”, Justin wrote:
The API and everything below that will give you a feel for code quality and some basic functionality. Testing the UI will help you know things from a different perspective: the user’s.
I like everything else in the article, but that second sentence kills me. Writing automated tests for the UI is as close to a user perspective as I am to the moon (I’m only on the 20th floor). I’m going to do Justin a favor and rewrite that paragraph for him here. Justin – if you read this, feel free to copy and paste the edit.
…some basic functionality. Testing the UI is difficult and prone to error, and automation can never, ever in a million years replace, replicate, or mimic a real users interaction with the software. However, sometimes it’s convenient – and often necessary to write UI automation for web pages, and in cases where that happens, Selenium is obvious choice.
Justin – your work is good – I just disagree (a LOT) with the trailing sentence of the paragraph in question.
Back to work for me…
I mentioned in my last post that I have a new job at Microsoft (and I discussed it a bit more on the last AB Testing). During the interviews for the job, I talked a lot about quality. I used the agile quadrants as one example of how a team builds quality software (including my roles in each of the quadrants), but I also talked about quality software coming from a pyramid of activities and processes. I’ve been dwelling on the model for the last week or so, and wanted to share it for comments and feedback…or to just brain-dump the idea.
The base of software quality (and my pyramid) is in the craftsmanship and approach of the team. Do they care about good unit testing and code reviews, or do they check in code willy nilly? Do they take pride in having a working product every day, or does the build fall on the floor for days on end? The base of the pyramid is critical for making quality software – but on established teams can be the most difficult thing to change.
An extension of PP&C above is code correctness. This is a more granular look specifically at code quality and code correctness. This includes attention to architecture, code design, use of analysis tools, programming tools, and overall attention to detail on writing great code.
Unit tests, functional tests, integration / acceptance tests, etc. are all part of product quality. I italicize, because for some reason, some folks think that quality ends here – that if the tests pass, the product is ready for consumers. (un?)Fortunately, readers of this blog know better, so I’ll save the soapbox rant for another day. However, a robust and trustworthy set of tests that range from the unit level to integration and acceptance tests is a critical part of building software quality.
There are some folks in software in the “Automate Everything” camp. A lot of testers don’t like this camp, because they think it will take away their job. Whatever.
As far as I can tell from my limited research on this camp, Automate Everything means automate all of the unit functional and integration tests…and maybe a chunk of the performance and reliability tests. For some definitions of “Everything”, I agree. Absolutely automate all of this stuff, and let (make) the developer of the code under test do it. The testers’ mind is much better put to use higher up the pyramid.
Performance, reliability, usability, I18N, and other non-functional requirements / ilities are what begins to take your product from something that is functionally correct to something that people may just want to use. Often, the ilities are ignored or postponed until late in the product cycle, but good software teams will pay a lot of attention to this part of the pyramid throughout the product cycle.
It doesn’t matter how much you kick ass everywhere else in the pyramid. If the customers don’t like your product, you made a shitty product. It may be a functionally correct masterpiece that passes every test you wrote, but it doesn’t matter if it doesn’t provide value for your customers. Team members can “act like the customer”, be an “advocate for the customer”, or flat out, “be the customer”, but I’ll tell you (for likely the twentieth time on this blog), as a member of the product team, you are not the customer! That said, this is the part of the pyramid where good testers can shine in finding the fit and finish bugs that cause a lot of software to die the death of a thousand paper cuts.
Now, if you do everything else in the pyramid well, you have a better shot at getting lucky at the top, but your best shot at creating a product that customers like crave is to get quantitative and qualitative feedback directly from your users. Use data collection to discover how they’re using the product and what errors they’re seeing, ask them questions (in person, or via surveys), monitor twitter, forums, uservoice, etc. to see what’s working (and not working), and use the feedback to adapt your product. Get it in their hands, listen to them, and make it better.
More to come as I continue to ponder.
In January of 1995, I began some contract work (testing networking) on the Windows 95 team at Microsoft. Apparently, my work was appreciated, because in late May, I was offered a full time position on the team.
My first official day as a full time Microsoft employee was June 5, 1995.
That was twenty years ago today!
I never (ever!) thought I would be at any company this long. I thought computer software would be a fun thing to do “for a while” – but I didn’t realize how much I’d enjoy creating software, and dealing with all of the technical and non-technical things aspects that come with it. I learned a lot – and even though my fiftieth birthday is close enough to see, I’m still learning, and still having fun – and that’s a good thing to have in a job.
I’ve had fourteen managers, and seventeen separate offices. I’ve made stuff work (and screwed stuff up) across a whole bunch of products. I’ve done a ton of testing, entered thousands of bugs, and written code that’s shipped in Windows, Xbox, and more (not bad for a music major who stumbled into software).
In a nice bit of coincidence, my twenty-year mark also is a time of change for me. After two years working on Project Astoria (look it up – it’s really cool stuff), it’s time for me to do something new at Microsoft…something that aligns more with my passions, skills, and experiences – and something that shows what someone with over two decades of software testing experience can do for modern software engineering.
I’ve joined (yet another) v1 product team at Microsoft. Other than a few contract vendors, the team of a hundred or so has no software testers. They hired me to be “the quality guy”. This set up could be bad news in many worlds, but my job is definitely not to try to test everything. Instead, my job is to offer quality assistance, help build a quality culture, assist in exploratory testing, and look at quality holistically across the product. I don’t know if any jobs like this exist elsewhere (inside or outside of Microsoft), but I’m excited (and a bit scared) of the challenge.
More to come as I figure out what I do, and what it means for me as well as everyone else interested in software quality.