{"id":973,"date":"2016-01-13T15:32:29","date_gmt":"2016-01-13T23:32:29","guid":{"rendered":"http:\/\/angryweasel.com\/blog\/?p=973"},"modified":"2016-01-13T15:32:29","modified_gmt":"2016-01-13T23:32:29","slug":"intelligence-and-insight","status":"publish","type":"post","link":"https:\/\/angryweasel.com\/blog\/intelligence-and-insight\/","title":{"rendered":"Intelligence and Insight"},"content":{"rendered":"

I have an after-work event tonight, and rather than leave my car in the garage overnight, I ran to work. Since I\u2019ve moved to downtown Bellevue, I\u2019ve done this a few times \u2013 and given that I\u2019m running another half-marathon in 10 days, it was a great opportunity for a long training run before I begin to taper my mileage down a bit leading up to the race.<\/p>\n

App Issues<\/h4>\n

I\u2019ve been a long-time user of a running companion app called Runtastic. It does the usual stuff of tracking mileage, route, pace, and giving voice updates at user specified intervals. I find it especially valuable when training or racing, because I usually have very specific pace goals. Having a voice tell me how long the last mile\/half-mile\/km took me let\u2019s me know if I\u2019m running too fast (and burning out) or too slow (and putting my goals in jeopardy). Granted, I have a pretty good internal clock, and usually run my paces pretty well withouth \u201chelp\u201d, but the feedback is really useful to me.<\/p>\n

Today, I took off from home, started spacing out, and before long, I was a mile or two from home\u2026when I noticed that I had not received any voice feedback yet. I knew exactly what happened (because it\u2019s happened before). When you start the app, it gives you a 15 second timer before the tracking actually starts, along with the ability to add time to the delay up to two minutes or so. I LOVE this feature, because I can give myself time to put my phone into my running belt, walk twenty yards and curse myself for having such a painful hobby before actually exerting any physical energy.<\/p>\n

\"Screenshot_2016-01-13-14-10-02\"<\/a><\/p>\n

Unfortunately, at least 25% of the time I use the app the countdown fails. But I never know<\/em> it failed until too late.<\/p>\n

What happens is that the countdown stops at 1 second. I hear the voice prompt count down 5-4-3-2-1, and I think<\/em> it\u2019s tracking, but it\u2019s stuck on one second.<\/p>\n

Today was extra painful not only because I wanted to see how I was doing on \u201crace pace\u201d, but after I discoverd it failed, I restarted the app, set the countdown again\u2026and it \u201chung\u201d on one second again. <\/p>\n

Grrrr\u2026<\/p>\n

Now, it\u2019s a bug \u2013 that\u2019s for sure. Some testers I know would automatically assume that every user in the world was hitting this bug and that public shaming of the company would be the next course of action. I, however, realize that context plays a role in many (every?)<\/em> part of software engineering, and that given the value I get from the product (and my very amateur level of running) that while this is a painfully annoying bug, it\u2019s not the end of the world.<\/p>\n

<\/h4>\n

Intelligence?<\/h4>\n

Given that I wasn\u2019t concerned at all with pace for the remaining 5+ miles of my commute, my mind began to wander. What follows is a completely made up story of how a Runtastic engineer may discover this issue without<\/em> me, or anyone else, reporting it.<\/em><\/p>\n

Pointy Haired Runtastic Manager:<\/strong> Hey super-smart employee (sse). Our default time for delay before a run is 15 seconds. Can you look at the data and see if our estimate for a default delay length is in the ballpark of what people actually use? Someone on the train told me that they though 30 seconds would be a lot better. I don\u2019t think they\u2019re right, but I want to make a decision based on data.<\/p>\n

Super Smart Employee (sse):<\/strong> Sure boss. That data is pretty easy to pull. I\u2019ll take a look!<\/p>\n

What SSE is about to do at this point is gather Business Intelligence. They want to use data to make a business decision. <\/em><\/p>\n

SSE looks at hundreds of thousands of activities from the past six months and sees that nearly 60% of the people just use the default 15 seconds. She quickly generates a scatter graph showing that shows the outliers and prepares it for her boss. Before sending, she realized that she wants to exclude the instances where people cancel the activity completely before the countdown completes (phone calls, cold feet (literally, and metaphorically), and a variety of other reasons could cause this). She filters the data and starts to send the report\u2026but \u2013 while she\u2019s there, she notices something\u2026interesting. First, the number of \u201ccancelled\u201d activities seem high to her (over 15%). She flips the filter to look only at cancelled activities and things get weirder. Of the 15% of cancelled activities, 90% are cancelled at exactly 1 second.<\/p>\n

That\u2019s too weird to be true.<\/p>\n

Insight<\/h4>\n

SSE looks at every activity where the timer was \u201ckilled\u201d at one second. Often, those users started another activity within 10-15 minutes.<\/p>\n

Or, maybe they all had the same model of phone.<\/p>\n

Or maybe they were all running Spotify at the same time.<\/p>\n

Or something. Remember. This story is completely made up.<\/p>\n

The point is that SSE quickly went from gathering BI to using discovery and insight to find a pretty cool bug. Using data!<\/p>\n

<\/h4>\n

Aside<\/h4>\n

I told a story at a conference recently about a team I worked on that used an offshore vendor team to run through a large number of applications for app compatability testing. We asked them to take notes and to send a report, but not to bother filing bug reports<\/u>.<\/p>\n

HuWhat?<\/p>\n

Yeah \u2013 we had sufficient telemetry and monitoring that we knew about all the bugs and glitches (and had collected call stacks and other helpful information) already. Many, in fact, that the test team didn\u2019t (or couldn\u2019t) notice. Entering the bugs would have been a waste of time. In the rare cases where something weird happened that we didn\u2019t track, we immediately added the appropriate instrumentation to track that class of failure in the future.<\/p>\n

I expect that for most of you, my world isn\u2019t your world. But in my world, data driven engineering is critical.<\/p>\n

Epilogue?<\/h4>\n

Since I don\u2019t know how made up my made up story really is, I\u2019m going to report it to Runtastic anyway. I can\u2019t predict the future (or anything else), but I hope the reply to my complaint is, \u201cYeah \u2013 we already knew about that. From the data\u201d.<\/p>\n","protected":false},"excerpt":{"rendered":"

I have an after-work event tonight, and rather than leave my car in the garage overnight, I ran to work. Since I\u2019ve moved to downtown Bellevue, I\u2019ve done this a few times \u2013 and given that I\u2019m running another half-marathon in 10 days, it was a great opportunity for a long training run before I…<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_kad_post_transparent":"","_kad_post_title":"","_kad_post_layout":"","_kad_post_sidebar_id":"","_kad_post_content_style":"","_kad_post_vertical_padding":"","_kad_post_feature":"","_kad_post_feature_position":"","_kad_post_header":false,"_kad_post_footer":false,"_kad_post_classname":"","_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[1],"tags":[],"class_list":["post-973","post","type-post","status-publish","format-standard","hentry","category-allposts"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_likes_enabled":true,"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/posts\/973","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/comments?post=973"}],"version-history":[{"count":0,"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/posts\/973\/revisions"}],"wp:attachment":[{"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/media?parent=973"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/categories?post=973"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/tags?post=973"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}