{"id":175,"date":"2010-08-11T11:11:54","date_gmt":"2010-08-11T18:11:54","guid":{"rendered":"http:\/\/angryweasel.com\/blog\/?p=175"},"modified":"2010-08-11T11:11:54","modified_gmt":"2010-08-11T18:11:54","slug":"code-coverage-with-manual-testing","status":"publish","type":"post","link":"https:\/\/angryweasel.com\/blog\/code-coverage-with-manual-testing\/","title":{"rendered":"Code Coverage with Manual Testing"},"content":{"rendered":"

Here\u2019s a tip. Want me to write about something in particular? Ask me a question. It\u2019s that easy. Tim Coulter<\/a> had a question about my last blog post<\/a>. He asked:<\/p>\n

\n

Would love to hear how you check code coverage of manual testing, actually. Is that just \u201cfeature coverage\u201d (break it up into logical blocks and test those)?<\/p>\n<\/blockquote>\n

I realized that a lot of people may not be measuring code coverage during manual testing \u2013 but it\u2019s quite valuable to do so, as measuring code coverage still helps you find missing tests.<\/em><\/p>\n

Let\u2019s consider a silly command line app that does math operations. It takes 3 parameters, the first two are the values to operate on, and the third parameter is the operator to execute. For example, a command line of 2 3 + would return a value of 5. Let\u2019s test it (manually).<\/p>\n

\n
c:\\example>calul8r.exe\nUsage:\ncalcul8r [value] [value] [operation]\nfor example:\n  calcul8r 4 2 +\n\nc:\\example>calul8r.exe 2 3 +\nThe result of 2 + 3 is 5\n\nc:\\example>calul8r.exe 4 2 -\nThe result of 4 - 2 is 2\n\nc:\\example>calul8r.exe 5 5 *\nThe result of 5 * 5 is 25\n\nc:\\example>calul8r.exe 15 3 \/\nThe result of 15 \/ 3 is 5<\/pre>\n
<\/div>\n
 <\/p>\n
OK \u2013 looks like it sort of works. But are there test cases we forgot? Let\u2019s look at what code coverage tells us.<\/p>\n
\n
1 int<\/span> DoMath(<\/span>int<\/span> val1, <\/span>int<\/span> val2, <\/span>int<\/span> valOperator)<\/span><\/p>\n
2 {<\/span><\/p>\n
3       <\/span>int<\/span> result = <\/span>0xbaadf00d<\/span>;<\/span><\/p>\n
4       <\/span>switch<\/span> (valOperator)<\/span><\/p>\n
5       {<\/span><\/p>\n
6       <\/span>case<\/span> <\/span>‘+’<\/span>:<\/span><\/p>\n
7             result = val1 + val2;<\/span><\/p>\n
8             <\/span>break<\/span>;<\/span><\/p>\n
9       <\/span>case<\/span> <\/span>‘x’<\/span>:<\/span><\/p>\n
10      <\/span>case<\/span> <\/span>‘*’<\/span>:<\/span><\/p>\n
11            result = val1 * val2;<\/span><\/p>\n
12            <\/span>break<\/span>;<\/span><\/p>\n
13 <\/span><\/p>\n
14      <\/span>case<\/span> <\/span>‘\/’<\/span>:<\/span><\/p>\n
15      <\/span>case<\/span> <\/span>‘\\\\’<\/span>:<\/span><\/p>\n
16            <\/span>if<\/span> (val2 ==<\/span>0<\/span>)<\/span><\/p>\n
17            {<\/span><\/p>\n
18                  printf(<\/span>"Invalid input. Cannot divide by zreo"<\/span>);<\/span><\/p>\n
19            }<\/span><\/p>\n
20            <\/span>else<\/span><\/p>\n
21            {<\/span><\/p>\n
22                  result = val1 \/ val2;<\/span><\/p>\n
23            }<\/span><\/p>\n
24            <\/span>break<\/span>;<\/span><\/p>\n
25 <\/span><\/p>\n
26      <\/span>case<\/span> <\/span>‘-‘<\/span>:<\/span><\/p>\n
27            result = val1 – val2;<\/span><\/p>\n
28            <\/span>break<\/span>;<\/span><\/p>\n
29 <\/span><\/p>\n
30      <\/span>default<\/span>:<\/span><\/p>\n
31            printf(<\/span><\/span>"Unknown oprator – %d\\n"<\/span>);<\/span><\/p>\n
32            <\/span>break<\/span>;<\/span><\/p>\n
33      }<\/span><\/p>\n
34      <\/span>return<\/span> result;<\/span><\/p>\n
35 }<\/span><\/p>\n<\/div>\n
The \u201cmeat\u201d of this app is in the DoMath function, and the results show that I missed a few lines of code in my first round of tests. The first two lines I missed (lines 9 and 15) are a bit of discovery. It looks like the app can take the letter x as well as an * (asterisk) for multiplication, and a backslash (\\) as well as a forward slash (\/) for division. The functionality is the same, but it\u2019s interesting nonetheless.<\/p>\n
Oh \u2013 but I was stupid and forgot to test for divide by zero. It\u2019s nice that there is some error handling for that, but if you look closer, you\u2019ll see that executing this particular test would reveal a typo in the output string.<\/p>\n
I also forgot to test with an invalid operator \u2013 and wouldn\u2019t you know it, there\u2019s another typo there. What I\u2019ve done is used the code coverage data to guide my design for new tests. This app is simple, but the idea works on a larger scale too. I\u2019ve known testers to discover large areas of functionality they weren\u2019t aware of before, or (more often) hidden functionality within the areas they had tested extensively. <\/p>\n
An important thing to remember is that once I\u2019ve added those test cases (and reported those bugs), is that I\u2019ll have 100% code coverage. Remember though, that 100% code coverage doesn\u2019t mean bug free. In fact, this app doesn\u2019t check for integer overflow. Sure enough, check this out:<\/p>\n
\n
c:\\example>calul8r.exe 2147483647 1 +\nThe result of 2147483647 + 1 is -2147483648<\/pre>\n
<\/div>\n
Oops! Remember \u2013 just because the code is \u201ccovered\u201d, it doesn\u2019t mean it\u2019s tested. Please don\u2019t forget that (and don\u2019t let your managers forget either).<\/p>\n
Thanks Tim, for prompting this post \u2013 I hope you (and others) find it helpful.<\/p>\n","protected":false},"excerpt":{"rendered":"
Here\u2019s a tip. Want me to write about something in particular? Ask me a question. It\u2019s that easy. Tim Coulter had a question about my last blog post. He asked: Would love to hear how you check code coverage of manual testing, actually. Is that just \u201cfeature coverage\u201d (break it up into logical blocks and…<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_kad_post_transparent":"","_kad_post_title":"","_kad_post_layout":"","_kad_post_sidebar_id":"","_kad_post_content_style":"","_kad_post_vertical_padding":"","_kad_post_feature":"","_kad_post_feature_position":"","_kad_post_header":false,"_kad_post_footer":false,"_kad_post_classname":"","_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[1],"tags":[],"class_list":["post-175","post","type-post","status-publish","format-standard","hentry","category-allposts"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_likes_enabled":true,"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/posts\/175","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/comments?post=175"}],"version-history":[{"count":0,"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/posts\/175\/revisions"}],"wp:attachment":[{"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/media?parent=175"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/categories?post=175"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/tags?post=175"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}