{"id":1387,"date":"2018-08-07T11:53:49","date_gmt":"2018-08-07T18:53:49","guid":{"rendered":"https:\/\/angryweasel.com\/blog\/?p=1387"},"modified":"2018-08-07T11:53:55","modified_gmt":"2018-08-07T18:53:55","slug":"getting-started-with-testproject","status":"publish","type":"post","link":"https:\/\/angryweasel.com\/blog\/getting-started-with-testproject\/","title":{"rendered":"Getting Started with TestProject"},"content":{"rendered":"

I wrote an article on the perils of Test Automation (Something\u2019s Rotten with Test Automation<\/a>) for TestProject<\/a> last year. I\u2019ve had a bit of time since then to look into their product since then (currently in beta), and thought it was worth sharing a bit about it, as I think it\u2019s a pretty good product.<\/p>\n

This post is a walk-through of my experience creating a simple web test. I plan to write a few follow-up posts to cover mobile device testing and finally a bit on my experiences playing with their Java SDK.<\/p>\n

Please note: I am accepting a small stipend from TestProject for posting this, but the article ideas and content are entirely mine, and I would not write this if I didn\u2019t endorse the product.<\/i><\/p>\n

What is TestProject?<\/h2>\n

On the surface, TestProject is yet-another web and mobile automation tool. It supports recorded tests (but better than any test recording frameworks I\u2019ve used before), as well as coded tests, which can be uploaded and run from the TestProject web site.<\/p>\n

Yet Another Automation Framework?<\/h2>\n

We\u2019ve had a recent resurgence of web automation frameworks, but for reasons I\u2019ll describe in this post – as well as in the next two follow-ups, TestProject has a lot of promise, and a lot of potential. It encapsulates Selenium and Appium (no separate download needed), will eventually support at least five programming languages, and makes it extremely simple to share and schedule tests.<\/p>\n

Better yet, the price<\/b> is right. For most users, it\u2019s FREE. The free package includes 5 agents, SDK support, 30 days of history, and some other goodies. For those who need the Professional version, it\u2019s $8 a month per agent – which is still ridiculously affordable.<\/p>\n

But…The Weasel Doesn\u2019t Like UI Automation<\/h2>\n

If you\u2019ve read my posts, seen my talks, or followed me on Twitter, you\u2019ve likely seen me speak out against UI automation. To be clear, I\u2019ve spoken out (and will speak out) against bad<\/i> UI automation, and against trying to do too much<\/i> testing in the UI.<\/p>\n

A few<\/i><\/span> targeted<\/i> tests that verify critical parts<\/i> of the UI are not just something I support, but a pretty damn good idea. I think TestProject is a good solution for this task for many teams.<\/p>\n

Recorded Tests – A Walkthrough<\/h2>\n

I have always been (very) leery of recorded tests, but somewhere when I wasn\u2019t paying attention, the tools got pretty good – or at least much better. It\u2019s still possible to create some pretty bad tests via recording tools, but I’ve learned that at least it\u2019s possible now to create reasonable tests with recording tools.<\/p>\n

To try things out, I made up a (slightly convoluted) test scenario where I wanted to make sure that all of the menu pages on angryweasel.com went to the right page.<\/p>\n

\"\"<\/p>\n

My \u201ctest plan\u201d is to click each link, and make sure it goes to where it\u2019s supposed to go (probably by verifying an expected element exists).<\/p>\n

Here\u2019s a quick walkthrough of the setup:<\/p>\n

Step 1 – Create a new project<\/h3>\n

\"\"<\/p>\n

Step 2 – Create a Test<\/h3>\n

Pretty straightforward – click the big blue \u201cNEW TEST\u201d button – select<\/p>\n

\"\"<\/p>\n

\"\"At this stage, I could go straight into \u201cDesign\u201d mode – but we\u2019ll go here later after we get the basics done via the recorder.<\/p>\n

The TestProject Agent<\/h2>\n

One thing I really like about TestProject is the TestProject agent – it\u2019s cross-platform (I\u2019m testing it now on my Ubuntu system), and it encapsulates Selenium or Appium as needed for the target platform. Once the agent is up and running, the TestProject web site will see it\u2019s running and connect.<\/p>\n

When the recorder page comes up, it looks (something) like this:<\/p>\n

\"\"<\/p>\n

I moved the recorder window so I could see the menu items, and then clicked them from right to left. That gives me a not-very-readable output like this:<\/p>\n

\"\"<\/p>\n

I can play this back, and it runs well. But at this point, there are two BIG problems. The first issue is that there\u2019s no validation. We\u2019ll add that in a few minutes, but first, I\u2019d like to document the \u201cClick A <Link>\u201d names and make them a little more debuggable.<\/p>\n

My preference would be to rename these, but that\u2019s not an option yet in TestProject – but to aid in debugging, I can at least add a comment.<\/p>\n

\"\"<\/p>\n

One cool thing to note here is that I can view and edit<\/i> the current selector, and select by ID, Classname, Tagname, Name or (the default) Xpath. Consistently named labels are essential for writing maintainable automation, so this is something that I think will be pretty handy.<\/p>\n

I won\u2019t show it here, but I ended up changing the selection method for each validation, and all worked exactly as I expected.<\/p>\n

Validation<\/h3>\n

I found creating the validation step to be pretty straightforward too. Just create a new test step, and have it verify that a specific element contains specific text. For those of us who have written coded web automation, it\u2019s pretty easy to picture what Selenium code is running under the hood. I also think that this is indirectly a pretty reasonable way for non-coders to learn how Selenium works. Probably not TestProject\u2019s intent, but I think there\u2019s potential there.<\/p>\n

\"\"\"\"<\/p>\n

I can run my tests from here, and the green bar on the left side shows that all of my test steps passed.<\/p>\n

\"\"<\/p>\n

From here, I created a Job on the TestProject site, set the job to run on Chrome and Firefox, and added this test to the job.<\/p>\n

Test Results\"\"\"\"<\/h3>\n

Not super exciting results, but certainlynot bad results for 30 minutes of \u201ctest development\u201d with a new tool.<\/p>\n

Parameters<\/h3>\n

I mentioned above that I gave every action a one second timeout (note that it doesn\u2019t wait or sleep<\/i> a second – one second is just the length of time I\u2019ve instructed the framework to wait for an element.<\/p>\n

What happens if I decide the timeout should be 2 seconds -or half a second. It\u2019s preferrable to change things like this in one place, and fortunately, TestProject allows you to add parameters. I added one called my_timeout<\/b>, and now, if I ever need to change it, I can change the timeout in once place.<\/p>\n

\"\"<\/p>\n

Failures<\/h3>\n

When you write tests, it\u2019s a good idea to test what happens when they fail. I changed the \u201cValidate \u201cAbout Angry Weasel\u201d test to look for the test \u201cAbout Angry easel\u201d. As expected, the test failed, and gave me reasonably actionable information to debug the failure from.<\/p>\n

\"\"<\/p>\n

Final Notes (for now)<\/h3>\n

Overall, I didn\u2019t find much to hate. Some things I\u2019d like to see in future updates include:<\/p>\n

    \n
  • Ability to rename steps for easier debugging<\/li>\n
  • There are a lot of \u201care you sure you want to do this\u201d type dialogs that I already feel a need to suppress<\/li>\n
  • Given that I can mentally \u201csee\u201d what the underlying code looks like, it would be nice to generate code as well for more subtle tweaks.<\/li>\n<\/ul>\n

    As I said in the intro, I have a few more posts planned to walk through more of what I learn while learning the tool, but happy to hear your feedback on the tool or on my approach in using it if you have them.<\/p>\n","protected":false},"excerpt":{"rendered":"

    I wrote an article on the perils of Test Automation (Something\u2019s Rotten with Test Automation) for TestProject last year. I\u2019ve had a bit of time since then to look into their product since then (currently in beta), and thought it was worth sharing a bit about it, as I think it\u2019s a pretty good product….<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"image","meta":{"_kad_post_transparent":"","_kad_post_title":"","_kad_post_layout":"","_kad_post_sidebar_id":"","_kad_post_content_style":"","_kad_post_vertical_padding":"","_kad_post_feature":"","_kad_post_feature_position":"","_kad_post_header":false,"_kad_post_footer":false,"_kad_post_classname":"","_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[1],"tags":[],"class_list":["post-1387","post","type-post","status-publish","format-image","hentry","category-allposts","post_format-post-format-image"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_likes_enabled":true,"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/posts\/1387","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/comments?post=1387"}],"version-history":[{"count":5,"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/posts\/1387\/revisions"}],"predecessor-version":[{"id":1409,"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/posts\/1387\/revisions\/1409"}],"wp:attachment":[{"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/media?parent=1387"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/categories?post=1387"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/angryweasel.com\/blog\/wp-json\/wp\/v2\/tags?post=1387"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}