- Jerry Weinberg passed away earlier this week, and I’ve taken some time to re-read parts of many of his books, where I’ve been constantly reminded of his leadership knowledge. I grabbed this quote from Becoming a Technical Leader to share
“People don’t become leaders because they never fail. They become leaders because of the way they respond to failure.”
- I spent a chunk of my early career in globalization testing (which reminds me that my last wikipedia edit was to fix this section of the software testing article). Netflix, in yet another valuable post, describe their approach to pseudo-localization.
- We all knew this was coming. Here’s what you need to know about California’s new data privacy laws.
- Modern Testing Principles overlap a reasonable amount with the Three Ways of Devops – which is probably why I found this article on the Three Ways of DevSecOps interesting.
- And finally, speaking of security, here’s something to remind you how much we all suck at software security. How I gained commit access to Homebrew in 30 minutes.
I posted this other places, but forgot to put it here. If you, or someone you know is interested an capable of embracing and driving Modern Testing principles with a fantastic group of developers, producers and other technical staff, and are in (or open to moving to) Austin, Texas, check out or share this position on my team.
Apply on the site but drop me a note on twitter (@alanpage), or email me (alan at angryweasel.com) if you apply, and I’ll fast track the application.
I wrote an article on the perils of Test Automation (Something’s Rotten with Test Automation) for TestProject last year. I’ve had a bit of time since then to look into their product since then (currently in beta), and thought it was worth sharing a bit about it, as I think it’s a pretty good product.
This post is a walk-through of my experience creating a simple web test. I plan to write a few follow-up posts to cover mobile device testing and finally a bit on my experiences playing with their Java SDK.
Please note: I am accepting a small stipend from TestProject for posting this, but the article ideas and content are entirely mine, and I would not write this if I didn’t endorse the product.
What is TestProject?
On the surface, TestProject is yet-another web and mobile automation tool. It supports recorded tests (but better than any test recording frameworks I’ve used before), as well as coded tests, which can be uploaded and run from the TestProject web site.
Yet Another Automation Framework?
We’ve had a recent resurgence of web automation frameworks, but for reasons I’ll describe in this post – as well as in the next two follow-ups, TestProject has a lot of promise, and a lot of potential. It encapsulates Selenium and Appium (no separate download needed), will eventually support at least five programming languages, and makes it extremely simple to share and schedule tests.
Better yet, the price is right. For most users, it’s FREE. The free package includes 5 agents, SDK support, 30 days of history, and some other goodies. For those who need the Professional version, it’s $8 a month per agent – which is still ridiculously affordable.
But…The Weasel Doesn’t Like UI Automation
If you’ve read my posts, seen my talks, or followed me on Twitter, you’ve likely seen me speak out against UI automation. To be clear, I’ve spoken out (and will speak out) against bad UI automation, and against trying to do too much testing in the UI.
A few targeted tests that verify critical parts of the UI are not just something I support, but a pretty damn good idea. I think TestProject is a good solution for this task for many teams.
Recorded Tests – A Walkthrough
I have always been (very) leery of recorded tests, but somewhere when I wasn’t paying attention, the tools got pretty good – or at least much better. It’s still possible to create some pretty bad tests via recording tools, but I’ve learned that at least it’s possible now to create reasonable tests with recording tools.
To try things out, I made up a (slightly convoluted) test scenario where I wanted to make sure that all of the menu pages on angryweasel.com went to the right page.
My “test plan” is to click each link, and make sure it goes to where it’s supposed to go (probably by verifying an expected element exists).
Here’s a quick walkthrough of the setup:
Step 1 – Create a new project
Step 2 – Create a Test
Pretty straightforward – click the big blue “NEW TEST” button – select
At this stage, I could go straight into “Design” mode – but we’ll go here later after we get the basics done via the recorder.
The TestProject Agent
One thing I really like about TestProject is the TestProject agent – it’s cross-platform (I’m testing it now on my Ubuntu system), and it encapsulates Selenium or Appium as needed for the target platform. Once the agent is up and running, the TestProject web site will see it’s running and connect.
When the recorder page comes up, it looks (something) like this:
I moved the recorder window so I could see the menu items, and then clicked them from right to left. That gives me a not-very-readable output like this:
I can play this back, and it runs well. But at this point, there are two BIG problems. The first issue is that there’s no validation. We’ll add that in a few minutes, but first, I’d like to document the “Click A <Link>” names and make them a little more debuggable.
My preference would be to rename these, but that’s not an option yet in TestProject – but to aid in debugging, I can at least add a comment.
One cool thing to note here is that I can view and edit the current selector, and select by ID, Classname, Tagname, Name or (the default) Xpath. Consistently named labels are essential for writing maintainable automation, so this is something that I think will be pretty handy.
I won’t show it here, but I ended up changing the selection method for each validation, and all worked exactly as I expected.
I found creating the validation step to be pretty straightforward too. Just create a new test step, and have it verify that a specific element contains specific text. For those of us who have written coded web automation, it’s pretty easy to picture what Selenium code is running under the hood. I also think that this is indirectly a pretty reasonable way for non-coders to learn how Selenium works. Probably not TestProject’s intent, but I think there’s potential there.
I can run my tests from here, and the green bar on the left side shows that all of my test steps passed.
From here, I created a Job on the TestProject site, set the job to run on Chrome and Firefox, and added this test to the job.
Not super exciting results, but certainlynot bad results for 30 minutes of “test development” with a new tool.
I mentioned above that I gave every action a one second timeout (note that it doesn’t wait or sleep a second – one second is just the length of time I’ve instructed the framework to wait for an element.
What happens if I decide the timeout should be 2 seconds -or half a second. It’s preferrable to change things like this in one place, and fortunately, TestProject allows you to add parameters. I added one called my_timeout, and now, if I ever need to change it, I can change the timeout in once place.
When you write tests, it’s a good idea to test what happens when they fail. I changed the “Validate “About Angry Weasel” test to look for the test “About Angry easel”. As expected, the test failed, and gave me reasonably actionable information to debug the failure from.
Final Notes (for now)
Overall, I didn’t find much to hate. Some things I’d like to see in future updates include:
- Ability to rename steps for easier debugging
- There are a lot of “are you sure you want to do this” type dialogs that I already feel a need to suppress
- Given that I can mentally “see” what the underlying code looks like, it would be nice to generate code as well for more subtle tweaks.
As I said in the intro, I have a few more posts planned to walk through more of what I learn while learning the tool, but happy to hear your feedback on the tool or on my approach in using it if you have them.
A few episodes ago on AB Testing, we talked about the Theory of Constraints and related topics. A lot of us frequently see “QA” or “Test” as a bottleneck, even on Agile teams – where several changes may be waiting for signoff from the designated quality or testing specialist before being deployed or marked as done.
If you listen to the podcast, you know I’m against this level of specialization. I’m all for people who have specialization – but things fall apart if they’re the only one who can do a “thing”. I like to imagine the extreme case of specialization where one person just writes design docs, one person writes code, but only internal functions; because another person writes external APIs, and another person tweaks all of the code for performance, and another person writes test cases, and another person runs them; unless they’re automated, then it’s another person…
At this point some of you are laughing because this sounds ridiculous, but others are thinking, “that’s pretty close to my current team – wtf?”
Today, I made a quick (or at least I planned for it to be quick) run to Home Depot to pick up a part. I knew exactly where the part was (aisle 40 – don’t ask how I knew that), so I jogged back, grabbed the part, and went to the front. I jumped in the self checkout line, scanned the part, shoved my credit card in the slot, and got ready to grab my receipt.
But the price was wrong. Or it was right, technically, but I was charged twice. Luckily, there was a cashier right there, so I explained that I scanned once, but was charged twice.
She stared for a moment, and said, “I can’t fix that. You’ll need to go to customer service.” Apparently, I needed the specialist. Or another specialist at least.
I went down to the customer service desk, where I was behind a customer with a cart full of stuff and a lot of questions. I waited 10 minutes for my meeting with the specialist, but it wasn’t moving very quickly, so I walked out of the main store area to the Returns counter to ask the returns specialist about it (the Returns counter is past the check out area to make (I assume) it easier to return stuff without entering the store.
Eventually, the Returns Specialist was able to help me, but he had a very difficult time. Apparently, he knew how to do returns, but since I wasn’t technically returning anything he thought it seemed like a problem that Customer Service should help with.
I try to be a quick thinker, and since my quick trip to the store was pushing twenty minutes in-store time, I grabbed my receipt and part, jogged back to aisle 40 to grab a duplicate part, returned to the counter with both, and I was able to return the item I didn’t want, and all was well.
On one hand, this is just another story of customer service gone south, but at the core, I see it as a flaw of too much specialization – and I think it’s important that we all recognize when we see these things around us. If your software team is ever blocked by a specialist – whether it’s waiting for a tester, waiting for a decision, or even waiting for a deployment, there’s a lesson to be learned, and a potential improvement to be made.
Time again to share a few things I found interesting this week.
- I’ve been reading Bad Blood (currently about 30% of the way through). So far, it’s a pretty damning investigative review of Theranos, but I’m still waiting for the plot twist :}. Wikipedia tells me that Jennifer Lawrence will star in the film adaptation.
- Steve Denning is at it again with this article on Agile Is Not Just Another Management Fad.
- A wonderful article on “Delivery Teams” from Janet Gregory.
- This (yet another) article on the failure of open plan offices is making the rounds. Buried in the end of the article is this quote:
“And probably least bad is small team rooms of fewer than ten people, preferably fewer than six. I’ve sat together with really small teams before and that’s been OK. Some people who don’t like the open office at all might even still enjoy this configuration.”
This setup (a “team room”) has been, in my experience the hands down winner of the most productive work environment I’ve ever worked in – and I hate that it’s lumped into the valid arguments that go against barnyard style work seating.
- Finally, a nice article on critical thinking and problem solving that I’ll probably re-read and refer to frequently.
It was a busy week, but my list of potential links was long. Here are 5 of the things I came across this week I felt were worth sharing.
- First off, PNSQC published a small interview with me in promotion of my talk at their conference in October. Alan Page on Tools, Technical Skills, and Modern Testing
- In case you were under a rock and missed it, this link on Testing in Production the Netflix Way includes both a summary, a transcript, and a video of Nora Jones (I don’t think this is the first time I’ve linked to one of her talks). If you do anything with services or the web, you should check it out.
By the way – the tech world needs more twenty-minute talks. The hour long presentation should be the exception, and not the norm. Someone remind me to write a full blog on this someday.
- This article on The changing role of QA managers in the Agile environment shouldn’t be a surprise to anyone who follows me, but sharing anyway so you can share with those less enlightened.
- Sadly, this article on This is what fear in leadership looks like captured the motivations of too many bad managers I’ve worked with (and for).
- And finally, another article from one of my heroes, Steve Denning that is fascinating, yet haunting. How Agile Helped Elect Donald Trump
Another Friday list to share:
- I finished off a few books while flying this week, including Lean Change Management, which is one of those books that makes you want to read a bunch more books. It’s full of models and references, and, given that so much of my role is about leading change and helping others lead change, is probably something I’ll keep close at hand.
- Ronan Mehigan sent me a link to an article on Specialists vs. Generalists. I like the article, but it also reminds me why I like the notion of Generalizing Specialists and Specializing Generalists so much.
- Maybe I’m late to the party, but I just discovered dev.to. I’ve found a nice handful of articles there, including the following bullet…
- I’m relatively new (just a few years) to Mac, and very new (a few months) to using Ubuntu as my daily machine, so I really appreciated The Shell Introduction I Wish I Had.
- Given the state of US politics and media, I generally defer to politifact.com or reuters.com for a (hopefully) more neutral view, but part of my daily reading continues to be the wonderful daily summary on What The Fuck Just Happened Today
- I try to remember that a big part of my job is growing leaders – and that I need to always try to improve as a leader myself. Most times when I need a little inspiration for reflection, I turn to Simon Sinek, and re-read Together is Better. Here’s one of the quotes that caught my eye this week.
When we tell people to do their jobs, we get workers. WHen we trust people to get the job done, we get leaders.
- It’s double-quote Friday. Harlan Ellison passed away earlier this month. He was a fantastic, and creative writer, with a lot of passion. One quote of his that has entered my mind a lot as I watch various bits of US politics is this one.
“You are not entitled to your opinion. You are entitled to your informed opinion. No one is entitled to be ignorant.”
- This article on moving from scrum to kanban captures a lot of my own feelings on the subject.
- Just when you thought it was safe to love open source, this happens (Compromise in ESLint)
- In addition to these weekly posts, I write a lot. Internal newsletters, articles for others (and in the past, books and book contributions). Writing has been a big part of my growth, and this article on why writing is important is worth a read for any lifelong learner.
Summer’s here, but good stuff keeps on happening. Here are 5 things I found interesting this week:
- Chris George wrote a great article on how he’s applying Modern Testing at his job.
- Not a new article, but new to me. Why being right doesn’t matter.
- Also not new, but something I watched this week and recommend is this Ted Talk from Atul Gawande (author of the Checklist Manifesto and more) on why you should get a coach.
- This. Is. So. Bad. Mining public passwords.
- Finally, the FIFA World Cup has been hugely entertaining this year, and barring a continued run of form for France (I’m cheering for France, so they’re bound to lose), we may have a first time champion. Even more interesting, this is the first time EVER where one of Germany, Brazil, or Argentina are not in the semi-final. (Wikipedia link)
Here’s what I found interesting this week:
- I’ve been thinking a lot about change recently – and I’m reminded of this quote from Peter Drucker:
The greatest danger in times of turbulence is not the turbulence – it is to act with yesterday’s logic
- I’m big on change models – and while I’ve known about it for some time, I’ve been finding a lot of value in the ADKAR model from Prosci a lot recently.
- Not sure why I didn’t know about this before, but this model on agile fluency hits a lot of the right buttons for me.
- Yet another article on manager readmes with even more readmes!
- I absolutely love the world cup. My prediction (as mentioned last time) is blown, as Germany failed to advance. But – I’m quite happy to see Japan advance. Especially interesting is that they are the first team ever to advance on the fair play rule.