Experimenting with Mobile Automation

As I type this, I realize I have a mildly embarrassing gap in my testing career. I don’t recall ever writing any automation for mobile applications.

Maybe that’s not exactly true – nearly 20 years ago, I wrote some automation in C++ to test Windows CE, and some applications that ran on it…and technically that was an OS used to power Windows Phone for a while (among other devices) – so I guess the gap is that I have never played with Appium. We used Appium when I worked on MS Teams for (some?) mobile automation, but I wasn’t directly involved.

Today is my chance.

TestProject Experiment Number Two

After my success playing with recorded test automation (I really want to call this recorded test assistance) for web, I decided to try the same thing using TestProject to test a mobile application.

Getting Started – The Setup

I was mentally prepared to install the Android SDK (or at least ADB), and spend a bunch of time getting a connection set up, but holy crap – was it always this easy?

I already have the TestProject agent running on my computer (for running the web tests), so all I had to do was select Android as my target, and everything else just worked.

Overall, the setup wizard is extremely easy to follow, and worked amazingly well. My phone was already in developer mode, so all I had to do was configure USB debugging, (optionally) disable a few animation features (for testability), and the TestProject web app connected to my phone and showed a mirror on screen in seconds. I thought for sure that I’d spend an hour or more getting this going, but I was wrong.

Granted – I don’t have much experience here other than half a dozen sessions debugging over ADB, so this sort of thing may be the norm now, so all I can say is setting up a remote development session can’t get much better than this.

The wizard is pretty much the same as the web wizard with one cool addition. Once the connection is configured, I can browse the device packages to choose which one I want to test:

I decided it would be fun to play with some automation for the Twitter app. There’s an option to reset the app before starting, and there’s even a default step added to reset, but for my experiment, I deleted that step.

As in my tests on angryweasel.com, I’m just going to record selecting the top 4 elements from right to left and see what happens.

Note that the steps here are able to pull more friendly names from the package, so the test steps are pretty readable as is.

Creating Tests

Using the same test strategy as I did with web tests, I then added validation to ensure that the correct screen showed in the right place.

For each screen, I just used the “Validations” design element to confirm the existence of a particular control. There are probably better ways to do this, but for my quick test, it’s fine.

Similar to what I wrote for web tests, I have some actions with validation.

The Element Locator

If you’ve written automation in Appium or Selenium (or other frameworks that rely on IDs, XPaths, etc.) you are already familiar with the game we play to find elements.

One of the other handy features of TestProject is a simple, but useful element locator that makes it easy to discover if the element you’re working with is currently visible. In the example below, I’ve clicked the Find button in the test designer to show me the Twitter Home button.

Pretty cool!

As a nice aside, notice that the screenshot above (unlike the other screenshots), my phone screen fills the emulator (vs. the other screenshots where it fills 70% or so of the emulator screen). The TestProject team fixed the issue practically moments after I reported it.

A Few Words About Automation

Let me be the first to point out that this isn’t particularly great automation, and that the validation I’m doing probably isn’t what needs to be tested at the UI level for Twitter.

When you write UI automation, use it to find bugs that can only be found at the UI level. If it’s a special visual effect that can only be detected via human eyes, then use human eyes to validate the effects.

Of equal importance is your strategy. Don’t start by deciding what to automate. Instead, start with what you want to test – and then use automation to assist your testing. I’ll refer again to my article on the TestProject site that expands on these ideas in more depth.

All that said, it seems to me that I could very well use TestProject for proper targeted UI tests where they were needed, and that it would serve me well.

Observations

I’m still surprised by how easy it was to get up and running testing a mobile app, and the test steps were pretty straightforward and intuitive.

One of the areas where mobile automation really helps is in making sure applications work across a variety of devices. I only have one phone, but TestProject does support working with multiple connected devices to a single desktop. You just connect a USB Hub with a suite of devices, and you have an instant test lab. It’s also worth mentioning that an account can have multiple agents assigned to it. This enables executing tests not only on your desktop (and devices connected to it), but also on remote agents (and devices connected to them) in different locations anywhere in the world. It could be fun to see if TestProject could target a big virtual and real mobile test farm like BitBar, but there’s plenty of flexibility and potential right out of the box.

As with the web tests, I’d love to see a way to view / edit the underlying code, but my curiosity was less with mobile apps – perhaps because my knowledge of Appium is much less than my knowledge of Selenium.

Overall, it seems like a pretty usable and robust choice for most mobile automation tasks.

One more investigation to go where I’ll see how TestProject does with some coded tests.

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.