How can I test my responsive mobile app - Part 2

The manual testing series #

In this series I’ll be uncovering the various techniques and tools that can help you get started to manually test a mobile web or hybrid app. Manual testing is a great way for verifying layouts and basic functionality. Automated tests afford us more verification but require a different set of tools and deeper understanding that’s not part of this series.

In [part 1](how-can-i-test-my-responsive-mobile-app-part-1), we tested with online and browser plug-ins to capture bugs.

Part 2, delves into testing on emulators specifically for mobile web apps running from the web. We’ll cover,

Embracing devices #

The layout of a page is influenced by more than just the width settings assigned to a media query. Testing on an emulator or on a physical device can highlight hiccups caused by how a specific version of a mobile browser renders an element before applying a media query. When you stick to basic tests you can never be sure how your app will truly render and function. More often than not, it will definitely be broken.

Also, there’s a breed of mobile apps like Forecast.io that render different layouts by detecting whether a user is visiting from a mobile browser. For these, emulators or devices are the only testing solution.

Hybrid mobile apps add their own complication, some have device dependencies that stop us dead in our tracks when testing on a desktop browser.

Emulators versus physical devices #

Emulators are great for sanity checks. They’re brilliant for quick feedback loops that can optimise your work-flow. Moreover, they give you access to a browser environment that is a one to one match with your target device. Bugs encountered here are not an approximation - these are real issues.

But, watch out! An emulator will mostly be running on either a server or a desktop machine. This gives it far more resources than you’ll ever get on the physical device. It allows emulators to be more forgiving. I have caught many a bugs on a physical device that were never obvious on emulator tests.

I want emulators #

If you’re working on a mobile web app, you might not have any emulators set up at all. It all depends on your development environment and the resources available. But its easy to get set up. Although, your access might be restricted to a smaller range.

Cloud based testing sites offer a wider range of emulators. Its very quick to get going and legacy emulators will always be available - something very hard on a personal development machine.

Let’s start by looking at cloud based emulators and then plunge into running our own.

Emulators on the cloud #

Some of the services include BrowserStack, Sauce Labs and Cross Browser Testing to name a few. They’re essentially the same, but provide different UI testing interfaces. My personal preference is BrowserStack but feel free to use any solution you (or your pocket) prefers.

Starting With Screen-shots #

You can start things off quick and dirty. Just take a screen-shot of a single screen across all available devices.

BrowserStack offers this,

It can take a few seconds. BrowserStack loads Up App’s homepage on a super huge range of mobile browsers, all processing screenshots and then displaying them.

Since I haven’t fixed the issues from part 1, you can see these cropping up again with the iPad 3 on portrait mode.

But there are other peculiarities, the animated sun illustration is broken on all Android devices from the 2.1 Droid Razr (320x640) to the 4.1 Nexus 7 (603x966). The layout renders well while the illustration positioning is slightly off or in case of the Droid Razr the illustration is bigger than it needs to be.

There’s great value in this exercise. Very quickly, we’ve realised bugs that were in no away apparent from simple plug-in or online testing.

In fact, here’s a screen-shot using the Droid Razr dimensions as rendered by the Viewport-Resizer browser plug-in. According to the plug-in, there are no issues whatsoever (but we know better).

Probing further with live testing #

Now that we have a bug benchmark for faulty OS and screen sizes, we don’t have to waste our time testing things that already work. Instead, we can hone in on those problems that need fixing straight away.

Most cloud based testing tools have a live environment where you can select an emulator and use it just as if it was running on your machine. We know that we’re having some serious problems running on Android.

To start testing on BrowserStack you need to be signed up and logged in (there’s a trial subscription).

Then, navigate to the Live testing section.

A few seconds go by while the environment is starting. Once complete, you’ll be connected to a server running a browser on a Google Nexus 7 emulator with your target URL loaded up.

Note: Your environment may never load if you’re browser does not have a recent version of the flash plugin installed - I found that the hard way.

Using the app, I can see the sun illustration dimensions are completely wrong. However, the layout is correct and the css3 animation works. This suggests that only a few simple css tweaks are necessary to support the Google Nexus 7 default browser for Android 4.1 - Yay!

Browser, browser, browser #

All this testing highlights that the peculiarities are browser centric. In the Android world there’s a high spread of browsers all supporting variations of CSS and the latest HTML5 capabilities. The Cross Browser Testing site has a good list of the different browser configurations for mobile devices. I particularly like their detailed listings for Android.

Coupling this knowledge with tools like Modernizr allow us to smooth out the discrepancies and build a unified set of instructions that can be applied to more browsers. Basically, less frustration, more fun and speedier development!

Running your own emulators #

We’ll attack iOS and Android emulators separately. Each platform requires some software installation before you can start. Once you’re up and running you can use the same setup over and over again with no hassle!

Bear in mind that iOS emulators are only available for Mac OSX operating systems. This pretty much means that you have to be running on a Mac. Android is more widespread and you can run the emulator on multiple operating systems.

Local iOS emulators #

To avoid confusion, I’ve been constantly talking about emulators but in iOS world they’re referred to as simulators. So, on a mac desktop if you’d like to access an iPhone simulator then you need to,

This gives you access to the latest iOS7 simulator. You can also access older simulators which would simulate an older browser environment. Here are more details.

To test an app on Safari,

Local Android emulators #

For Android, the installation is slightly more cumbersome but definitely doable with speed improvements here. Also, a few vendors have popped up to help get you get up and running in no time.

I like Genymotion simulator. Although slightly long-winded because of the dependencies, it is simple to set-up and they have a free personal licence.

Genymotion relies on having a Virtual Box VM installed to run the different configurations for an Android Simulator. Its very similar to the emulator configurations seen on cloud based services.

Let’s install this first,

You also need to ensure that you have Android SDK for your OS installed including the platform-tools directory. On Mac OSX,

Your done once you can now see a new folder called ‘platform-tools’ in the (android-sdk-macosx) folder.

To get Genymotion up and running,

After the device is up and running,

Finally #

There you have it, all the emulator love we can handle!

Part 3 will showcase how to manually test hybrid mobile apps using both cloud services and local emulators mainly for the Cordova ecosystem.

Links

 
7
Kudos
 
7
Kudos

Now read this

Can you easily deploy your Cordova/PhoneGap apps?

99 problems # Houston, we have a problem! It’s failing, our app is constantly breaking UAT - but nothing has changed. Apart from we now build from JS and generate our iOS platform project on the fly. Could that be it? Why would that be... Continue →