Verifying a Cordova/PhoneGap build with Smoke Tests & Appium
Let’s jump right in! I’m going to flesh out a smoke test that verifies our mobile sign in process works.
The test is written using Mocha.js and runs on Node.js. While working through the assertions, I’ll explain how Appium works. I’ll then verify the build on iOS/Android emulators as well as physical devices too!
You can review all the code in this post at my Appium smoke test bootstrap project. As a bonus, its includes both a detailed README to help you easily setup your environment and the sample app builds I’ll be testing against in this post.
The assertions #
I should have an email field #
To sign in, the app uses email as ID. My first assertion simply checks the existence of an email field.
it("should have an email field", function () {
return driver
.elementByCss('input[type=\'email\']', 500)
.should.eventually.exist;
});
I should have a password field #
My second assertion checks the existence of a password field.
it("should have a password field", function () {
return driver
.elementByCss('input[type=\'password\']')
.should.eventually.exist;
});
I can login with a correct email and password #
it("should login with correct email and password", function () {
return driver
.elementByCss('input[type=\'email\']')
.sendKeys(username)
.elementByCss('input[type=\'password\']')
.sendKeys(password)
.elementByCss('.sign-in').click()
.waitForElementByCss('.app-title', 5000)
.waitForElementByCss('.app-button', 5000)
.text().should.eventually.include('+');
});
In all the above assertions, I’m using a driver to check that an element should eventually exist based on a CSS selector. In the last assertion I go further, I am able to complete form fields and click a button.
But where does this driver come from?
The driver #
It’s an Appium client, in my case this is the wd.js webdriver library. Appium is a server that exposes a restful WebDriver API very similar to the API used to automate web browser testing. However, it builds on it with mobile friendly commands.
To test an app build, the Appium server accepts incoming webdriver requests and fires the necessary UIAutomation (iOS) or UiAutomator/Selendroid (Android) commands.
This is a huge WIN! I can use any client implemented in any language as long as it speaks WebDriver API and have one set of tests to drive both iOS and Android apps.
Configuring the driver #
Below I illustrate some very basic configuration. Refer to the actual CanLogin.uat.js smoke test for full details.
Base config #
var wd = require("wd"),
chai = require("chai"),
chaiAsPromised = require("chai-as-promised"),
driver;
chai.use(chaiAsPromised);
var should = chai.should();
chaiAsPromised.transferPromiseness = wd.transferPromiseness;
exports.should = should;
driver = wd.promiseChainRemote({ host: 'localhost', port: 4723 });
Here I’m setting up our tests to run in a Promises based mode.
The driver is then configured to point at an instance of Appium running locally. This was started simply on the shell using.
appium
Hint: this is not a daemon process so do start it in a separate shell to run test commands
Adding capabilities #
I initialise a test session with a set of capabilities. These inform Appium of our target platform (iOS or Android), location of the app under test and any other details required by the wrapped driver.
var capabilities = {
browserName: '',
'appium-version': '1.1',
platformName: 'iOS',
platformVersion: '7.1',
deviceName: 'iPhone',
app: "sample-code/apps/ios/sim/BeanThere.app"
};
driver.init(capabilities)
In this case, I’ll be executing our test against an iOS app on an iPhone running iOS7.1.
Switching to WEBVIEW context #
Appium can run tests in either a Native or WEBVIEW context. The Native context is the default setting. It’s used to drive native applications.
Hybrid apps tend to run in some sort of web view. Hence, I’m configuring the driver to switch to a WEBVIEW context before running any tests.
driver.setImplicitWaitTimeout(3000)
.contexts()
.then(function (contexts) {
var webViewContext = _.find(contexts, function (context) {
return context.indexOf('WEBVIEW') !== -1;
});
return driver.context(webViewContext);
});
That’s it, time to run the test.
Testing on an iOS simulator #
As I’m working against my bootstrap project, the spec is available at uat/hybrid/CanLogin.uat.js.
To run the tests for iOS the Appium server has to be up and running on MacOSX hardware. Also, the app has to be built to target a simulator e.g. an i386 ‘.app’ file build.
I simply run the tests using Mocha.js on the shell:
export PRIVATE_USERNAME='coffee@email.com'
export PRIVATE_PASSWORD='password'
mocha uat/hybrid/CanLogin.uat.js
I first setup private env vars for the username and password since these should never be hardcoded in our smoke tests.
Then, as per our capability settings, the test was run on iOS and on a local server. In reality I use shell flags to wire this up e.g.
DEV=true PLATFORM=ios mocha uat/hybrid/CanLogin.uat.js
Woot! we can sign in successfully! #
Here’s a sample screen dump that showcases the webdriver API calls. It verifies that all is well.
Also, this log is a very helpful tool for understanding any errors that might occur with the test. It can help highlight CSS selector typos, etc…
...
CALL elementByCss("input[type='email']",500)
POST /session/:sessionID/element {"using":"css selector","value":"input[type='email']"}
RESPONSE elementByCss("input[type='email']",500) {"ELEMENT":"5000"}
...
CALL quit()
DELETE /session/:sessionID
Ending your web drivage..
RESPONSE quit()
3 passing (12s)
Testing on an Android virtual device #
This is slightly more involved.
Setting up Genymotion #
I’ve been using Genymotion to keep Android development and testing work simple. It packages an AVD as a VM and allows adb to interact with it as if it was a physical device.
adb devices -l
Should result in
List of devices attached
192.168.56.101:5555 device product:vbox86p model:Google_Nexus_5___4_4_2___API_19___1080x1920 device:vbox86p
You can find more helpful setting up tips here.
Updating the capabilities #
var capabilities = {
browserName: '',
'appium-version': '1.1',
platformName: 'Android',
platformVersion: '4.4',
deviceName: 'Android Emulator',
automationName: 'selendroid',
deviceType: "phone",
androidUseRunningApp: false,
appPackage: 'com.the_experimenters.hybrid_sign_in',
appActivity: '.BeanThere',
app: "sample-code/apps/android/BeanThere-debug.apk"
};
Capability: automationName #
I ask Appium to use the Selendroid automation framework, I’ve found this to be more robust than using the default Chrome driver.
Capabilities: appPackage + appActivity #
They help Appium/Selendroid locate the correct package and activity combo to fire up and test. They are normally setup in an Android project’s AndroidManifest.xml.
My .apk under test was created using Apache Cordova where the AndroidManifest.xml resides at:
(project-root)/platforms/android/AndroidManifest.xml
Running the actual test #
With a Nexus 5 Genymotion virtual device up and running, I simply run the tests using Mocha.js on the shell:
export PRIVATE_USERNAME='coffee@email.com'
export PRIVATE_PASSWORD='password'
mocha uat/hybrid/CanLogin.uat.js
This works with a similar screen dump to the iOS test:
...
CALL elementByCss("input[type='email']",500)
POST /session/:sessionID/element {"using":"css selector","value":"input[type='email']"}
RESPONSE elementByCss("input[type='email']",500) {"ELEMENT":":wdc:1406876329653"}
...
CALL quit()
DELETE /session/:sessionID
Ending your web drivage..
RESPONSE quit()
3 passing (9s)
Awesome, by re-configuring capabilities I’ve created assertions that target two different mobile platforms!
Testing on an iOS device #
For this to work correctly, I compiled the app for a device (armv7 build) and signed it with a developer cert. To help you get setup with your own app, I’ve documented further instructions here.
Finding a device’s UDID #
Provided my iPhone is connected to a usb port, I run a system profiler command to list all connected devices.
system_profiler SPUSBDataType
This prints all attached USB devices. The iPhone’s serial number is the UDID.
The iOS WebKit Debug Proxy #
I know more config! But, since I’m running the tests against a physical device with its own IP address. I use a proxy to ensure Appium can point to the correct UIWebView’s WEBVIEW context.
I install this using homebrew:
brew install ios-webkit-debug-proxy
Once, installed I start the proxy with the discovered UDID.
ios_webkit_debug_proxy -c UDID:27753 -d
Reconfigure capabilities #
I reconfigure the driver’s capabilities to point to the correct build and add the device’s UDID.
var capabilities = {
browserName: '',
'appium-version': '1.1',
platformName: 'iOS',
platformVersion: '7.1',
deviceName: 'iPhone',
app: "sample-code/apps/ios/device/BeanThere.app",
udid: "my-udid-12f5e12f5e112f5edf99ffc5258919c68"
};
Running the tests #
export PRIVATE_USERNAME='coffee@email.com'
export PRIVATE_PASSWORD='password'
mocha uat/hybrid/CanLogin.uat.js
This is no different from the other commands. By now, I hope you can see that all the heavy lifting is done by configuring the necessary capabilities.
Appium uses Fruitstrap to install the app on the device and then remove it after running the tests. This ensures a fresh build every time. For debugging purposes I use ios-deploy to ensure I can simulate any errors that might occur and then work through the issues.
Testing on an Android device #
I attach my Nexus 5 phone to my laptop as a usb device. Then I simply run the tests with the same capabilities used for testing the Android virtual device.
That is all - Android is simple :)
The end #
There you have it. There a few hoops to jump through to make this whole thing work correctly. However, they can all be easily automated.
With this post I wanted to jump straight into the meat of the problem highlighting only the important bits. Please refer to my Appium smoke test bootstrap project for full the full smoke test.
Smoke tests are an essential tool in a mobile develper’s arsenal. Appium makes it a pleasure.