Most of us now carry the Internet around in our pockets as mobile is rapidly becoming the primary way that we shop, work, play, interact with each other, and everything in between.
Mobile does, however, present a unique set of challenges for quality assurance. The quality bar is already high and only getting higher because mobile users expect a fast and seamless experience every time they open an app. Just one crash can lead a user to ditch an app or write a scathing review. In fact, Equation Research found that 79% of users would only retry a mobile app once or twice if it failed to work the first time while performance problems would steer 48% of users away from using the app again.
This post explains some of best practices on test driven development, automated testing, and user testing with automated distribution, and there are links to several additional resources, including the new App Distribution for Pivotal Cloud Foundry service and related webinar, Insider’s Guide to Mobile Apps.
What Makes Mobile Testing So Hard?
Top notch app testing is crucial. However, it is difficult since device fragmentation is a major issue with thousands of Android devices on the market. Even iOS is not entirely immune from this issue. Between all of the different “iDevices” currently supported, there are over 30 different iOS device models on the market today. Throw in multiple languages for different countries and you can see how the test matrix is daunting to say the least.
Taking a multi-pronged approach to testing helps to make the overall process more manageable, thorough, and successful. Let’s take a look at three key steps to addressing the challenge of delivering high quality in a mobile world, as well as a new Pivotal Cloud Foundry microservice to help streamline the process of distributing test builds.
1. Test Driven Development
Testing early and often is critical for creating high quality apps. This helps catch bugs early in the development process, fixing minor issues before they turn into expensive problems and require tedious debugging later. A great way to achieve this is with Test Driven Development (TDD), one of the key concepts within the framework of agile development methodologies.
TDD is a reversal of the traditional approach to software development in which you write code, then write the automated tests for that code. TDD instead involves initially writing an automated test for a given feature, then coding the feature itself and ensuring that it passes the test.
This methodology has been shown to not only produce higher quality software but increase programmer productivity as well. It ensures that all code is covered by at least one test, which raises the baseline of quality. Beyond that, it encourages developers to focus on actual use cases throughout the development process and can reduce extraneous code. The net result is better quality software, delivered faster.
2. Automated Testing
With the myriad devices on the market today, you’ll want to automate as much as possible. This involves executing test scripts on devices, and it is particularly useful for regression testing and smoke tests (simple tests to ensure the app functions at a very basic level) because these are generally repetitive and time consuming to do manually.
There are many tools available for automated testing and several of them require the use of rooted or jail broken devices. We recommend avoiding those since rooting phones voids warranties, opens security holes, and even changes the behavior of the device being tested. Don’t waste time testing on devices that are not set up for their legally intended use.
Automated testing saves significant time and money by allowing you to run tests on many devices simultaneously and quickly. Since machines never miss a step, leveraging automation reduces risk of human error throughout the testing process. You can read more about requirements, goals, and solutions for automation testing on mobile in this blog from Pivotal Labs.
3. User Testing and Automated Distribution
Despite the advantages of automation, it doesn’t address everything. User testing is still required in order to cover the unexpected aspects of human behavior that happen in real life. It is also critical to obtain valuable insights about how people actually use your app.
Once you’ve completed the automated tests, it’s time to put your app in the hands of real people, starting with the QA team and continuing with test user groups. You’ll want to stick to this “order of operations” in order to catch and fix the most basic issues before bringing your app in front of your most valuable testing audience.
Getting a pre-release mobile app in the hands (or pockets, as it were) of users can be a painful process. It often involves emailing a file, dragging/dropping it into a desktop application, and syncing it via cable to the device. This can especially be a nuisance for typical non-tech savvy users who don’t do it every day.
The challenges with user testing process don’t stop there. Keeping tabs on test user groups for different apps and making sure everyone has the correct versions of the correct apps is another burden for mobile development and QA teams. Using a platform to manage the distribution and version management of test builds will streamline this process and reduce frustration. Ultimately, an effective process is necessary to scale your user testing practice for the multiple apps and frequent app updates that define a best-in-class mobile strategy.
The new App Distribution for Pivotal Cloud Foundry simplifies this process by providing an easy, intuitive way for users to do over-the-air (OTA) installs of pre-release apps with the tap of a button. It handles device registration, user and group management, distribution of apps, notification of new available app versions and more. It supports all of the major mobile platforms and runs on Pivotal Cloud Foundry, so enterprises can deploy it in a private cloud, on premises for full control and privacy. This makes life a lot easier for users testing the apps, as well as the mobile development and QA teams releasing apps.
Following these three steps diligently will lead to higher quality apps, better user engagement, and stronger app ratings. Moreover, it will help your organization achieve overall success in your mobile app efforts in a world where mobile is increasingly the primary way customers, employees, and business partners work and interact.
Learning More
Check out the related webinar, Insider’s Guide to Mobile Apps to get more information on mistakes to avoid, best practices, and secret weapons.
Related resources:
Infographic: 5 tips for building engaging, data-driven mobile apps
Video: Driving value from an open cloud platform
Blog: 3 killer ways to up your Mobile app game in 2015
About the Author
