Today, there are many smartphone users in the world and so is the popularity of mobile apps. In order to be competent enough, mobile apps have to be unique and should provide the best user experience to increase the user base. With the users getting more informed and intelligent, the apps built should keep up with the pace. In order to be impeccable, the mobile app should undergo a rigorous testing process and during that process, the testing team faces many challenges in this aspect which will be covered in this blog. But before we dive in, let’s look at the different types of apps that are available in the market.
Types of mobile applications
The creation of mobile applications is a fantastic approach to boost brand recognition, attract new clients, and improve the user experience for existing customers. In light of this, let’s examine the three primary categories of mobile apps: native, web, and hybrid.
Native apps:
Native mobile applications are ones created exclusively for a given operating system. As a result, software created for one System cannot be used on another, and vice versa. Native applications are more effective, quick, and offer greater phone-specific functionality. Thus, the difficulties of testing mobile apps for compatibility with native user interfaces of devices involve ensuring that such traits are preserved strictly.
Web apps:
Similar to native apps, web applications do not require users to download them. Instead, the users’ web browsers on their phones can access these apps because they are incorporated within the website. So, it is envisaged that web applications will operate flawlessly across all platforms. Testing teams must carefully examine the application on a wide range of real devices and browsers to ensure high app quality. Yet in addition to taking a lot of time, this operation is essential because failing to work on a few devices can severely reduce the app quality and incur heavy losses when the app doesn’t function as required.
Hybrid apps:
The features of both online and native apps are available in hybrid apps. These are essentially web applications that mimic native apps in design. These applications are easy to maintain and load quickly. Teams that test mobile apps are in charge of making sure hybrid applications don’t lag on some devices. Any operating systems with the capacity to support the said features have access to all their functionality.
While each of these app types are slightly similar to each other the technical teams face a different challenge with each type of mobile application. Combining these challenges greatly increases the complexity, making the entire procedure laborious and time-consuming. Let’s quickly look into what some of these challenges are?
Different Operating Systems and their versions
There are different types of operating systems available in the market such as iOS, Android, Windows etc. Also, these OS have different versions too. So, it becomes challenging to test so many versions of the mobile app in a shorter period of time. One app that works well in one type of OS may not work well in the other. It is very important to test the application with all supported platforms and their version because we don’t know where the user is going to install the application. As per research, iOS users upgrade quickly as compared to Android but in Android the device fragmentation is larger. That means the developers have to support older versions and APIs and testers also have to test accordingly.
Device Variations: Based on Screen size
Android comes with a mix of features and variations in pixels densities and ratios which varies in each screen size. Even in the case of Apple, the screen new size was introduced with the launch of the iPhone 6. Now, it is not just about being picture perfect screen design rather designing an adaptive screen design. Well with such a variety in screen sizes, the role of the tester becomes serious as they need to check if all the features are working well in different screens and pixel and aspect ratios are maintained well.
Based on the number of Devices
The picture below shows the number of devices in the market by different brands. The number of device manufacturers has increased. According to OpenSignal, there are around 1294 distinct Android phone manufacturers alone, imagine if we add up other brands. The pace with which this data is increasing is a bit alarming for the testers as the testers have to check the app performances on different devices, they would probably need a device library to do the same. The challenge remains in context to functionalities like Complex user interactions on touch screen and keypad devices as well. Having a device library is certainly is a costly affair unless emulation is adopted which can simulate multiple device types and testing can run easily on it.
Image Source: venturebeat.com
Various Networks
The QA team also faces challenges when it has to test the devices connected to different networks. Generally, there are 2G, 3G,4G mobile data available. These provide different data transfer speed and transmission. These varying speeds of the networks by various providers remain a challenge for the testers even today. In this case, testers have to check that the app must perform well at different network speeds and connectivity quality and a check on bandwidth usage of the app. This remains a challenge as it is partially controllable based on different network providers and connectivity access in different geographies.
Frequent OS releases
Mobile Operating Systems keeps changing. Both Android and iOS have more than 10 versions of their operating systems. They keep enhancing and updating their versions for better performance and user experiences. This frequent OS release comes as a testing challenge as the testers needs to validate the complete application with each of new OS release. It is very important to test the application with the latest OS release otherwise the app performance would be a major issue and consequently loss of users using the app.
Script Execution
Another major challenge of mobile testing is what we call scripting, the method of defining a test. Script execution can either be manual or automated. You can write down the scripts in a document, which is then used by a test engineer who manually interacts with the test environment to determine the result, else you can run automated scripts that in turn drive interaction with the device and app, and record the results.
Automated scripting needs to be kept away from the device to be of any real use because there are so many different devices with different interface options. A script that follows strict keystrokes on an Apple iPhone would not have any chance of working on a Samsung device, because the UI is different. Fortunately, most real device automated testing software provides high-level scripting that operates on the text, image, or object layer. Device emulators can automate testexecution using a higher-level, abstracted scripting language that is not device dependent. When you use automated scripting, the cost of setting up the script will typically be higher than the cost of a single manual execution of a test. But if it is a test script that you run on a periodic basis, every time that you subsequently run the script, the more time and effort you will save. You will eventually recover the cost of initial scripting If you run the script enough.
So to conclude, to build a better user experience, an app tester needs to work had in overcoming the challenges of testing. By adopting some analytical skills and methods, testers can really cope up with these situations. For eg. Testing only those apps and OS which are mostly used by their user segment, by adopting a strong testing strategy to take situational decisions eg. Decisions regarding when to choose Automation and manual testing. Strategically, the challenges can be overcome.
Screen Size
The Android world is not simple. The variety of different aspect ratios and pixel densities can be overwhelming. With the launch of iPhone Xs Max which has a screen size of 6.5 inches, Apple brings new screen sizes to the iOS world as well. Though iOS developers are used to pixel perfect screen design, they now need to change their mindset to the adaptive screen design instead. For testing, it means that we need to check on various devices that all the necessary screen elements are accessible with different screen sizes and aspect ratios. There are many phones with a screen size of 5 inches which are still popular.
Security Issues
Traditional testing tools like selenium and QTP weren’t designed with cross-platform in mind. Automation tools for web apps and mobile apps are different. Operating systems especially Android further adds to the complexity with API level fragmentation. The most common automation testing tools for mobile app automation testing are Appium and calabash. Each tool has it’s own advantages and disadvantages and you need to choose on the basis for your app’s functioning.
Weak Hosting Controls is one of the most common issues. The server on which your app is hosted should have security measures to prevent unauthorized users Weak Encryptions can lead to data theft which will impact the trust factor of the users. Most of the mobile apps require user data such as email ID, password, age, location etc. This data should be encrypted and stored with proper security. Hackers often use this kind of data to get money out of users account online. Encryption will make it difficult for anyone unauthorized to intrude and retrieve that data rather than keeping it in plain text.
Power consumption and battery life
We haven’t seen much innovations in the mobile battery but the mobile usage and specifications are increasing rapidly. People are using more apps nowadays and the apps are more complex than ever. This is why testers need to test the apps power consumption because if the apps use lots of CPU cycles and some apps will also run in the background than the battery will drain out quickly. We need to make sure that the app uses less battery power so that users can use it for a longer period of time.
Conclusion
Mobile apps are evolving with device technology and user expectations. Developers are emphasizing on reducing the app size and battery usage. Testers play a major role to ensure that the app works smoothly and does not crash or have bugs. This is why testers must be aware of the latest trends in mobile app testing to deal with the mobile app testing challenges.
Android uses a centralized system for all logs and application programmers can also write custom log messages. The tooling to develop Android applications allows you to define filters for the log statements you are interested in. Log analysis is a phase of development and developers encounter it from time to time. Bug log helps in identifying the bugs in the Android application while in the development phase. Once the app is released into the market, the support engineers analyze the bug logs to resolve the issue. There are for different types of Logs available in the Android Eco-System – Application Logs, Android System Logs, Event Logs & Radio Logs.
The logging system consists of a kernel driver and kernel buffers for storing Android log messages, C, C++ and Java classes for making log entries and for accessing the log messages, a standalone program for viewing log messages (logcat) and the ability to view and filter the log messages from the host machine (via eclipse or ddms).
There are four different log buffers in the Linux kernel, which provide logging for different parts of the system. Access to the different buffers is via device nodes in the file system, in /dev/log. The four Android log buffers are main, events, radio and system. The main log is for the application, events is for system event information, radio is for phone related information and system is low level system messages and debugging.
Each message in the log consists of a tag indicating the part of the system or application that the message came from, a timestamp (at what time this message came), the message log level (or priority of the event represented by the message) and the log message itself (detail description of error or exception or information etc).
What Each Android Log Type Contains:
1. Application log
use android.util.Log class methods to write messages of different priority into the log
Java classes declare their tag statically as a string, which they pass to the log method
The log method used indicates the message “severity” (or log level)
Messages can be filtered by tag or priority when the logs are processed by retrieval tools (logcat)
2. System log
Use the android.util.Slog class to write message with different priority with its associated messages
Many Android framework classes utilize the system log to keep their messages separate from (possibly noisy) application log messages
A formatted message is delivered through the C/C++ library down to the kernel driver, which stores the message in the appropriate buffer(system buffer)
3. Event log
Event logs messages are created using android.util.EventLog class, which create binary-formatted log messages.
Log entries consist of binary tag codes, followed by binary parameters.
The message tag codes are stored on the system at: /system/etc/event-log-tags.
Each message has the string for the log message, as well as codes indicating the values associated with (stored with) that entry.
4. Radio log
Used for radio and phone (modem) related information
Log entries consist of binary tags code and message for Network info
Logging system automatically routes messages with specific tags into the radio buffer
Device Logs on pCloudy Platform.
Device Logs Filter Options
Log format on Android
A common log format in android:
tv_sectv_nsec priority pidtid tag messageLen Message
tag: log tag
tv_sec&tv_nsec: the timestamp of the log messages
pid: the process id of where log messages come from
tid: the thread id
Priority value is one of the following character values, ordered from lowest to highest priority:
V — Verbose (lowest priority)*
D — Debug*
I — Info*
W — Warning*
E — Error*
F — Fatal*
S — Silent (highest priority, on which nothing is ever printed)
Log-File locations
There are several directories where Android log (including those from crashes) stores and it are not standardized (i.e. some may be ROM-specific). I am putting some of common here.
/data/anr : Dalvik writes stack traces here on ANR, i.e. “Application Not Responding” aka “Force-Close”
/data/dontpanic : contains some crash logs including traces
/data/kernelpanics :- Stores “kernel panic” related logs
/data/tombstones :- may hold several tombstone_nn files (nn is a number from 0 to 10 and after 10 again repeat it)
‘Log’ command line tool
To capture Android log from the devices/emulator Below is the some command line tool. In real life project there are log capture application/tool used to capture the logs on user device and shared it back to Developer/maintainer for analysis .
adb logcat (shows all type logs for current android system
adb logcat -v threadtime (it will include date and time)
adb logcat -v threadtime> logfile.txt (Store logs in logfile.txt)
Useful filter patterns
You can use below filter in your adb command to filter logs. You can also use this filter to search your logs file(Android logs provided by user device).
adb logcat -f Save all logs into a file
adb logcat “*:E” Get all errors and fatals
adb logcat | grep -i “foo.example.” #get all logs related to “foo.example.*” tagname
adb logcat “application_or_tag_name:*” “*:S” Get all logs by application name
adb logcat -b events “gsm_service_state_change” “*:S” Get all GSM state changes
adb logcat -b radio Get all Radio events
Log Analysis
In this blog we got the basic understanding of the Android Log System. While analyzing the Android logs from your app you can divide it in two parts, debug Logs – file coming during development and testing phase and production Log – file coming directly from end user. The best way to do it is by using useful Filter Patterns and by using Some Tool like GoogleLogTool and SonyLogTool.
We are committed to keeping you ahead of others. pCloudy is one of the fastest to release the support of Android Q beta devices on the cloud.
Google has launched the final Android Q beta and Android Q official release is just a few weeks away. Testers and developers can test their app against this final beta version. Android Q beta 6 includes the final API 29 SDK and updated build tools for Android Studio. It also supports all the features, developer APIs and system behaviors that will be in the final release.
Now you can test your mobile apps on devices loaded with Android Q beta version in pCloudy.
AndroidX is an improved version of the android support libraries that the android team uses to develop, test, package, version and release libraries within the jetpack. AndroidX fully replaces the support library by providing feature parity and new libraries. In addition, AndroidX includes the following features:
All packages in AndroidX are in consistent namespace starting with the string AndroidX. The support library packages have been mapped into androidx.* packages. For a full mapping of all the old classes and built artifacts to the new ones.
Unlike the support libraries, AndroidX packages are separately maintained and updated. The AndroidX uses strict semantic versioning.
All new android development will occur in the AndroidX library. This includes maintenance of the original support library artifacts and introduction of new jetpack components.
Android Jetpack
Android jetpack is a set of components and tools along with architecture guidance designed to help you accelerate your android development. It gives a template to write production ready android code. Jetpack is made up of components in four categories, foundation architecture behaviour and UI. Each component is individually adaptable and build to maintain backwards compatibility. Android architecture components are very modular, so we are allowed to choose what feature sets we want that are compatible to our app.
Espresso is now a part of the AndroidX family
Espresso is a testing framework designed to provide a fluent API for writing concise and reliable UI test. Writing reliable UI test is difficult as user interfaces are asynchronous driven by events, transitions and data loaded from background threats. Coding around that without any help from UI testing framework would require a lot of boilerplate. Espresso takes care of any UI events, so that in most cases you don’t have to worry about any view state transition and implementation details. The basic UI test flow when using Espresso includes:
View Matchers: To find view in the current view hierarchy for e.g. to find UI elements like buttons, textbox etc.
View Action: To perform action on the view, e.g. to click on a button, double click, scrolling etc.
View Assertions: Allows to assert state of a view.
Application of Espresso test recorder
Allows us to create effective UI test cases with user interactions.
We can capture assertions and interactions without accessing app structure directly which increases execution speed and optimizes test case.
Saves time searching for locators and then writing the test cases.
It supports multiple assertions making more reliable test cases.
Pcloudy supports androidX instrumentation with Espresso
Now you can write test cases in espresso and test the APIs in pCloudy using androidX Junit instrumentation. Here are the steps for running your Test scripts on multiple android devices:
Login over https://device.pcloudy.com with your registered Email ID & Password.
To schedule “Espresso” over pCloudy, follow the below mentioned steps-
Go to the “Automation” page.
Select the Automation tool as “Espresso”.
Select “Instrumentation Type” based on your Test Scripts you’ve written.
Note: pCloudy provides support for Instrumentation Type (InstrumentationTestRunner, AndroidJUnitRunner and AndroidXJUnitRunner) for Android.
Select the Application APK and Test APK that you must have uploaded in the MY APP/DATA section.
Select the single device execution time and assign a name to your test cycle.
In the next step, Click on “ADD” to add the device for testing and click on ”
Next”.
Click on “Schedule” to start the test.
Go to your mailbox and open pCloudy Automation Alert mail.
Click on the given link “Click to view Report”.
Now you have the result of your scheduled test automation.
How to migrate to AndroidX?
To migrate from support libraries to AndroidX the Google has provided a refractor tool in Android Studio. Projects can be migrated to AndroidX by clicking on ‘Refractor’ in the menu bar and then clicking on ‘Refactor to AndroidX’. Then it will search for the usage and show the result. To refactor click ‘Do Refactor’.
pCloudy is leading the way in the field of automated mobile testing solutions.
It is a new publishing format by Google which is a more efficient way to develop and release app. App bundle helps to reduce your app size and deliver features on demand. Earlier, android operating system used android packaging kit (APK) to distribute and install applications on a device. These applications are downloaded by users across the world on various devices. These devices have different configurations and language inputs. To meet all the users demands, the application becomes bulky as all the features are to be downloaded.
Android App Bundle is a zip archive with .aab extension. It contains codes and resources for all the devices that the app supports. Google Play handles signing and generation, once it is uploaded for publishing. In app bundle, dynamic delivery is used to generate an optimized APK for users, based on their device configuration.
Benefits of .aab
The key benefit of android app bundle is that it the developers need to write less code to push the app in Play store. The users save space in their device by saving a small size APK. App bundles can use uncompressed native libraries in android 6.0 and up, that are stored in the APK instead of the users device. This lowers the download size and the size on disk. It serves users with functions they need on demand, instead of installing all the functions at one go. We don’t need to build and publish multiple APKs, therefore, app bundle also simplify the built and release management.
How Android App Bundle works
Android delivers APKs with the required resources using split APK mechanism. Google Play uses this mechanism to split large apps into smaller APKs, as per the device requirements.
According the Google, there are 3 types of APKs:
a) Base APK: This is the first mandatory APK to be installed. It contains the basic requirements for the application. This APK contains codes and resources that other split APKs can provide. Only the base APK’s contains full declaration of your app’s services, permissions, platform version providers and dependencies of system features. It is important that all codes and resources included in this module are included in the base APK.
b) Configuration APK: It contains specific data, based on the device requirements. Configuration APK is generated by Google Play from the app bundle that is uploaded to the store. Each of these APKs includes native libraries and resources for a specific screen density, CPU architecture or language. When a user downloads the app, their device downloads only the specific APKs for that device. You don’t create separate module for configuration APKs. If you use standard practices to organize alternative, configuration specific resources for your base and dynamic modules, Google Play automatically generates configuration APKs for you.
c) Dynamic Feature APK: These are the optional features installed required by the user. Each of these APKs contains code and resources for a feature of your app that is not needed when your app is first installed. Using the play core library, dynamic APKs may be installed on demand after the base APK is installed on the device to provide additional functionality.
Android Application Bundle Format
An Android App Bundle is a file with .aab extension which you can upload to Google Play to support dynamic delivery. App bundles are signed binaries that organize your apps resources into modules. Each of these modules may be generated as separate APKs. Google Play uses the app bundles to generate various APKs that are served to users.
App Bundle’s files and directories: Base/, feature 1/ and feature 2/: Top level folders that contain different modules of your app. The base directory contains base module of the app. The directory for dynamic feature module is given the name specified by the split attribute in the module’s manifest.
Bundle-Metadata: Metadata files include complete list of the app’s DEX files and Proguard Mappings. Files in this directory are not packed into the app’s APKs.
Module Protocol Buffer files (*.pb): Provides metadata that describe the content of each module to the play store. For example, native.pb and resource.pb describe the code and resources in each module, which is used when Google Play optimizes APKs for different device configurations.
Manifest/,DEX/: Unlike APKs, app bundles stores the androidmanifest.xml and DEX files for each module in a separate directory.
res/, libs/and assets/: These directories are used in the same ways as APK, except that for an app bundle, they are used by Google Play to package only the files that satisfy the target device configuration.
root/: This directory stores files that are later relocated to the root of any APK including corresponding module.
How to deploy App Bundle
Unlike APKs, App Bundles cannot be installed on a device. It is an uploaded format which contains compiled code and resources in a single build framework. Once we upload out signed app bundle, Google Play builds and signs the apps APKs and serve them to users through dynamic delivery.
Testing your app bundle with Google Play Internal Test Track
You need to generate signed in app bundle before you can upload your app bundle to the play console. Proceed with these steps to generate a signed app bundle.
Select Build then select Generate Signed Bundle/APK from the menu bar. In the Generate Signed Bundle/APK dialogue, select Android app bundle and click on Next.
In the Module dropdown menu, select the base module for the app you want to generate app bundle for.
Provide information for an existing key and keystore, or create a new. This is the same type of key and keystore information you provide when building a signed APK.
I you want Android Studio to also save your signing key as an encrypted file, check the box next to Export encrypted key. To be able to upload your app bundle and take advantage of dynamic delivery, you need to upload this encrypted file to the play console and enrol in app signing by Google Play.
Click Next and provide a Destination Folder for your app bundle. Select the Build Type and flavours that you want to generate app bundles for.
Click Finish.
Now you have generated a signed bundle, you can upload your app bundle to the play console.
Testing your .aab file on pCloudy pCloudy supports .aab format and the user can upload the App Bundle instead of “.apk” to test their app on the device cloud.
Conclusion
Android Application Bundles is a big step forward in the area of application publishing and uploading. It has reduced the size of APK of your application which leads to more download of the application.
The much awaited bonanza for Appium users is here. We are up with a newer and simpler Appium integration architecture which can simplify your Appium test automation execution on Android and iOS devices with pCloudy. The newer architecture make developers’ life simpler with lesser changes in the code. The older architecture demanded using APIs and also changes in your code that required some level of expertise. We addressed this issue and have come up with a simpler architecture where you need to mention some desired capabilities instead of calling APIs or doing changes in the code to run the Appium scripts. And wonder what! it makes you save 50% of your app testing time.
Watch the video to know more about the new simpler and faster pCloudy Appium architecture.
Pre-requisites
Appium Script
APK or IPA file
pCloudy Account
The basic steps to replace your local Appium driver to pCloudy Appium Driver are given below:
Upload the apk/ipa from your local system to pCloudy. Check this link to know steps to upload an app for test.
Set pCloudy capabilities
pCloudy_Username: Enter the email ID with which you have registered on pCloudy. For reference, check this link
pCloudy_ApiKey: API key is important for user’s verification. You can generate the API key from Settings page on device.pcloudy.com. Check this link to get your API key.
Note: If you don’t know the Device full name then you can enter pCloudy_DeviceManufacturer and pCloudy_DeviceVersion in the code and it will automatically run command on those devices.
Android now occupies the number one place in the world Smartphone arena with a market share of 87% at the end of 2016 that means 9 out of 10 Smartphones in the world run on Android. With such dominance in the space, the creation of mobile apps has reached never before heights. But the constant innovation that fuels this market has major problems in terms of the development and testing timelines. Running an online Android emulator, which used to be a solution once up a time, is not good enough anymore. So what can companies and developers do to stay ahead in the development race? What alternates exist to improve your app’s usability, performance and customer satisfaction? Let us explore.
With the explosion of Smartphones, customers’ screen size is reducing. People are moving from desktops and laptops to Smartphones for their everyday work and personal needs. This massive shift from big screens to a personal device has created a huge opportunity for developers to create tools that can help people with their work and their personal needs. Users can now view their spreadsheets and book movie tickets from their phones.
Apple’s Appstore and Android Play store has become the ‘go to’ place for people to discover apps that meet their requirements. The ratings left on those apps will decide how many people will install those apps. Customers have once again become the king of the market.
Days of unique apps are over. For your every need you will find at least two or more apps, giving you a choice. So if you want people to choose your app, install it, use and positively review it, your app needs to user-friendly, work on most handsets and be bug-free.
In the past, companies and developers determined product lifecycle. They planned the updates, feature upgrades etc based on the resources and other factors. Today the scenario is changed. The Market dictates the timeline. If your app is incompatible with the latest OS, or not render properly, or suck a lot of battery juice, off it goes into oblivion to be replaced by a better app.
The fight for the screen space is constant.
You can win the fight only if you develop faster than others, test it better and offer a good user experience. That means developers will need to compress their development cycle and testers should speed up their testing while at the same time, maintain high usability and reliability of the app.
Android and iPhone both offer an easy to use platform for developers. iPhone has a few products (IP6, IP7, IP8 etc.) and versions (OS 8, OS9, OS10 etc). The permutation of devices and OS makes testing with real devices easier although buying that many Apple devices is quite an expensive investment. You don’t need an online emulator while testing for iPhone.
In contrast, Android has a highly fragmented market. Apart from different versions (KitKat, Jellybean etc), there are different forks of Android (Stock, Cyanogen, OxygenOS etc) and there are also different skins that manufacturers put on their devices (TouchWiz, Optimus, Sense etc). Real android device testing to cover all the combinations is close to impossible and quite expensive.
Enter Online Android Emulators
When they started, online Android emulators were like a boon to developers. They could cross test their apps across different devices without physically buying the phones. Most of the Android emulators were easy to set up and a fairly inexpensive solution. Most importantly, online Android emulators could mimic hardware and software behaviour making easy for developers to identify unexpected behaviour during the early stage testing.
But as the market for apps grew, the demands on the app became even higher. Testing all the features of the app on an Online Android Emulators was not providing the full picture of how the app would behave on a customer’s phone.
For example, a user installs a new app on the phone and the next day he notices the battery is getting low very fast. Looking at the battery usage, he discovers the culprit is the new app and bam! He hits uninstall. He then visits the Playstore and writes a negative review of the app’s battery-sucking problem. The next 20 people who see that review will not install the app.
These kinds of bugs cannot be easily identified using an online android emulator. Apart from that, online emulators are slow because they have to replicate both software and the hardware components. thus slowing down the whole testing cycle.
Online android mobile emulator stimulate android devices on a PC to test an app on a variety of devices and API levels without the need for physical devices. Nox is the best online android emulator although BlueStacks is the best android emulator for online games. Online android emulators come with predefined configurations for various Android phone, tablet, Wear OS, and Android TV devices. There are many online android emulators for iOS available in the market.
Online Android emulators also have other limitations that make them unsuitable for large-scale testing. There are a limited number of OS versions you can run on an Android emulator. Even on a good PC with HAXM acceleration support, you can run approximately 8 emulators at a time. Even if you manage to set up all the online emulators you need, one small problem can send the whole system crashing forcing you to start all over again.
So what can app developers do to speed up their product lifecycle while releasing a relatively bug-free app? Is there any alternative for Online Android Emulators?
Testing on real devices has a few important benefits. Your testing can be in real conditions i.e. weather, location network accessibility, interruptions like SMS, calls etc can be tested accurately. You can also validate the screen brightness, visibility in different lighting conditions. Testing on a real device will be a lot faster than on an Android emulator.
The biggest problem in using actual android phones is the cost of buying all the phones you need to test. There are approximately 11k Android phone models in the market making it financially impossible to test on all available Android devices. This is one reason Online Android Emulators became famous in the initial stages.
You can run only one test at a time. If you have staff in another city or country, they cannot access the device. If you need to install an app, you have to do it manually on every device. Apart from that, there is also a question of logistics, maintaining the devices, updating them, etc all of which are time-consuming tasks.
One tactic used earlier involved a combination of using online Android emulators during the early stages and a small selected list of real Android devices during the beta testing. While this would work for a localized team, it still does not optimize device usage and covers a tiny portion of devices while the fragmentation continues. Thus it is not an optimal solution.
Is there a third alternative that can beat the disadvantages of online android emulator and real devices testing?
In recent times, a new mode of testing is gaining popularity among both amateurs as well as professional developers. It is called Mobile cloud testing.
What is Mobile cloud testing?
Testing on real devices using the cloud as the Interface is the new way of balancing the real android testing while making it economical and scalable. As a developer, you need not own any phones or buy expensive software. You connect to a lab that has a huge selection of Smartphones, select the ones you want to test on and start, it is that simple. You can run functional tests, automated tests, performance and other forms of tests easily. Since the interface is cloud-based, you can test from anywhere in the world, at any time.
There will be some changes in the way you set up your test, install an app, etc. All the testing though will happen on actual devices under user-level conditions. You can set the location, observe CPU utilization, battery drainage and pretty much anything else that you can do with a device in your hand. These are the things you can never test with an online Android emulator.
The best part is that you need not invest in any special infrastructure for such a facility. There are independent services providers who can allow you access to the device on a cost per use basis. You can even reserve devices in advance if you want to plan a battery of test over an extended period.
Opting for mobile cloud testing service gives you access to real devices at a fraction of a cost of setting up your own lab. There are no recurring charges as most of them use a ‘pay as you go’ model. You can spend all your time in testing rather than worry about the infrastructure, maintenance and other problems that come from the ‘owning’ model.
While there may not be a cost associated with getting Android emulators, they need a lot of expensive hardware to run a sufficient number of an online android emulator. Mobile cloud testing does not have a problem.
Using Mobile cloud testing, you can cover more ground over a large number of devices than if you were using either only real android device testing or the online android emulator. For example, if you have 10k scenarios to cover, you can spread it over 1k devices through automation rather than 1k test scenarios over 100 different online android emulator.
One of the biggest benefits of testing mobiles over the cloud is that you are more likely to discover bugs that affect real-world customers than what an emulator can reveal. From a user satisfaction perspective, this is probably one of the biggest advantages mobile cloud testing has over online android emulator.
Most of the app development work has moved to the agile methodology which means testing will run almost parallel to development work. You will need a way to speed up the testing process to meet the sprint deadlines.
Using a mobile cloud, it is easy to streamline the testing process whether you are using a DevOps or an agile approach. Continuous testing is a lot easier to streamline over the cloud setup rather than via online Android emulators or even with real devices.
Many development teams are spread across the globe. Cloud setup gives them an easy way to test seamlessly from different geographies.
Security for Mobile apps testing
Different apps have different needs in terms of security. An Astrology app that gives general predictions may have little data security requirements while a banking app might need the testing to be done in a secure environment. Such clients can always go for a private cloud or even opt for on-premises setup. When you use a good solution provider, you can just hook up your existing devices to their setup and give access to your cross-border teams to test on the devices.
Testing performance issues, be it CPU load, battery discharge or performance over 2G, 3G and other networks work a lot easier when you use the mobile testing via the cloud platform. You can select which network mode you want to be on and apply the test. This kind of flexibility is difficult and sometimes, impossible using an android emulator.
Mobile Apps Performance Testing
Apart from the need to test for compatibility with existing apps, battery usage, network usage etc, and mobile apps will need to go through a round of performance testing. Due to the ever-increasing storage space in a Smartphone, users tend to store a lot of content on their phones. So it is necessary to find how the increase in storage affects the application performance.
For example, if you are developing a photo gallery, how does your app perform when accessing 5 GBs worth of pictures and videos, v/s accessing 50 GBs worth of pictures and videos? Does it slow the pre-fetch? Customers are very unforgiving of apps that slow down their phones.
In case your app has a server-side component, it is important you run a performance test to verify how many concurrent users the server can handle. There have been many instances where e-commerce sites crashed being unable to keep up with increased demands during holiday sales.
These are a few areas that we cannot test on an online android emulator.
A customer kept waiting is a customer lost
A couple of years ago, it was acceptable to wait for two-three minutes to download an image on a phone. But today if your app takes longer than few seconds to start up, it is killed and replaced. The tide has shifted to the customer’s side in the Appstore and Playstore. The only way to win this game is to be Nimble, easy to use and not crash the phone.
Thanks to Mobile cloud testing, you don’t have to wait for the slow, unreliable online Android Emulators anymore. Testing on Mobile cloud gives you the opportunity to cut your infrastructure cost, speed up your testing cycle, spread your testing over a larger set of devices and gives you close to real-world results. As the completion heats up for the screen space, you have one tool in your kit that can help your app survive the tough jungle of user reviews.
