Quick Links:

• What is Mobile App Testing and why is it needed?
• Android and iOS: Basics and Comparison
• Types of Mobile Apps
• Challenges in Mobile App Testing
• 7 Types Mobile App Testing
• Mobile App Testing Strategies

Different Operating Systems and their versions

There are different types of operating systems available in the market such as iOS, Android, Windows etc. Also, these OS have different versions too. So, it becomes challenging to test so many versions of the mobile app in a shorter period of time. One app that works well in one type of OS may not work well in the other. It is very important to test the application with all supported platforms and their version because we don’t know where the user is going to install the application. As per research, iOS users upgrade quickly as compared to Android but in Android the device fragmentation is larger. That means the developers have to support older versions and APIs and testers also have to test accordingly.

Device Variations: Based on Screen size

Android comes with a mix of features and variations in pixels densities and ratios which varies in each screen size. Even in the case of Apple, the screen new size was introduced with the launch of the iPhone 6. Now, it is not just about being picture perfect screen design rather designing an adaptive screen design. Well with such a variety in screen sizes, the role of the tester becomes serious as they need to check if all the features are working well in different screens and pixel and aspect ratios are maintained well.

Based on the number of Devices

The picture below shows the number of devices in the market by different brands. The number of device manufacturers has increased. According to OpenSignal, there are around 1294 distinct Android phone manufacturers alone, imagine if we add up other brands. The pace with which this data is increasing is a bit alarming for the testers as the testers have to check the app performances on different devices, they would probably need a device library to do the same. The challenge remains in context to functionalities like Complex user interactions on touch screen and keypad devices as well. Having a device library is certainly is a costly affair unless emulation is adopted which can simulate multiple device types and testing can run easily on it.

Image Source: venturebeat.com

Various Networks

The QA team also faces challenges when it has to test the devices connected to different networks. Generally, there are 2G, 3G,4G mobile data available. These provide different data transfer speed and transmission. These varying speeds of the networks by various providers remain a challenge for the testers even today. In this case, testers have to check that the app must perform well at different network speeds and connectivity quality and a check on bandwidth usage of the app. This remains a challenge as it is partially controllable based on different network providers and connectivity access in different geographies.

Frequent OS releases

Mobile Operating Systems keeps changing. Both Android and iOS have more than 10 versions of their operating systems. They keep enhancing and updating their versions for better performance and user experiences. This frequent OS release comes as a testing challenge as the testers needs to validate the complete application with each of new OS release. It is very important to test the application with the latest OS release otherwise the app performance would be a major issue and consequently loss of users using the app.

Script Execution

Another major challenge of mobile testing is what we call scripting, the method of defining a test. Script execution can either be manual or automated. You can write down the scripts in a document, which is then used by a test engineer who manually interacts with the test environment to determine the result, else you can run automated scripts that in turn drive interaction with the device and app, and record the results.

Automated scripting needs to be kept away from the device to be of any real use because there are so many different devices with different interface options. A script that follows strict keystrokes on an Apple iPhone would not have any chance of working on a Samsung device, because the UI is different. Fortunately, most real device automated testing software provides high-level scripting that operates on the text, image, or object layer. Device emulators can automate testexecution using a higher-level, abstracted scripting language that is not device dependent. When you use automated scripting, the cost of setting up the script will typically be higher than the cost of a single manual execution of a test. But if it is a test script that you run on a periodic basis, every time that you subsequently run the script, the more time and effort you will save. You will eventually recover the cost of initial scripting If you run the script enough.

So to conclude, to build a better user experience, an app tester needs to work had in overcoming the challenges of testing. By adopting some analytical skills and methods, testers can really cope up with these situations. For eg. Testing only those apps and OS which are mostly used by their user segment, by adopting a strong testing strategy to take situational decisions eg. Decisions regarding when to choose Automation and manual testing. Strategically, the challenges can be overcome.

Screen Size

The Android world is not simple. The variety of different aspect ratios and pixel densities can be overwhelming. With the launch of iPhone Xs Max which has a screen size of 6.5 inches, Apple brings new screen sizes to the iOS world as well. Though iOS developers are used to pixel perfect screen design, they now need to change their mindset to the adaptive screen design instead. For testing, it means that we need to check on various devices that all the necessary screen elements are accessible with different screen sizes and aspect ratios. There are many phones with a screen size of 5 inches which are still popular in 2019.

Security Issues

Traditional testing tools like selenium and QTP weren’t designed with cross-platform in mind. Automation tools for web apps and mobile apps are different.Operating systems especially Android further adds to the complexity with API level fragmentation. The most common automation testing tools for mobile app automation testing are Appium and calabash. Each tool has it’s own advantages and disadvantages and you need to choose on the basis for your app’s functioning.

Weak Hosting Controls is one of the most common issues. The server on which your app is hosted should have security measures to prevent unauthorized users Weak Encryptions can lead to data theft which will impact the trust factor of the users. Most of the mobile apps require user data such as email ID, password, age, location etc. This data should be encrypted and stored with proper security. Hackers often use this kind of data to get money out of users account online. Encryption will make it difficult for anyone unauthorized to intrude and retrieve that data rather than keeping it in plain text.

Power consumption and battery life

We haven’t seen much innovations in the mobile battery but the mobile usage and specifications are increasing rapidly. People are using more apps nowadays and the apps are more complex than ever. This is why testers need to test the apps power consumption because if the apps use lots of CPU cycles and some apps will also run in the background than the battery will drain out quickly. We need to make sure that the app uses less battery power so that users can use it for a longer period of time.

Conclusion

Mobile apps are evolving with device technology and user expectations. Developers are emphasizing on reducing the app size and battery usage. Testers play a major role to ensure that the app works smoothly and does not crash or have bugs. This is why testers must be aware of the latest trends in mobile app testing to deal with the mobile app testing challenges.

In the next blog, We will talk about Different types of mobile app testing.

Related Articles:

Appium vs Espresso

Mobile app automation testing has evolved as a crucial aspect of the mobile app development process to help deliver better quality solutions, under controlled time cycles and cost schedules. But for delivering bug-free app, choosing the best suitable automation testing framework for your app is very important. There are many automation testing frameworks available in the market with exceptional capacities. This blog is all about Appium vs Espresso and we will analyze which of these two most widely used Automation testing frameworks is preferable for your app testing.

Espresso was not preferred because of its flakiness and instability issues. But, from the time Google has brought Android Test Orchestrator, a Gradle test option, instability and unreliability of Android Espresso tests have vanished. This, in turn, is creating a serious problem for the most popular automation framework Appium.

Let’s find out in this blog if Espresso now comes with a power to kill Appium or Appium can hold its stand in this fiercely competitive market.

Let’s get into the details.

What is Appium?

It is an open source, cross-platform mobile app automation testing framework. Appium allows native, hybrid and web app testing and supports automation test on physical devices as well as emulators or simulators. The Appium server uses selenium web driver which permits platform independence and allows the user to use the same code for Android or iOS.

Advantages of using Appium

• Facilitates test execution without server machines

Appium is developed using cross-platform runtime environment like NodeJs which enables programmers to write server-side code in javascript. It is designed as an HTTP server and you can run the test without requiring a server machine.

• Does not require app code recompilation

Most of the automation testing tools require testers to alter app code. Some of the test automation frameworks require testing professionals to recompile the code according to the targeted mobile platforms. Appium enables testers to evaluate both cross-platform and native apps without recompiling and altering the code that often.

• Automates various types of mobile apps

Testers can avoid using different automation tools for different types of apps as Appium can be used for web apps, hybrid, and native apps too. It facilitates the testing of hybrid and mobile web apps as a cross-platform test automation framework. At the same time, it enables testers to test native apps through web driver protocol.

• Testers can use real devices, emulators, and simulators

Testers use real devices to evaluate mobile app’s usability and user experience more precisely. Although, to speed up the mobile app testing one needs to use emulators or simulators too. Appium helps testers to produce reliable test results and reduce testing time by supporting real devices, emulators and simulators.

• Provides a record and playback tool

In Appium, testers can use the inspector to accelerate testing through record and playback functionality. Appium inspector can record the behavior of native apps by inspecting their document object model (DOM). Record and playback tool can produce test scripts in a number of programming languages.

• Testers can automate apps without adding extra components

Testers can execute the same test across multiple mobile platforms without putting extra time and efforts or adding extra component. Appium simplifies automation by keeping complexities in Appium server.

• Supports several web driver compatible languages

You can integrate Appium with many testing frameworks and WebDriver – compatible languages including PHP, Java, Ruby, Javascript, C# and Objective C. Hence, a tester has the option to write test scripts in his preferred programming language.

Disadvantages of using Appium

• Common gestures

Appium lacks commonly used gestures like double-clicking in java-client libraries. It also does not support Android alert handling directly and the users cannot evaluate alert handling through native API. Testers have to put extra time and effort to test these gestures.

• No script execution on multiple iOS simulators

Simulators make it easier for testers to mimic internal behavior of the underlying iOS devices. Although Appium does not allow users to run multiple test scripts on multiple simulators simultaneously.

• Lacks the capability to recognize images

Appium cannot locate and recognize images automatically to evaluate games and apps precisely. The testers have to take help of screen coordinates to make Appium locate and recognize images.

• Does not support older versions of android

Appium supports only Android 4.2 and later and does not supports older APIs for Android. There are still many people using devices which run on older versions of Android and developers find it difficult to test mobile apps developed targeting older Android API level.

What is Espresso?

Espresso is a tool developed by Google which is used for testing the UI of Android apps. It automatically synchronizes your test actions with the user interface of the mobile app and ensures that the activity is started before the tests run.

Although when you execute an Espresso test you will have shared state in separate tests and some flakiness. For this Google came up with a solution. Android Test Orchestrator is a Gradle test option that helps in testing and increases the reliability of our automated test suites.

If you use Gradle build tools in any version of Android Studio below 3.0 then you also have to update the dependency setup. Let’s take a look at the advantages of using Android Espresso.

Advantages of using Espresso

• Integration with Gradle

The new Android Espresso now has the power of the Android Studio and Gradle that comes along with it. So now invoking your tests, running it or modifying it is just a matter of calling a Gradle command. This gives the full power of command line to the developer and makes testability much easier.

• Test Orchestrator

The new Android Espresso comes with the power of Android Test Orchestrator that allows you to run each of your app’s tests within its own invocation of Instrumentor. It ensures that there is minimum shared state and crashes being isolated. It allows you to filter the tests that you want to run and also distribute tests across devices. This implies that you have finer control over how your tests run.

• Less flakiness

The scalability of the test cycle in Android Espresso is high due to the synchronized method of execution. A built-in mechanism in Espresso that validates that the object is actually displayed on the screen. This saves test execution from breaking when confronted with “Objects not detected” and other errors.

• It’s easy to develop Espresso test automation

Test automation is based on Java and JUnit which Android developers are familiar with. There is no setup or ramping up to implement quality in the in-cycle stage of the app SDLC.

• Reliable and fast feedback

Android Espresso does not need any server to communicate with, instead, it runs side by side with the app and delivers fast results. It gives fast feedback to the code changes so that developers can move to the next bug fix.

• Simple workflow

Espresso allows developers to build a test suite as a stand-alone APK that can be installed on the target mobile alongside the app under test and be executed quickly.

Disadvantages of using Espresso

• It requires access to the application source code

Without the source code, you won’t be able to do anything. Also, There is a risk to get used to the in-built test synchronization and UI – then it might be hard to work with WebDriver.

• Narrow focus

If UI tests are required for both Android and iOS, it will be necessary to write twice, for two different systems. If tests require to work with Android outside the application (for example, open a received notification with a text message), you’ll have to use additional tools, such as UIAutomator.

• Knowledge of launching Android app on emulators required

It is desirable to have at least minimal experience of building and launching Android applications on emulators.

Conclusion

Appium and Espresso both can be used to perform UI testing on Android app but if you have to choose one of them then you need to decide on the bases of your requirements. What kind of app is it and what kind of testing you want to perform. Developers who want to perform UI testing for their native Android app should go for Android Espresso. Although, if the test needs to support iOS and Android both and you want to test at a functional level then you can use Appium.

Related Articles:

Opkey is Now pCloudy

We are thriving in a 'Digital First' world. In this digital era, quality assurance remains the major gear behind a digital success and for that it has to work in tandem with the other. As per Gartner majority of organizations have taken up digital transformation projects. These are in the areas of Mobility, Cloud or Data analytics.

Today digital transformation is at its peak because after decades of development it has become incredibly useful. Still, if you look closely, trough is what is waiting after a crest. And it's true for digital transformation as well as you can already see the contours of its inevitable descent into the mundane. 

So what's next?

It's Continuous Next.

We have seen that digital transformation projects taken up by organizations are siloed in nature. To take the next leap which Gartner has termed as Continuous Next there is a need to unify the siloed tech effort by connecting the major touchpoints like mobility, cloud, data etc.


With a mission to propel success of largest digital transformation projects through the power of Futuristic Testing, pCloudy saw this trend early.

This led us to unify our 2 different products – Opkey and pCloudy and go ahead with this product enhancement merger. Opkey is now officially pCloudy Automation Studio.

This product enhancement merger gives way to a complete Digital Assurance platform which enables Quality @ Speed for apps while you connect touch points to achieve Continuous Next.

We do it through portfolio of tools which connect seamlessly in one single platform.

Platform is comprised of three suites of tools:
1) Autonomous Mobile Device Cloud – reliable Test infrastructure for Mobile App Testing
2) Zero code factory Automation Studio allowing rapid Automation for Mobile, Web, and APIs
3) Pre-built Test Accelerators for Enterprise Apps like Salesforce, Oracle Fusion, Workday etc..


This is yet another step to ensure to help you propel success and take a big leap in your digital transformation journey.

We assure the smooth merger of these 2 products which will make organizations take leap in their digital transformation journey. If you have any questions regarding this merger feel free to write back to us.

Related Articles:

What is Expresso Framework?

Espresso is a testing framework used to write UI test cases. It automatically tests your actions with the UI of your application. The framework also ensures that your activity is started before the test run. It can be used to test across multiple applications. If used for testing outside application, you can perform only black box testing. as you cannot the classes inside the application.

Espresso has three components:

ViewMatchers: Allows to find the view in the current view library.
ViewActions: Allows to perform actions on the view
ViewAssertions: Allows to assert state of the view

Why Espresso?

It's fast: It is really important to run test cases at a fast pace as there can be many tests. UI test takes time and is costly and it is possible that you might not be able to run all the tests.

It's easy to setup: Setup process on local machines is easy and Android developers feel comfortable in the mobile SDK language which they use every day.

It's less flaky: Automated DevOps process required fast and reliable feedback. Test written in native tools is much less flaky. It gives more control over the application: It is white box testing, Espresso is inside the application, it has access to and knows how to use the code that actually runs the application for more thorough testing of each element.

You can use Espresso cheat sheet for quick reference as it contains most available instances of Matcher, ViewAction, and ViewAssertion.

Steps for running your Test scripts in Espresso using pCloudy:

Login over https://device.pcloudy.com with your registered Email ID & Password.
To schedule "Espresso" over pCloudy, follow the following steps-

• Go to the "Automation" page.
• Select the Automation tool as "Espresso".
• Select "Instrumentation Type" based on the Test Scripts you've written.



Note: pCloudy provides support for Instrumentation Type(InstrumentationTestRunner, AndroidJUnitRunner, and AndroidXJUnitRunner) for Android.


• Select the Application APK and Test APK that you must have uploaded in the MY APP/DATA section.
• Select the single device execution time and assign a name to your test cycle.
• In the next step, Click on "ADD" to add the device for testing and click on " Next".
• Now you can select the devices by clicking on the "ADD" button. You can also use the device filter to make it easier to search for a device. Click on next once the devices are selected.




• Click on "Schedule" to start the test.




• Go to your email inbox and open pCloudy Automation Alert mail. Click on the given link “Click to view Report”.




• Now you have the result of your scheduled test automation.

Take a look at this GIF to understand the flow of actions taken to run the test cycle.

Software developers tend to make mistakes as app development is a very complex process. This means that the app will contain bugs. Espresso helps us detect errors that may have been made in the development, ensuring the reliability of the client and his satisfaction with the application.

Related Articles:

Since pCloudy is committed to provide to you with the newest the fastest, here is another one. The newly released device Samsung s10 is now available on pCloudy for you to test your app.

Related Articles:

Types of Mobile Apps

Quick Links:

• What is Mobile App Testing and why is it needed?
• Android and iOS: Basics and Comparison
• Types of Mobile Apps
• Challenges in Mobile App Testing
• 7 Types Mobile App Testing
• Mobile App Testing Strategies

This is an era of smartphones and mobile apps and thousands of apps are launched in different app stores every day. With the increasing competition, the companies are focusing more on the decisions regarding what type of apps they need to build that will serve their user market.

Generally, the mobile apps are classified into four categories – native, Hybrid, Web and Progressive Web apps, each serving its own purpose. There is a new category of mobile apps which we will discuss in this blog. Let us understand more about them one by one.

Native Apps:

Native mobile apps are exclusively built for a specific type of Operating system. They are called
native because they are native to a particular device or platform. Apps built on one type of operating system cannot be used on another OS. In other words, Android apps can’t be used on the iPhone. They use the development tools and language that the respective platform supports (e.g., Xcode and Objective-C with iOS, Eclipse, and Java with Android. It provides full access to all device controls like contacts, camera, sensors, etc. The native apps ensure high performance and great user experience as the developers use the native device UI. Native apps can be accessed via respective app stores eg- Android apps on Google Play Stores, iOS apps on App Store, etc.

Advantages of native apps:

1. Natives are very fast.
2. Easily distributed in google apple app stores.
3. More interactive and intuitive.
4. Easily interact with any feature of the phone.

Disadvantages of native apps:

1. Built for a single platform
2. Languages like swift and java used to build these types of apps are hard to learn.
3. Expensive to develop.
4. hard to maintain.

Examples of Native Apps:

Mobile native apps are built using the native device operating system APIs and SDKs. These are coded using a platform specific language like Objective C for iOS, Java for Android, and C# for Windows phone. One can use the standard GUI components that are part of the platform SDK, easily creating a look and feel that is native to the OS and straightforward.

These apps can access all the device hardware including the various sensors and peripherals if any. These apps are quite fast since the executable is compiled for the specific OS and are run directly on the OS. These come with their development environments including various simulators and infrastructure to do actual device testing.

Mobile Web Apps

These are the web applications to deliver web pages on web browsers running on mobile devices. These are web-based mobile apps that do not get installed on your handheld mobile device and are run on web-hosted servers. Mobile web apps typically use HTML, CSS, Javascript, JQuery web technologies. They cannot access all features of native device functionality(camera, calendar, geolocation, etc.).

Advantages of web apps:

1. Reduced business cost.
2. No installation needed.
3. Better reach as it can be accessed from anywhere.
4. Always up-to-date.

Disadvantages of web apps:

1. Dependent on internet speed.
2. Interface not that sophisticated.
3. Take a longer time to develop.
4. Security risk.

Examples of Mobile Web Apps:

The Architecture of Mobile Web Apps:

Mobile web apps are designed to run on a mobile web browser. They are built using multiplatform web technologies like HTML5, CSS, and JavaScript. HTML5 is the most popular and promising technology for ’Write Once Run Anywhere’. Almost all mobile web browsers running on high-end mobile devices support HTML5 to a large extent, and all are trying to achieve full compliance. Thus, it is safe to consider HTML5 as the technology of choice for developing mobile web apps.

Hybrid Apps

Hybrid apps are the mixtures of native and mobile web apps. Like native apps, they live in an app store and can take advantage of the many device features available. Like web apps, they rely on HTML being rendered in a browser, with the caveat that the browser is embedded within the app. These are developed using technologies like HTML, CSS, Javascript, JQuery, Mobile Javascript frameworks, Cordova/PhoneGap etc. Like Native apps, Hybrid apps are also installed in the device and distributed through the app store. These are good for building apps that do not have a high-performance requirement but need full device access.

Advantages of hybrid apps:

1. Easy to build
2. Much cheaper than a native app
3. Single app for all platforms.
4. No browser needed
5. Can usually access device utilities using an API
6. Faster to develop than native apps.

Disadvantages of Hybrid apps:

1. Slower than native apps
2. more expensive than web apps
3. Less interactive than native apps

Examples of Hybrid Apps:

Hybrid App Architecture

Hybrid app architecture emerged in 2009 but has only become popular recently. This approach achieves the middle ground between native mobile applications and mobile web applications. While mobile web apps attempt to provide platform independence, the price one pays for this is that they do not function when the device is offline and they cannot access device hardware like the camera, Bluetooth, accelerometer, or compass. The Hybrid App approach evolved to deliver platform independence while providing access to devise hardware and offline operation. This is achieved by building apps using HTML5 pages that run in the browser (e.g. uiwebview) embedded inside a native container app. This app then provides a bridge for the HTML5 pages to access the low-level device functions. The hybrid app is packaged as a native app and thus can be distributed from the app store. They can operate offline since the HTML5 pages are typically inserted inside the app; however, a good hybrid app development framework would allow these pages to be refreshed and update the app without having to update the native app container. Just like the Web App approach, hybrid apps leverage software staff with web technology skills while limiting the need for expertise in the native mobile operating system development.

Progressive Web Apps

The term ‘progressive web apps were coined by designer Frances Berriman and Google Chrome engineer Alex Russell in 2015. Progressive web apps are like regular web pages but provide additional user functionalities like working offline, push notifications and device hardware access which was earlier only available to native mobile apps. The great thing about PWAs is that they can be accessed via app icon on the device home screen and as soon as clicked, leads to the app website. PWAs are modern technology aimed at providing a seamless mobile experience. They are ‘native app like’, get automatically updated, are served through HTTPS so are quite safe, they can run fast regardless of operating systems and device types yet providing similar user experience and are easily installable.

Examples of Progressive Web Apps:

The architecture of Progressive Web Apps:

It is required to separate static content from dynamic content for building a PWA. The sole approach to their development is Application Shell Architecture which is the base of the UI. In order to run the app in offline mode, the app shell must contain core design elements required for app development. This approach works well in case of heavy JavaScript apps with one page and with apps whose content keeps changing and but with stable navigation.

TLS –Stands for Transport Layer Security Protocol. It is the standard for secure data exchange between two apps. The website requires serving HTTPS and SSL Certificate on the server.

Advantages of Progressive web apps:

1. Low on Data. An app which takes close to 10 MBs as a native app, can be reduced to about 500KB when made a PWA.
2. PWAs gets updated like web-pages. You get the latest version when you use. No need to update them every now and then.
3. You don’t need to install them to start using. They are simple web-pages. Users choose to ‘install’ when they like it.
4. Sharing is Easy. Unlike an app, you can share a PWA with its URL

Disadvantages of progressive apps:

1. PWA experience on social media is not popular these days as more and more social media companies are making their own in-app browser.

2. Plugins can’t fetch data from Facebook and Google Apps. You need to separately login on the web too.
3. Full support is not available in default browsers of some of the manufacturer's.
4. It cannot use the latest hardware advancements (like fingerprint scanner).
5. Key re-engagement features are limited to Android, such as add to home screen, notifications etc.
6. Traffic from Play Store cannot be directed to the mobile app. You miss significant traffic who use Play store for their primary search.

Xamarin/React native apps

These are essentially a native app built using JavaScript and React or C# in the case of Xamarin. Xamarin is a mobile development platform which supports cross-platform mobile app development. This means you done need to create a separate UI for different operating systems. Xamarin forms is a cross-platform UI tool kit that allows developers to easily create native user interface layouts that can be shared across Android and iOS phones. React native is a JavaScript framework designed for building native apps. It's based on React, Facebook's JavaScript library for building user interfaces targeting mobile platforms.

Advantages of Xamarin/React native apps:

1. Cross-Platform App Development
2. Native User-Interface
3. Less Number of Bugs
4. API Integration
5. Huge community and Support available
6. Shared Code Base

Disadvantages of Xamarin/React native apps:

1. Limited Access to Open Source Libraries
2. Slightly Delayed Support for the Latest Platform Updates
3. Larger App Size
4. Not Suitable for Apps with Heavy Graphics
5. Compatibility Issues with Third-Party Libraries and Tools
let's look at the comparative analysis of the types of mobile apps

Comparison between native vs hybrid vs web apps

Conclusion

Each type of app has its own strong and weak points and it's up to you and your business requirement which type of app you choose. Mobile app developers and testers have to ensure that the app runs smooth and functions well.

Related Articles: