Mobile App Testing Comparison

 
Mobile App Testing Comparison

Apps are the ‘new thing’. We are living in an era, where, you click on your mobile screen for your evening cup of ‘cuppa’. With the advent of technology and internet, mobile phones and services are getting personalized like never before! And to compliance these services, numerous number of apps are appearing every day to suit different platforms. Among the apps, the demand for consumer apps is the highest, but various enterprise applications are also evolving to support various businesses. Due to the ever-growing demand for mobile apps, app makers and developers are shifting their focus towards creating strategies and roadmap before launching their applications. It is easier to launch an app with features and functionality required by a particular service, but it is pretty critical to have a rigorous and a flawless mobile testing plan before the deployment.


70% of Apps

Testing is a crucial step, where developers can ensure the quality of the app they are going to release. No one wants to release an app full of bugs, or which allows a lot of pop-ups. A faulty GUI, or a confusing UX, can be the reasons for the sad demise of your app before it creates its own space in the app market. That’s why mastering the art of mobile app testing has become the primary objective for any developer to sustain in the market.

Mobile App testing throws many new kinds of challenges

  • There are numbers of real devices available in the market and for which developers have to go through large numbers of tests

Device Fragmentation

  • There are frequent changes in mobile apps. It has become difficult to keep pace with the changes.
  • Availability of the right Mobile App Infrastructure, the right device matrix and the right tool set is a very selective process!
  • Simulating or reproducing bugs reported by users are extremely difficult as they are mostly related to a specific device.. Bugs are always progressive in nature. And that’s why the cost of development increases significantly.

40% of bugs

Current Approach

The typical testing approach involves use of simulators or emulators with a combination of internal mobile lab (selected set of devices). These approaches don’t provide full testing coverage. Bugs are found when an app is launched in the market. It leads to overall high cost of quality. Organizations have started to look at Automation to reduce cycle time and increase test coverage. But, without a proper Mobile lab infrastructure, it doesn’t yield good result.

Solution: Future is cloud

Blog site e-testing notes, “Cloud-based testing makes it possible to collaborate more effectively, to test across multiple platforms with less hardware, and to spend less on testing efforts, both long and short term.”

Why is cloud based Mobile App Testing the way to go?

  • Produces better quality apps by testing on the widest possible variety of devices available, while reducing capital budgets.
  • Cloud offers a ready infrastructure for Automation runs, which results an Increase in test coverage.
  • Quickly reproduce bugs reported from the market.
  • It allows easy collaboration between various teams.
  • Boosts the Agile development process by allowing easy integration within a continuous process.
  • It saves cost as the same lab can be accessed by geographically distributed teams.
  • Cloud lets you access devices which have been stopped in the market. For example, there are still many users in India use the old Android versions like 2.3 on phone, like Samsung Galaxy Star Duos.

20% of devices

Conclusion:

Effective use of cloud can shorten cycle times, improve app quality, and reduce capital budgets by bringing order to the chaos of mobile app development and testing.

pCloudy Appium integration

 

1. pCloudy-java-Connector for Appium

 

1.1 Overview

Appium is an open source test automation framework which is used with wide range of mobile apps that supports only android/ios or both. pCloudy is a popular mobile testing platform integrated with Appium script which enables you to automate test of mobile apps as well. If you have your Appium Scripts (or you plan to build them) in Java then you can use the pCloudy-java-connector.jar to integrate with the pCloudy platform. In case, you don’t have your Appium script then, you will need to use the rest web services APIs which are given in the end of this document.

 

1.2 Objective of this document

This user help guide has been developed for pCloudy end users. This help guide will help you to understand the APIs that pCloudy platform provides to integrate your existing Appium Scripts with pCloudy. After the integration, you will be able to execute your Appium scripts from your PC directly to the pCloudy real device cloud.

 

1.3 System Requirements

  • Java based Appium Script
  • Eclipse or Netbeans
  • >APK or IPA file
  • pCloudy Account
  • pCloudy-java-connector jar

NoteThe APIs are evolving and we will come back with a better set of APIs & the jar soon. You might need a little tweaking in the code.

 

1.4 Stepwise Workflow of the APIs for getting an Appium Driver

 

The basic steps to get APIs for Appium Driver are given below:

appium testing

 

1.5 How to get my API Key for using the services

If you are a registered user then you just need to follow few simple steps to get API key for using our services.

Login to pcloudy.com –

 

Appium

 

After successful login, Instant Access page opens. Now, you finds different devices which can be filtered from the above given dropdown list of devices. From this page, you can book your device by clicking on Book your Device link. Furthermore, you can connect with any given device to perform mobile testing but you will be given 10 minutes only. While performing testing operation on selected device, you cannot perform testing on other device simultaneously.  For this, you need to release previously connected device.

When you click on settings option, support page opens by default which helps you to drop support ticket. It also includes various options like support, History, Test Runs, UDID, Profile, Credits, Unsubscribe, and API.

 

1.6 How to integrate my existing Appium Scripts on Java with pCloudy

You need pCloudy java connector jar file and corresponding APIs for Appium based mobile testing platform, pCloudy. You should add the pCloudy-java-connector.jar in its build path as a reference within the your java project containing appium scripts. See this link to learn how to add reference in eclipse.You can download pCloudy-java-connector.jar from this link –

http://support.pcloudy.com/solution/articles/9000062255-adding-reference-to-jars-in-eclipse

While downloading such file, you may come across message as given below which alerts you regarding safety of your computer.

 

pCloudy java connector jar file

 

Now, you can reference the JAR file in Eclipse or Android studio. Please refer the link.

The pCloudy-java-connector.jar comprises the following methods which gives you a wrapper code over the web services. Furthermore, you will not need to call the Rest APIs if you are working on JAVA:

 

Appium Automation Testing

 

1.7 Useful Codes to get Web-Driver Object

 

A. Run on Android Native

 

public void runExecutionOnPCloudy() {
Connector pCloudyCONNECTOR = new Connector();

// User Authentication over pCloudy
String authToken = pCloudyCONNECTOR.authenticateUser(“abc@gmail.com”, “asd53sd3fa4asd5fsasda”);
ArrayList selectedDevices = new ArrayList<>();

// Populate the selected Devices here
selectedDevices.add(MobileDevice.getNew(“Gionee_Ctrl-V6L_Android_4.4.2”, 73, “Ctrl-V6L”, “Ctrl V6L”, “android”, “4.4.2”, “Gionee”));

// Book the selected devices in pCloudy

BookingDtoDevice[] bookedDevicesIDs = pCloudyCONNECTOR.bookDevicesForAppium(authToken, selectedDevices, 5, “friendlySessionName”);
System.out.println(“Devices booked successfully”);

// Upload apk in pCloudy
PDriveFileDTO pDriveFile = pCloudyCONNECTOR.uploadApp(authToken, new File(appPath));
System.out.println(“ipa file uploaded successfully”);
pCloudyCONNECTOR.initAppiumHubForApp(authToken, pDriveFile);

// Get the endpoint from pCloudy
URL endpoint = pCloudyCONNECTOR.getAppiumEndpoint(authToken);
System.out.println(“Appium Endpoint:” + endpoint);

// Create multiple driver objects in multiple threads
for (int i = 0; i < bookedDevicesIDs.length; i++) {
Thread.sleep(5000);
new Thread(getTestCaseClass(endpoint, bookedDevicesIDs[i])).start();
}
}

private Runnable getTestCaseClass(final URL endpoint, final BookingDtoDevice dto) {

// this will give a Thread Safe TestScript class.

// You may also like to have this as a named class in a separate file

return new Runnable() {
@Override

public void run() {
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability(“newCommandTimeout”, 600);
capabilities.setCapability(“launchTimeout”, 90000);
capabilities.setCapability(“deviceName”, dto.capabilities.deviceName);
capabilities.setCapability(“browserName”, dto.capabilities.deviceName);
capabilities.setCapability(“platformName”, “Android”);
capabilities.setCapability(“appPackage”, appPackage);
capabilities.setCapability(“appActivity”, appActivity);
capabilities.setCapability(“rotatable”, true);
AppiumDriver driver = new AndroidDriver(endpoint, capabilities);

// Your Test Script Goes Here…

}
};
}

 

B. Run on Android Web

 

public void runExecutionOnPCloudy() {
Connector pCloudyCONNECTOR = new Connector();

// User Authentication over pCloudy

String authToken = pCloudyCONNECTOR.authenticateUser(“uttam.kumar@sstsinc.com”, “sadf54sad65fds6sdsdsw”);
ArrayList selectedDevices = new ArrayList<>();

// Populate the selected Devices here
selectedDevices.add(MobileDevice.getNew(“Gionee_Ctrl-V6L_Android_4.4.2”, 73, “Ctrl-V6L”, “Ctrl V6L”, “android”, “4.4.2”, “Gionee”));

// Book the selected devices in pCloudy

BookingDtoDevice[] bookedDevicesIDs = pCloudyCONNECTOR.bookDevicesForAppium(authToken, selectedDevices, 5, “friendlySessionName”);
System.out.println(“Devices booked successfully”);
pCloudyCONNECTOR.initAppiumHubForBrower(authToken, “Chrome”);

// Get the endpoint from pCloudy
URL endpoint = pCloudyCONNECTOR.getAppiumEndpoint(authToken);
System.out.println(“Appium Endpoint:” + endpoint);

// Create multiple driver objects in multiple threads
for (int i = 0; i < bookedDevicesIDs.length; i++) {
Thread.sleep(5000);
new Thread(getTestCaseClass(endpoint, bookedDevicesIDs[i])).start();
}
}

private Runnable getTestCaseClass(final URL endpoint, final BookingDtoDevice dto) {

// this will give a Thread Safe TestScript class.

// You may also like to have this as a named class in a separate file
return new Runnable() {
@Override

public void run() {
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setBrowserName(“chrome”);
capabilities.setCapability(“newCommandTimeout”, 600);
capabilities.setCapability(“deviceName”, dto.capabilities.deviceName);
capabilities.setCapability(“platformName”, dto.capabilities.platformName);
AppiumDriver driver = new AndroidDriver(endpoint, capabilities);
// Your Test Script Goes Here…
}
};
}

 

C. Run on IOS

public void runExecutionOnPCloudy() {
Connector pCloudyCONNECTOR = new Connector();

// User Authentication over pCloudy
String authToken = pCloudyCONNECTOR.authenticateUser(“uttam.kumar@sstsinc.com”, “sadf5sd5fds5sfd5a5fdas1”);
ArrayList selectedDevices = new ArrayList<>();

// Populate the selected Devices here
selectedDevices.add(MobileDevice.getNew(“Apple_iPad2_Ios_7.1.2”, 70, “iPad2”, “iPad 2”, “ios”, “7.1.2”, “Apple”));

// Book the selected devices in pCloudy
        BookingDtoDevice[] bookedDevicesIDs = pCloudyCONNECTOR.bookDevicesForAppium(authToken, selectedDevices, 5, “friendlySessionName”);
System.out.println(“Devices booked successfully”);

// Upload apk in pCloudy
PDriveFileDTO pDriveFile = pCloudyCONNECTOR.uploadApp(authToken, new File(appPath));
System.out.println(“ipa file uploaded successfully”);
pCloudyCONNECTOR.initAppiumHubForApp(authToken, pDriveFile);

// Get the endpoint from pCloudy

URL endpoint = pCloudyCONNECTOR.getAppiumEndpoint(authToken);
System.out.println(“Appium Endpoint:” + endpoint);

// Create multiple driver objects in multiple threads
for (int i = 0; i < bookedDevicesIDs.length; i++) {
Thread.sleep(5000);
new Thread(getTestCaseClass(endpoint, bookedDevicesIDs[i])).start();
}
}

private Runnable getTestCaseClass(final URL endpoint, final BookingDtoDevice dto) {
// this will give a Thread Safe TestScript class.
// You may also like to have this as a named class in a separate file

return new Runnable() {
@Override
public void run() {
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability(“newCommandTimeout”, 600);
capabilities.setCapability(“launchTimeout”, 90000);
capabilities.setCapability(“deviceName”, dto.capabilities.deviceName);
capabilities.setCapability(“browserName”, dto.capabilities.deviceName);
capabilities.setCapability(“platformName”, “ios”);
capabilities.setCapability(“bundleId”, bundleId);
IOSDriver driver = new IOSDriver(endpoint, capabilities);
// Your Test Script Goes Here…
}
};
}

 

2. Rest APIs if needed to be used in platforms other than Java

 

https://s3.amazonaws.com/pcloudy-content-distribution/v2/API_USAGE_V2.pdf

What’s New with 3.0

 

We have been continuously working towards a mission to create a world class yet affordable platform for testing of apps in connected world.

In this journey we are committed to bring you best of features to make your App testing quicker, easier and cost effective.

I would like to introduce PCloudy 3.0, our latest release. So, what’s new here?

IOS support

We have added latest IOS devices on our platform which means you can test your IOS apps on real devices exactly the same way you tested your Android apps. You can find latest devices like IPhone 6S, IPhone 6 plus and IPad Air 2. Moreover, you can find IOS devices with older versions as well.

IOS devices

Appium support

Running your Mobile App automation was never so easy. Now you can focus on building your scripts. Leave the execution to us. Just create your Appium scripts in Eclipse or using your own framework. Insert a small piece of code within your script and you are ready to run your scripts on multiple devices in parallel.

Appium support

Test your Apps on real Network

We have enabled real network on many of our devices. Now, you can test your Apps across 2G, 3G, 4G and wifi across various network providers.
You can switch between networks while executing your test cases.

On-device performance Metrics:

This has been a big pain area of testers and developers. How to measure and analyze resource consumption (CPU, Memory and Network consumption) for App on a real device. Well, it’s a click away on our platform.

On-device performance Metrics

Latest Devices:

We are constantly updating our device matrix to provide our customers latest and greatest devices available in the market.


Improved Performance:

Our commitment is to provide you a working experience as close as possible to a real device. We have done many perf improvements on our platform. Moreover, we moved to a new Data center. It’s an ISO 27001 certified Data center with 99.99% uptime commitment.