Suyash Dubey | Posted on February 24, 2020March 11, 2020 | 2 min Read

5 Best Practices To Avoid Test Automation Failure

Automation testing forms the core of any CI/CD pipeline and enterprises are keen to practice test automation to enhance the efficiency of the development process. Test automation saves resources and reduces the cost of any project in the long run. But there are some important points to keep in mind while testing to avoid automation failure. Let’s have a look at these salient points.

Leverage Parallel Execution

Once you are done automating the test cases, the challenge will be the complex test suites taking a long time to get executed. It affects the quality of the test queue in the test automation framework or IDE. This leads to queue timeout issues and test cases being halted abruptly due to the sequential execution of the test cases.

Parallel execution in different test environments is preferred over sequential execution as it saves a lot of time. Although in automated testing, unintended code interactions could happen. This is why you need a thorough reporting mechanism to debug the causes of test failure.

Pick The Right Tools

Choosing the right tool for test automation is critical to the success of automation testing. There has to be a set of clear requirements/parameters on the basis of which the tools have to be selected. Some important points that are to be kept in mind while selecting the tool are:

The team should be clear about the test tool requirements.

The testing requirements of the application under test (AUT) should be analyzed thoroughly.

The team’s skill set should be accessed accurately.

The cost-benefit analysis should be performed to calculate the return on investment.

Tool vendor and capability should be evaluated as technical support might be required while using the tools.

One tool might not be enough to meet any organization’s automation needs. Also, test automation engineers have to be a part of the tool evaluation process so that they can help in selecting the right set of tools. For example, you can use Appium for test automation but you need pCloudy to perform automation testing on multiple real devices in parallel.

Analyze The Test Reports

Test reports provide insights into the underlying issues that are to be resolved. A detailed test report gives an idea of the efficiency of the test automation and the automation team can analyze the report to look for the scope of improvement. While selecting an automation tool you need to make sure that the tool generates test reports to be analyzed by the test automation engineer. There will always be some tests that will fail to execute and it is necessary to analyze the test report to get an understanding of the scenario.

Test Automation Metrics

Test automation metrics will help you gauge the quality of the tests performed based on some essential parameters like test duration, unit test coverage, path coverage, number of defects found, percentage of broken builds, etc. The test metrics will give you a clear picture of how well the code is tested. In an agile process, there are frequent iterations to the builds and it becomes important to track the quality of each build. With test automation metrics you can figure out what is the percentage of your tests that passed and what was the reason behind the failed tests.

Optimum Device Coverage

Test automation is effective when the tests are executed on multiple devices in parallel. Device coverage is the most prevalent challenge as we have witnessed growing device fragmentation across the world. To ensure the smooth functioning of mobile apps on all the devices, you need to perform automation testing on hundreds of device-OS combinations.

Test automation should be designed to make the app compatible with most of the popular devices. The frequent release of new versions of OS from both Android and iOS is a major factor that drives device fragmentation. The only way to overcome this challenge is by testing the app on a cloud-based testing platform. In pCloudy, you will get the benefit of testing the app on more than 5000 device browser combinations in parallel ensuring optimum device coverage.

Summing It Up

Test automation has many benefits like better test coverage, faster feedback, and accelerated results which reduces the time to market of any application. Using the practices mentioned above you can ensure coherent test automation and increased productivity. Apart from these points, there are many other things you can do like writing original code and not copying it as the code taken from other sources might not work in your test environment. But you will always find new roadblocks which you will have to deal with spontaneously.

Suyash Dubey | Posted on November 4, 2019November 5, 2019 | 2 min Read

Flutter App Automation Using Appium Flutter Driver

Flutter has been creating waves in the app development space since its launch. The reason is simple, it solves the problem of creating two different codebases for a single app to run on Android and iOS devices. Some experts say that Flutter which was launched by Google is to counter React Native which is used by Facebook. Nevertheless, this is a much-needed relief for enterprises as Flutter will help them save a lot of their time and resources. So let’s first understand what is Flutter.

What is Flutter

Flutter is an opensource mobile UI framework that can create Android and iOS apps from a single codebase. The Flutter framework includes the Flutter engine, Dart platform, the Foundation library, and design-specific widgets. Flutter apps are written in Dart programming language and runs on dart virtual machine.

How to Create a Flutter App

Before we start the process of app development it is important to know that there are two types of widgets, stateless and stateful. A stateless widget describes part of the UI by building a structure of other widgets that describe the UI more definitive. Stateful widgets are the widgets that can change their state multiple times and can be redrawn on to the screen any number of times while the app is in action.

To create a Flutter app you need to install Flutter SDK and other tools required to support the framework. We will create the app in Intelli J IDE and for that, you need to install two plugins. The first one is Dart and the second plugin is Flutter. You can install the plugins by clicking on File in the menu bar of the IDE and then click on Settings.

Then you need to create a new project and you will get a basic code layout to start with your app development. In the first line, we have the main method where we will enable the flutter driver extension and there is a run app method with MyApp class instance, to run the app.

In this sample code, we have used the title as the stateless widget and we have set the title as the key and for the home screen, we are using one more instance which is MyStatefulWidget(). So in this class, we have a small piece of code, where we have set values for elements like appBar, background color, and in the body, we have placed a floating action button for a counter.

After adding the code you can run the program and you will be able to see the changed state of the counter with every click on the action button.

Appium Flutter Driver

Appium Flutter driver is a test automation tool for flutter apps on multiple platforms. It is a part of the Appium mobile test automation tool. But how does the Appium Flutter driver work? It uses webSocket communication to carry WDIO script to instantiate the flutter driver and the request will be sent to the particular application. Then the AUT will send the response to the script through the Appium Flutter driver.

You can also build Appium in your machine through the source code. Once you have cloned the source code, run npm install then npm run built and type a command node to start the Appium server. You can download the package from GitHub. The package contains Appium Flutter Driver and Appium Flutter Finder. You can clone the sample project to get the WDIO code.

In the terminal, you can see that to automate flutter application we have used Flutter as an automation name for both Android and iOS apps. You can see that port is 4723 and in the capabilities, we have mentioned osSpecificOps, automation name: ‘Flutter’. We have mentioned the app path at the top with the device name and platform name and we created an object.

Appium==OS environment is ‘android’ which will pick all the information and specify in the capabilities and it will pass a post request to the server.

Now here we are trying to find the element using CounterTestFinder and a ButtonFinder to click on the button. In the next line, we have created a driver using a remote method.

Then we have switched the Native App context to Flutter. You can switch this if you are using the Flutter app. Once the app is launched, we are validating that the counter text finder has value as zero by calling Flutter APIs internally. So here we are using two actions, one is clicking and the other is touch.

The button is to be clicked two times and thus we have used strictEqual function. In the last line of code, we are deleting the session.

Now to run the test we need to pass the environment variable to connect the device.

Then we will check if our Appium server is running.

Then once you run the code you will be able to see the actions performed on the device and the counter will change to two.

This is how you can automate your Android and iOS app testing using Appium Flutter driver. Although there are many benefits of using Flutter, there are some limitations as well. The first thing is that you need to learn Dart language to develop and automate the Flutter app.

Suppose you need to run automated tests on multiple devices, then you will have to fire multiple commands in the terminal. Appium flutter driver depends on Flutter Driver APIs and as it’s still not matured enough, some actions cannot be performed.

Varun Kumar | Posted on October 23, 2019November 26, 2019 | 2 min Read

Run Espresso on pCloudy using pCoudy Utility

Espresso is a testing framework for Android which automatically synchronizes your test actions with the UI of your app. It also lets the test wait until all the background activities have finished.

The pCloudy utility uploads your Application APK, test APK and other APK files to pCloudy. It will then acquire a device to run your test on, execute those tests and report back the status and test report.

In pCloudy you don’t have to set up the environment. Just follow these simple steps.

After the test cases are run you can see the detailed reports. Once automation ends, the device under test will be released automatically so that you or other users can use it later.

There are two ways you can execute espresso test cases. Here are the steps for both

1. pCloudy UI Dashboard
a. In pCloudy UI there is a section called Automation where you will find multiple automation frameworks integrated to the platform.
b.Select the Espresso automation and to fill in the required details.

c. pCloudy Support two TestRunners.
i. AndroidJUnitRunner (android.support.test.runner.AndroidJUnitRunner)
ii. AndroidXJUnitRunner (androidx.test.runner.AndroidJUnitRunner)

d. Select your Application and Test Application.

e. Select the required devices.

f. Click on Next button and Schedule the automation by clicking on the Schedule button.

g. You will get a dialog box. Click on Ok.

h. You will get the active session details on the Dashboard and you will also be able to see the Live view.

i. Liveview will help you to see the live execution on your test cases. Click on the Liveview icon

2.pCloudy Utility

a. Download the espresso starter pack from here and Unzip it
b. You will find two files,
i. pCloudy_Espresso.jar
ii.Config.properties
c. Place the config.properties file and pCloudy_Espresso.js file into the same directory.
d. Update the config.properties file as guided in the comments in the file.
e. In your workspace, run the command by typing the java -jar pCloudy_Espresso.jar

f. This will upload the required files which you have mentioned inside the config.properties files.

g. You will get the status of the Automation in pCloudy Dashboard ActiveSession.

h. Even on the pCloudy dashboard, you can use the Liveview by which you can see the running test cases.

i. Once the test execution is over, You will get a URL of the test execution report on the terminal.

j. You can open the report from the dashboard Report section also. In the report, all the test details will be there as shown in the screenshots below.

k. You will also get individual device reports. Inside the test case level details, you will find performance details, test case logs, device logs, and video files.

Priyanka Charak | Posted on October 25, 2018July 31, 2019 | 2 min Read

3 Reasons Why Test Automation Should Be Included at the API Level

Importance of Test Automation

QA & testing become indomitable especially in the context of API testing. The development teams benefit from automated integration tests at API levels as it helps them in easy integration of apps and higher product adoption rate, ultimately helping the firm increase their profits. API layer is frequently ignored while developing and executing automated tests and mostly end to end user interface driven tests are executed to check if final application works perfectly after integration. API layer is not to be ignored and for a QA strategy to be successful, it should consider API testing as a fundamental part of the overall testing strategy.

Let us understand what API testing means:

Application Programming Interfaces (APIs) are a set of programs that allow two or more software to communicate with each other and be able to exchange data smoothly. In the testing world, testing APIs is different than other types of testing. UI and functional testing is often given importance while API testing is ignored. The foremost responsibility of a tester is to ensure that API testing is performed in order to attain seamless functioning, performance, and the reliability of different data-driven applications and systems by verifying exchanges and communications between applications, systems, databases, and networks.

While integration testing has been a component of the test automation pyramid since its inception, it’s also been the most undermined layer in functional test automation. All tests that supercede the scope of an individual unit, and can’t be covered by unit testing are often executed through user interface-driven, end-to-end test scenarios. But while end-to-end tests can be seen as the ultimate integration test, where all components come together, having too many of them leads to test suites that take unnecessarily long to execute, and that are hard to maintain and keep stable. It’s often possible to test important parts of an application’s technical and business logic through an API. To understand the importance of API Testing more in depth, let’s understand the undermentioned benefits of Automated API testing:

1. Wider Scope than Unit testing
2. More Stability and Speed as compared to end-to-end tests
3. Introducing API Level Testing more seriously- A significant decision in Automation Testing

Wider Scope than Unit testing:

Unit tests are often run on small group of components within a single application layer. Issues are often not detected in distributed apps where one application layer ends and the next one begins. In order validate if the components are communicating as desired, API level tests are designed. So, the need of a strong API testing Strategy is of dire importance at the integration level especially when it comes to integrating with external components. Managing the testing environment is a critical task to perform especially when the components are with different teams.

More Stability and Speed as compared to end-to-end tests:

It is true that end-to-end tests are vaster in scope than API testing. They cover all the layers and components of the application but API tests cope up with its loss with its stability and speed. API tests are certainly faster than the former. End-to-end tests take longer time to load browsers and screens whereas API tests are based on individual request-response interactions (for example in JSON or XML format)which makes the execution faster with shorter feedback cycles. Moreover, user interfaces tend to change frequently due to various user change requests and adherence to advanced front end frameworks API tends to be more stable in terms of Interfaces that yields fewer faults and requires less maintenance.

Introducing API Level Testing more seriously- A significant decision in Automation Testing

API level integration testing should be a mandatory aspect of any automated testing agenda. It could be an unnerving move for those with no prior expertise in this area. It is mostly the tester’s cup of tea more than the developer’s as it can go beyond the component or application. The absence of user interface can be a blocker in accessing and testing the API. But you can opt from various automated API testing tools available in the market. Tools like RestAssured, SoapUI, and Postman are preferred among the most.

Conclusion

In the era of Artificial intelligence and IoT, the need for API tests is becoming intense. Automated API Testing strategy is the most effective strategy for faster delivery and reduced human inaccuracies. With the help of the tools, more API test requirements could be covered and API testing could be made more advantageous for secured and quality deployment of mobile applications in shorter time. Why don’t you integrate your automation framework using pCloudy’s API and make your app release faster?

Try pCloudy for free now.

Shivani Sinha | Posted on July 6, 2018September 5, 2022 | 2 min Read

pCloudy Among Top 3 Test Automation Software

pCloudy Among Top 3

We are proud to announce that pCloudy has been ranked among the top 3 Test Automation Software products based on the user satisfaction. With this, we have got another feather added in our cap after achieving the 50k milestone. It has become possible only because of your constant trust and support that motivates us to keep growing stronger.

Check out this list of the top Test Automation Software products based on user satisfaction. As per G2Crowd, the product’s satisfaction score is calculated by a proprietary algorithm that factors in real-user satisfaction ratings from review data. Check yourself.

Avinash Tiwari | Posted on October 24, 2017September 5, 2022 | 2 min Read

pCloudy Part of G2 Crowd Grid

G2 Crowd

pCloudy now part of G2 Crowd Test Automation Grid. We thank our customers for making us high performer.

Avinash Tiwari | Posted on September 19, 2016September 5, 2022 | 2 min Read

Automation Execution on Multiple Devices in Parallel

By using pCloudy Test Automation, you can execute your test automation Parallel Runs on Multiple Devices as well.

Navigate to Test Automator page of pCloudy.
Select the filter to choose if execution needs to be done Android or iOS.
Select the devices from the given list. User can select minimum one & maximum all the devices for the parallel run.
Choose the test framework for test automation execution.
Choose the application to be tested and test suite for the same.
If the test suite takes 25 minutes to execute completely on one device locally, then you need to add some buffer say for another 10 minutes and change the time to 35 minutes.
Finally set the test cycle name of your choice and click on the Schedule button.
Upon confirmation the same the platform will schedule the automation execution selected devices.
When test automation process gets completed on all devices, you will be informed through a mail with the link to see the test report.

Avinash Tiwari | Posted on August 28, 2016September 5, 2022 | 2 min Read

Robotium Automation Execution

Robotium automation testing tool for Android

pCloudy allows you to run your Robotium test script over android devices. Robotium is an open-source test framework for Android applications.

Here are steps to run Robotium test script over android device:

Login over www.pcloudy.com with your registered Email ID & Password
Go to the Automator Page
Select Android OS version & Manufacturer
Select and add desired device
Choose Robotium (to run Robotium test scripts)
Select Application .apk file (the App to be tested)
Select Test Script .apk file (the Test Script developed for testing)
Enter Single Device execution Time (Time required for the entire script to be executed on single device)
Enter Name of your Test Cycle (this name will represent your Test Cycle)
Click on Schedule Test Cycle

In the next step, you need to confirm schedule of test automation. Here, you can see your selected application, test script, device etc.

Further, you get a pop up message of test automation schedule confirmation.
Click on OK to proceed.

Finally, you get Cloud Automator Report of your scheduled test automation.

Test Automation

Test Infrastructure

Test Management

Tag: test automation