Suyash Dubey | Posted on | 2 min Read

Testing framework plays a crucial role in the success of any automated testing process. Choosing the right test automation framework is important as it will maximize the test coverage and improve test efficiency which means a better return on investment.

There are some key points you need to keep in mind while choosing a suitable python testing framework. The framework should justify your testing needs and it should be easy to use. Check if the framework has integrations with other tools and frameworks that you might use. The features, support, stability, and extensibility are also important. So let’s compare the most popular python testing frameworks to make it easier for you to choose the right one.

Robot framework

It is still the most popular python testing framework that uses a keyword-driven testing approach and is used for acceptance testing. To run Robot you will have to install python 2.7.14 or any later version, python package manager, and a development framework like Pycharm.
Advantages

  • Opensource
  • Platform independent
  • No need to learn a programming language to write Robot Framework test cases
  • Automatic report generation after executing each built
  • Supports behavior-driven, data-driven and keyword-driven approaches
  • Easy installation

Disadvantages

    • Not enough support for parallel testing
    • It’s difficult to create customized HTML reports

Gauge

It is an opensource tool developed by the team that made Selenium. Gauge is immensely useful while integrating continuous testing in the CI/CD pipeline. It is gaining popularity as it supports many plugins like python runner, IDE plugins, build management, java runner, etc.
Advantages

      • Quick defect detection
      • Easy to write test cases
      • Supports multiple programming languages
      • Command-line support
      • Supports all major plugins
      • Cross-browser tests can be automated

Disadvantages

      • It is relatively new so it will evolve in the coming years

Pytest

Although Pytest is used for different types of testing, it is more preferred for functional and API testing. There are no prerequisites needed for Pytest, just knowledge of python will be enough to get started. It has a simple syntax which makes test execution easier.
Advantages

      • Supports Fixtures and Classes that help in creating common test objects available throughout a module
      • It allows the use of multiple fixtures
      • It does not require a debugger
      • Xdist and other plugins support makes parallel execution easier
      • It supports parameterization, which is essential while executing the same test with different
      • configurations using a simple marker
      • Large community support

Disadvantages

      • Test written in Pytest cannot be shared with other platforms

Pyunit

It is a unit testing framework much like Junit but for python language. Also referred to as unittest, it has five core modules. The test loader class is used to load all the test cases and test suites. The test runner shows the result of the test executed using an interface. The test suite is a collection of test cases that are clubbed logically based on the functionalities. A test case contains the actual implementation of the code and the test report contains the organized data of the test results.
python automation
Advantages

      • No need for high-level python knowledge for test execution
      • Extensive report generation
      • Pyunit comes with Python package, no need to install any additional module
      • Simple and flexible test case execution

Disadvantages

      • Requires boilerplate code
      • Pyunit is derived from Junit and so it still uses camelCase naming instead of snake_case naming method
      • It supports abstraction so the code intent sometimes becomes unclear

Behave

In Behave, test cases can be written in simple language and lets teams execute behavior-driven development (BDD) testing with ease. Behavior-driven development encourages quality analysts, developers, and business managers to work in collaboration to achieve higher efficiency.
Advantages

      • Easy execution of all kind of test cases and easy coordination
      • Better clarity on the developers and testers output as the format of the spec is similar
      • Domain vocabulary that keeps the behavior consistent in the organization and the system behavior is expressed in a semi-formal language
      • Detailed reasoning and thinking promotes better product specs

Disadvantages

      • Only for black-box testing

To sum it up

All the above-mentioned frameworks have their specialties like Pyunit is used for unit testing and Behave is good for behavior-driven testing. Although Robot framework is the best tool for a beginner to learn the nuances of automation framework. It’s always better to jot down your requirements based on their priority and then choose the right python testing framework.

Suyash Dubey | Posted on | 2 min Read

In the year 2028, there will be around 7.8 Billion mobile users which accounts for 70% of the world population. More mobile users mean more apps and more competition and to lead the competition we need to make sure that our app is flawless. If nearly half of the bugs in your mobile app are discovered by the users, your app’s ratings are going to decline and so are the downloads. This is why the right choice of mobile app testing techniques must be followed in the decision-making process.

Mobile App Testing Strategies

Today, the mobile app market is highly competitive. To be better every day and survive for long, the QA team has to follow a mix of plans that would be responsible for taking the right testing decisions. The testers have to formulate testing strategies to face every situation fearlessly and immaculately. Mobile apps have to be perfect before reaching to the end users so there have to be certain decisions to be taken regarding the testing plan. The following model of mobile app testing plans can be considered for better execution.

In the planning Stage, decisions like Selection of Device matrix, Test Infrastructure (In-house vs. Cloud, Simulator vs. Real device), Testing scope, Testing Tools, Automation (Framework/Tool) are taken. Since it is the first stage, it is the most important one as all the further stages would depend on these decisions. In the next stage which is execution and review, decisions regarding Test Case Design, Testing of user stories, testing types as per Sprint Objective, Progressive Automation, Regression Testing, Review and course correction are taken.

We are going to discuss the planning stage aspects more elaborately

Device Matrix:

It is an important factor, choosing the device as per your target audience’s behavior matters in decisions regarding resting. There are different approaches to the selection of the device matrix.

Approach 1- Selection of Devices based on market research.

Determine the set of devices with your target operating System that will have the highest occurrence of accessing your application by using app purchase data and analytics. For Example- if you support both Android and iOS, and your application will be used across millions of Samsung, Google Nexus and Moto G devices but only thousands of iPhones, you prioritize testing on the Google Nexus and Moto G above the iPhone device. So, this test plan will consist of testing on devices which are prioritized by your market analysis.

Approach 2: Categorize the devices based on Key mobile aspects

This approach highlights the categorization of the devices based on certain mobile aspects which can be considered in formulating the testing strategy. The categorization goes as:
Mobile device categorisation

Test infrastructure

This is another element of the planning stage. This focuses on Strategizing on the Infrastructure components like hardware, software, and network which are an integral part of test infrastructure. It ensures that the applications are managed in a controlled way.

Real device, Emulators or Mobile cloud-Where to test?

Choosing the right platform to test as per the testing needs is very important i.e whether to test on the Real device or an emulator or on the cloud

Real Devices

Testing on a real device is anytime more reliable than testing on a simulator. The results are accurate as real-time testing takes place on the device in a live environment. It carries its own disadvantages as it is a costly affair and not all the organizations are able to afford a complete real device laboratory of their own.

Pros:

Reliable- Testing on Real devices always gives you an accurate result

Live Environment- Testing on real devices enables you to test your application on the actual environment on which your target audience working on. You can test your application with different network technologies like HSPDA, UMTS, LTE, Wi-Fi, etc.

User experience- Testing on Real devices is the only way to test your Real-time User experience. It cannot be tested through Emulators or devices Available on Cloud.

Cons:
Maintaining the matrix- You cannot maintain such a huge matrix of mobile devices in your own test lab.
Maintenance- Maintaining these physical devices is a big challenge for organizations.
Network providers- There are more than 400 network providers all over the world. Covering all these network providers in their own test lab is impossible.
Locations- You cannot test how your application behaves when it is used in different locations.

Emulators

The emulator is another option to test mobile apps. These are free, open source and can be easily connected with the IDE for testing. The emulator simulates the real device environment and certain types of testing can be run on it easily. However, we cannot say that the results of emulators are as good as those of real devices. It is slower and cannot test issues like network connection, overheating, battery behavior, etc.

Pros:

Price- Mobile emulators are completely free and are provided as part of the SDK on every new OS release.

Fast- As Emulators are available on the local machine so they run faster and with less latency than Real devices connected to a local network or devices available on the cloud.

Cons:

The wrong impression- Even if you have executed all test cases on emulators, you cannot be 100 % sure it will actually work in the real environment.

Testing Gestures- Gestures like Pinching, Swipe or drag, long press using the mouse on simulators are different in using these gestures on real devices. We cannot test these functionalities on emulators.
Can’t test Network Interoperability- With the help of Simulators you cannot test your application with different network technologies. Like HSPDA, UMTS, LTE, Wi-Fi, etc.

Testing on Mobile Cloud

Mobile cloud testing can overcome the cost challenges like purchasing and maintaining mobile devices. It has all different sets of device types are available in the cloud to test, deploy and manage mobile applications. The tests run virtually with the benefit of choosing the right type device-OS combinations. Privacy, security, and dependency on the internet can be a challenge in this case but it has many benefits that can cater to different testing scenarios.
Mobile cloud

The organization can choose the right mix of above-mentioned platforms as every platform carries its own advantages and disadvantages. Sometimes a combination of real and emulators is preferred and sometimes all three can be considered as per the testing strategy.

Pros:

Devices Availability- Availability of Devices and network providers is a big gain for cloud users.
Maintenance- When you are using cloud services. Forget about maintenance. These providers take responsibility for maintaining these devices.
Pay per use- You don’t need to buy a device. You only have to pay for the duration you use that device.

Parallel Execution- You can test your complete test suite on multiple devices.

Cons:
Cost- Some providers are a bit costly

Automation Tools for Mobile App Testing on Android and iOS

Nowadays, there are so many automation tools available in the market. Some are expensive and some are freely available in the market. Every tool has its own pros and cons. Choosing the right tool for testing would reduce the QA team effort providing seamless performance at the same time. We will discuss the best mobile app testing automation tools for iOS and Android platforms in 2018.

1. Appium: It is one of the preferred MAT tools by testers. It is open source and free tool available for Android and iOS. It automates any mobile app across many languages and testing frameworks like TestNG. It supports programming languages like Java, C# and other Webdriver languages. It provides access to complete back end APIs and database of the test codes.
Top Features:
-Appium supports Safari on Ios and Other browsers on Android
-Many Webdriver compatible languages can be used such as Java, Objective-C, JavaScript to write test cases
-Support languages like Ruby, Java, PHP, Node, Python.

2. Robotium: It is a free Android UI testing tool. It supports in writing powerful black box test cases for Android Applications. It supports Android version 1.6 and above. The tests are written in Java language and basically, Robotium contains a library of unit tests. Apart from this, Robotium takes a little more effort in preparing tests, one must work with program source code to automate tests. Robotium does not have play record and screenshot function.

Top Features:
-The tests can be created with minimum knowledge of the project
-Numerous android exercises can be executed simultaneously.
-Syncronises easily with Ant or Maven to run tests.

3. Calabash: It is an open source MAT tool allowing testers to write and execute tests for Android and iOS. Its libraries enable the test codes to interact with native and hybrid apps. It supports cucumber framework which makes it understandable to non-tech staff. It can be configured for Android and Ios devices. It works well with languages like Ruby, Java, .NET, Flex and many others. It runs automated functional testing for Android and ios. It is a framework that is maintained by Xamarin and Calabash.

4. Espresso: It is a mobile app testing automation tool for Android. It allows writing precise and reliable Android UI tests. It is a tool targeted for developers who believer automated testing is an important part of CI CD process. Espresso framework is provided by the Android X Test and it provides APIs for writing UI tests to simulate user interactions on the target app. Espresso tests can run on Android 2.33 and above. Provides automatic sync of test actions with the app UI.

5. Selendroid: An open source automation framework which drives off the UI of Android native, hybrid and mobile web application. A powerful testing tool that can be used on emulators and real devices. And because it still reuses the existing infrastructure for web, you can write tests using the Selenium 2 client APIs.

6. Frank: Is an open source automation testing tool for the only iOS with combined features of cucumber and JSON. The app code needs not to be modified in this tool. It includes Symboite live app inspector tool and allows to write structured acceptance tests. It is tough to use directly on the device but is flexible for web and native apps. It can run test both on simulator and device. It shows the app in action by showing its recorded video of test runs.

Above are a few promising, popular and most commonly used and mobile app testing automation tools. Choice of tools certainly resolves many testing-related problems faster and efficiently. Implementing these tools requires skill and experience and so an organization needs to have a proper testing team in place to make all of this possible.
Related Articles:

Suyash Dubey | Posted on | 2 min Read

Today, there are many smartphone users in the world and so is the popularity of mobile apps. In order to be competent enough, mobile apps have to be unique and should provide the best user experience to increase the user base. With the users getting more informed and intelligent, the apps built should keep up with the pace. In order to be impeccable, the mobile app should undergo a rigorous testing process and during that process, the testing team faces many challenges in this aspect which will be covered in this blog. But before we dive in, let’s look at the different types of apps that are available in the market.

Types of mobile applications

The creation of mobile applications is a fantastic approach to boost brand recognition, attract new clients, and improve the user experience for existing customers. In light of this, let’s examine the three primary categories of mobile apps: native, web, and hybrid.

Native apps:

Native mobile applications are ones created exclusively for a given operating system. As a result, software created for one System cannot be used on another, and vice versa. Native applications are more effective, quick, and offer greater phone-specific functionality. Thus, the difficulties of testing mobile apps for compatibility with native user interfaces of devices involve ensuring that such traits are preserved strictly.

Web apps:

Similar to native apps, web applications do not require users to download them. Instead, the users’ web browsers on their phones can access these apps because they are incorporated within the website. So, it is envisaged that web applications will operate flawlessly across all platforms. Testing teams must carefully examine the application on a wide range of real devices and browsers to ensure high app quality. Yet in addition to taking a lot of time, this operation is essential because failing to work on a few devices can severely reduce the app quality and incur heavy losses when the app doesn’t function as required.

Hybrid apps:

The features of both online and native apps are available in hybrid apps. These are essentially web applications that mimic native apps in design. These applications are easy to maintain and load quickly. Teams that test mobile apps are in charge of making sure hybrid applications don’t lag on some devices. Any operating systems with the capacity to support the said features have access to all their functionality.

While each of these app types are slightly similar to each other the technical teams face a different challenge with each type of mobile application. Combining these challenges greatly increases the complexity, making the entire procedure laborious and time-consuming. Let’s quickly look into what some of these challenges are?

Challenges in Mobile App Testing


Different Operating Systems and their versions

There are different types of operating systems available in the market such as iOS, Android, Windows etc. Also, these OS have different versions too. So, it becomes challenging to test so many versions of the mobile app in a shorter period of time. One app that works well in one type of OS may not work well in the other. It is very important to test the application with all supported platforms and their version because we don’t know where the user is going to install the application. As per research, iOS users upgrade quickly as compared to Android but in Android the device fragmentation is larger. That means the developers have to support older versions and APIs and testers also have to test accordingly.

2019-03-27 (1)


Device Variations: Based on Screen size

Android comes with a mix of features and variations in pixels densities and ratios which varies in each screen size. Even in the case of Apple, the screen new size was introduced with the launch of the iPhone 6. Now, it is not just about being picture perfect screen design rather designing an adaptive screen design. Well with such a variety in screen sizes, the role of the tester becomes serious as they need to check if all the features are working well in different screens and pixel and aspect ratios are maintained well.

devices-screen-sizes


Based on the number of Devices

The picture below shows the number of devices in the market by different brands. The number of device manufacturers has increased. According to OpenSignal, there are around 1294 distinct Android phone manufacturers alone, imagine if we add up other brands. The pace with which this data is increasing is a bit alarming for the testers as the testers have to check the app performances on different devices, they would probably need a device library to do the same. The challenge remains in context to functionalities like Complex user interactions on touch screen and keypad devices as well. Having a device library is certainly is a costly affair unless emulation is adopted which can simulate multiple device types and testing can run easily on it.

OpenSignal-Brand-Fragmentation

Image Source: venturebeat.com


Various Networks

The QA team also faces challenges when it has to test the devices connected to different networks. Generally, there are 2G, 3G,4G mobile data available. These provide different data transfer speed and transmission. These varying speeds of the networks by various providers remain a challenge for the testers even today. In this case, testers have to check that the app must perform well at different network speeds and connectivity quality and a check on bandwidth usage of the app. This remains a challenge as it is partially controllable based on different network providers and connectivity access in different geographies.

 

Frequent OS releases

Mobile Operating Systems keeps changing. Both Android and iOS have more than 10 versions of their operating systems. They keep enhancing and updating their versions for better performance and user experiences. This frequent OS release comes as a testing challenge as the testers needs to validate the complete application with each of new OS release. It is very important to test the application with the latest OS release otherwise the app performance would be a major issue and consequently loss of users using the app.

screen-shot-2018-07-12-at-1.35.22-pm

Script Execution

Another major challenge of mobile testing is what we call scripting, the method of defining a test. Script execution can either be manual or automated. You can write down the scripts in a document, which is then used by a test engineer who manually interacts with the test environment to determine the result, else you can run automated scripts that in turn drive interaction with the device and app, and record the results.

 

Automated scripting needs to be kept away from the device to be of any real use because there are so many different devices with different interface options. A script that follows strict keystrokes on an Apple iPhone would not have any chance of working on a Samsung device, because the UI is different. Fortunately, most real device automated testing software provides high-level scripting that operates on the text, image, or object layer. Device emulators can automate testexecution using a higher-level, abstracted scripting language that is not device dependent. When you use automated scripting, the cost of setting up the script will typically be higher than the cost of a single manual execution of a test. But if it is a test script that you run on a periodic basis, every time that you subsequently run the script, the more time and effort you will save. You will eventually recover the cost of initial scripting If you run the script enough.

 

So to conclude, to build a better user experience, an app tester needs to work had in overcoming the challenges of testing. By adopting some analytical skills and methods, testers can really cope up with these situations. For eg. Testing only those apps and OS which are mostly used by their user segment, by adopting a strong testing strategy to take situational decisions eg. Decisions regarding when to choose Automation and manual testing. Strategically, the challenges can be overcome.

 

Screen Size

The Android world is not simple. The variety of different aspect ratios and pixel densities can be overwhelming. With the launch of iPhone Xs Max which has a screen size of 6.5 inches, Apple brings new screen sizes to the iOS world as well. Though iOS developers are used to pixel perfect screen design, they now need to change their mindset to the adaptive screen design instead. For testing, it means that we need to check on various devices that all the necessary screen elements are accessible with different screen sizes and aspect ratios. There are many phones with a screen size of 5 inches which are still popular.

 

2019-03-28

Security Issues

Traditional testing tools like selenium and QTP weren’t designed with cross-platform in mind. Automation tools for web apps and mobile apps are different. Operating systems especially Android further adds to the complexity with API level fragmentation. The most common automation testing tools for mobile app automation testing are Appium and calabash. Each tool has it’s own advantages and disadvantages and you need to choose on the basis for your app’s functioning.

Weak Hosting Controls is one of the most common issues. The server on which your app is hosted should have security measures to prevent unauthorized users Weak Encryptions can lead to data theft which will impact the trust factor of the users. Most of the mobile apps require user data such as email ID, password, age, location etc. This data should be encrypted and stored with proper security. Hackers often use this kind of data to get money out of users account online. Encryption will make it difficult for anyone unauthorized to intrude and retrieve that data rather than keeping it in plain text.

Power consumption and battery life

We haven’t seen much innovations in the mobile battery but the mobile usage and specifications are increasing rapidly. People are using more apps nowadays and the apps are more complex than ever. This is why testers need to test the apps power consumption because if the apps use lots of CPU cycles and some apps will also run in the background than the battery will drain out quickly. We need to make sure that the app uses less battery power so that users can use it for a longer period of time.

2019-03-27

Conclusion

Mobile apps are evolving with device technology and user expectations. Developers are emphasizing on reducing the app size and battery usage. Testers play a major role to ensure that the app works smoothly and does not crash or have bugs. This is why testers must be aware of the latest trends in mobile app testing to deal with the mobile app testing challenges.


whitepaper

Related Articles:

Priyanka Charak | Posted on | 2 min Read

Appium Mobile Test Automation

 

As we all know that Appium is the most preferred test automation tool for mobile applications. It is the first choice of the testers because of its flexibility i.e. it is open source, it has the best supported and highly active community of experts, Appium works across different platforms and works well with different scripting languages. Even after gaining such popularity and having a strong community base, surprisingly, the users still make mistakes while running the mobile test automation with Appium.

 

Here are a few common mistakes that Appium users encounter while using Appium Mobile Test Automation Tool:

 

1.Unrestricted X Path Usage:

Over usage of X Path could be found in case of Selenium as well but when it comes to Appium world, it has more outrageous effects because X Path is a more dynamic way to unearth the elements. But the biggest stumbling block in this scenario is its huge performance cost due to which it becomes elusive an area. This is because Google and Apple do not provide XML or XPath related queries in a way we would need them to be. All of this hospitalizes an unmanageable cost finding elements using X Path. Undoubtedly, X Path is the most trusted method but there are several other better locator methods like ‘Accessibility IDs’ that can be used in this sitch.

 

2. Neglected usage of Accessibility IDs:

The accessibility ID locator strategy is formulated to read a unique identifier for a UI element. For both iOS and Android, getting an element by their accessibility ID is the best method. Most importantly, it is preferable because it is quicker. It should be noted that semantically accessible IDs are different than the Web IDs. However, the two shouldn’t be combined. In many cases, the accessibility IDs are used only for the purpose of testing even though they have a larger purpose. So, in order not to spoil the accessibility of the applications just for the sake of testing, the bigger purpose of the accessibility IDs should be known. If the accessible IDs are set up on the elements to make the apps testable, the app accessibility also increases, provided that the accessibility labels are sensible to the users using them. But the foremost criteria not to make automation efforts a failure is to make the application testable in the first place.

 

3. Not making a testable App:

Developing an app should be a forecasted move where in the developers, even before writing the first line of code, plan to develop an app keeping the automation in mind. This they can achieve by keeping in mind the hooks and the unique IDs for the elements in order to make the app more testable. This strategic approach would be a reason for a successful mobile app test automation. Apart from this, there should also be a concentration on the different testing scenarios in order to elude the chances of overlapping before even getting into Appium coding. An open discussion forum with the development teams to discuss the plotting of right accessibility IDs, labels or Unique IDs for the application’s elements would reduce many test automation reliability concerns.

 

4. Disregarding Application View States:

One of the challenge faced in Mobile Test Automation is not setting up the application. Most of the developers do not set up the application in order to access specific views and user states rapidly. To quote an example given by Jonathan Lipps,one of the key contributors to the Appium project.

A shopping cart functionality of the app might have ten different tests, and ninety percent of the tests might go through the same steps of logging in and searching items to be put in the cart which is a huge waste of time.

So, your team should be able to set up an app’s state and start the test in that apposite state, straightaway. This is highlighted in Appium scenario due to the fact that the mobile simulators and emulators are slow and take longer than usual to reach to the right position on the test.

 

5. Query for every element visibility:

Not querying for the visibility of every element is another way to speed up the run-time of the Appium test scripts. This leads to an additional load of calls and waiting time for Appium while in the every activity of retrieving an element. The lag can be reduced by only requesting the element attributes which are important in perspective of the test code.

 

6. Native Testing tools – Always better?

According to some developers, usage of the native testing tools is the best way to get unfailing mobile tests. For example- Espresso in case of Android and XCUITest in case of iOS. This is not good advice as neither of Google or Apple are going to release and new automation technology. When the question is of stability, the stability of code should be chosen rather than technology and in this scenario, Appium is the best choice!

As an exception, if the development team writes the tests and it’s most comfortable in mobile SDK language, using Google and Apple providers to build development environments. Also, when the constricted test-app or test code-app code integration is needed, then Appium might not be of much help. The utmost value is that Appium provides the WebDriver layer on top of the technology, which means the code can be written in any language and acts as a stable interface to that specific automation technology. Also, being a cross-platform tool, Appium allows to saves a lot of code and architecture in case of testing both iOS and Android devices.

 

7. Appium is slow:

Appium might be slower in some circumstances and also there are points in Appium codes where it does not seem efficient. Appium backs upon technologies that are slower than Appium and the curators of Appium have strategically chosen to utilize slower strategies for specific instances. For eg: Appium will certainly be slower if you are relying on XPath. The efficiency of the tool depends on how it is being used. Mostly, Appium is favored because of stability than the speed.

 

8. Not Using Appium Documentation:

The earlier Appium docs were not very user friendly. As a result, they weren’t used as much as they needed to be but the new Appium documents have been completely redesigned and reorganized. Information about the API reference to Appium, client libraries, supported drivers and Appium commands, code examples which was not provided before is all documented in the updated version. It deserves a revisit due to its newness and can be accessed at Appium.io.

9. Not creating reusable code

Repetitive or duplicate code can cause several issues for Appium mobile testing process. When the same code is repeated in several test scripts, any updates or modifications must be copied across all instances. This increases the chance of failures and complicates the debugging procedure. It also increases the maintenance cost for the organization.

Duplicate code makes identifying the primary reason for a test failure challenging. It can be difficult to pinpoint which occurrence of the code caused a test to fail if the same code is used repeatedly throughout several test scripts. This could increase the duration required for debugging the software and expand the market time.

Therefore, it’s crucial to use appropriate coding principles to prevent these issues. This includes creating simple reusable code, utilizing libraries and frameworks, and modifying code to remove redundancies.

10. Ignoring the Test-driven Deployment

Test-driven deployment is an Appium mobile test automation approach that stresses developing automated test cases before writing application code.

It lets the developer test all the application features before being deployed. This way, they can detect and resolve any bug in the early stage. This saves not only the organization’s time but also valuable resources.

Not just this, Test-driven deployment also helps developers in developing more reliable and enduring codes. By allowing them to understand the needs and provide a solution accordingly. Which lets them create more modular, maintainable, and error-free codes.

11. Choosing the right tool for API testing

APIs may have several endpoints, each with a unique set of parameters, methods, and authentication requirements. Writing and maintaining tests that accurately reflect the behavior of the API can be challenging due to this complexity.

Another difficulty is the requirement to simulate various scenarios and inputs. Testing for different response codes, error messages, and payloads might be part of this. To ensure the API can manage a high volume of queries, developers might also need to simulate load and stress testing.

Developers use a testing framework or library like RestAssured and Postman, which provides integrated support for API testing. These tools streamline and simplify the testing process by providing pre-build methods for common API testing scenarios, these tools streamline and simplifies the testing process. Alternatively, they can mimic various scenarios and inputs using mock data or a staging environment.

Developers can test their API in a safe setting before releasing it for production. Moreover, they can evaluate the API’s capacity by simulating heavy requests using load-testing tools like JMeter or LoadRunner.

12. Need to follow a good design pattern.

In Appium mobile test automation, the Page Object Model (POM) and the Page Automation Layer Model (PALM) are popular architectural patterns. Both techniques increase automated tests’ maintainability, scalability, and reusability.

POM is a design pattern that emphasizes the creation of reusable and modular code by isolating the application’s user interface from the test automation code, making it simpler to update tests when UI changes are made. POM builds an object repository that holds all the web elements and methods.
PALM is another design pattern that builds upon the principles of POM. But unlike POM, which focuses on UI elements, PALM emphasizes creating an abstraction layer between the test automation code and the application’s business logic. This approach distinguishes the test automation code from the underlying implementation. Thus making it easier to modify the test code without affecting the business logic.

Suyash Dubey | Posted on | 2 min Read

Some testing teams set up their own staging environment to test internal servers but as there is no public access which makes it difficult to perform local testing on remote devices. So how would you test your app which can be accessed through your internal network? We launched a feature called Wildnet, that enables you to test private or internal servers using pCloudy. You can test your privately hosted apps on any Android or iOS devices and implement continuous testing to ensure reduced release cycle.
 
Some times testers want to test their internal servers to protecting unsecured data and credentials, hide new features, and general security. But if an app hasn’t been made public, it should not delay testing across environments. When the Wildnet feature is enabled, a secure connection is created between your machine and pCloudy remote devices so you can perform testing even if the devices are connected remotely.

Let’s see how you can test local or private servers using Wildnet.

Prerequisite:
 
User should be registered on pCloudy platform
Executables can be downloaded from below links
 
Linux :
https://content.pcloudy.com/executables/utilities/WildNet_Executables/pCloudyWildNetLinux_v1
 
Windows :
https://content.pcloudy.com/executables/utilities/WildNet_Executables/pCloudyWildNetWindows_v1.exe
 
OSX :
https://content.pcloudy.com/executables/utilities/WildNet_Executables/pCloudyWildNetMac_v1
 
Manual Process to enable Wildnet
 
1. Run the below mentioned executable command on the terminal:
 
For Windows:
.\pCloudyWildNetWindows -i Emailadress –k accesskey -u URL
 
For Mac:
./pCloudyWildNetOSX -i Emailadress –k accesskey -u URL
 
For Linux
./pCloudyWildNetLinux -i Emailadress –k accesskey -u URL
 
Example:
./Win-Wildnet -i sample@test.com -k pass123 -u https://app.pcloudy.com
 
Note: Please use below mentioned URL for-
 
Public cloud(India): https://device.pcloudy.com
 
Public cloud(US) : https://us.pcloudy.com
 
Private cloud: https://private-cloud.pcloudy.com
 
Wildnet Example

Note:

  • This feature will work on all platforms.
  • Once “Wildnet” is enabled, the user can connect more than one device.
  • User will get authenticated as shown in the below screenshot
     
    Test Local Wildnet enabled

    This is how users can initiate Wildnet.
    2. Go to the Device page in pCloudy platform, connect the device and click on Enable Wildnet as shown in the screenshot:
     
    Device page

    Open any local URL that you wish to test and enjoy local testing on pCloudy platform.
     
    Local testing

    3. To disable the service simply run ctrl+c to exit
     
    Wildnet disable

    Activating Wildnet for Appium Automation
     
    1. Run the below mentioned executable command on the terminal:
    For Windows:
    .\pCloudyWildNetWindows -i Emailadress –k accesskey -u URL
     
    For Mac:
    ./pCloudyWildNetOSX -i Emailadress –k accesskey -u URL
     
    For Linux
    ./pCloudyWildNetLinux -i Emailadress –k accesskey -u URL
     
    Example:
    ./Win-Wildnet -i sample@test.com -k pass123 -u https://app.pcloudy.com
     
    Note: Please use below mentioned URL for
     
    Public cloud(India): https://device.pcloudy.com
     
    Public cloud(US): https://us.pcloudy.com
     
    Private cloud: https://private-cloud.pcloudy.com
     
    Wildnet Example

    Note:

  • This feature will work on all platforms.
  • Once “Wildnet” is enabled ,you can connect more than one device.
  • User will get authenticated as shown in the below screenshot
     

    Test Local Wildnet enabled

    2. Once the user is successfully authenticated, they need to add the capability mentioned below, in the Appium automation script.
     
    capabilities.setCapability( “pCloudy_WildNet”, true);
     
    3. Once the capability is added, users can perform local testing on pCloudy devices and verify the execution in live view page.
     
    The most important advantage of Wildnet is that you can be confident about the application as you can rectify all the defects prior to delivery. Try it yourself. Use Wildnet to find more bugs and defects in your internal server before deploying it to production.

    Suyash Dubey | Posted on | 2 min Read

    pCloudy is committed to delivering the best solutions in mobile app testing and therefore we come up with product updates at regular intervals. This time we are thrilled to announce the release of pCloudy 5.6 with exciting new features to make app testing simpler than ever. Our customers trust us because we believe in exceeding their expectations and add new useful features to enhance the platform’s capabilities. For those who are new to pCloudy, it is a cloud-based continuous testing platform designed to enrich your apps. In this new version of pCloudy, we have addressed the concerns of our users and added new features like Wildnet, Capability configurator, Object spy, and more. Let’s know more about all these features.

    Wildnet

    An application server is accessible only from within your lab and if you want to test your mobile application with that server, you are forced to work within that lab. If you move with the device in hand outside the lab, you will not have access to that server. How do you test such kind of setup using pCloudy? Because if you are using pCloudy, that means the devices are outside your network. The answer is Wildnet.
    wildnet ss
    This feature enables our users to test private or internal servers on any Android or iOS devices present on pCloudy. For this, you just need to download the executables for OS (Mac, Linux, Windows) on your local machine, Then fetch the user name and API access key for your pCloudy account settings and use it in the terminal to build a link between a device in pCloudy and your internal staging server. To connect a booked device to your internal staging server, you need to click on the Wildnet icon as shown in the picture below.

    Object Spy

    While creating automation test cases you need the object names of the objects that are available in your application. For example, if your app is launched on a device and you want to click on a particular button, you have to mention the object name in the script to perform that action in the app. Earlier testers used to extract the object name using Appium desktop, but now they don’t need Appium desktop for that. pCloudy now has an inbuilt object spy through which you can run the inspector sessions directly on Android and iOS devices. One big advantage is that its bilateral view which makes it easy for users to click on any element on one side and view the details another side. Also, the generated code can be copied for future use.
     
    object spy ss
    In the picture above you can see on the right side we have the object inspector where you will get all the attributes and values for any application. You can select any attribute and perform an action using the tap, send keys, clear and the more option. Following the action, an XPath will be generated which can be used in the automation script.

    Data-Persistence

    This feature will enable System-admins of our Private and On-Premises cloud to delete older reports based on pre-defined settings. To use this feature, system administrators will have to go to control settings in their pCloudy account and under Infrastructure Management there is an option for Data-Persistence. Whenever any user tests an app on any device in pCloudy, certain artifacts are generated in the form of test reports. Each report will have videos, logs, screenshots, and other elements. These artifacts take up a lot of space so now system admin can delete the older data using the data persistence feature.
    data per
    As shown in the pic, the system admin can select the date, time and the number of days for which the data has to be deleted. This will help in managing the limited space provided for every team account.

    Capability Configurator

    In an automation script, we have to include certain capabilities like device name, platform version, etc., to execute the test cases. But now you won’t have to gather the capabilities from different sources, capability configurator will generate the desired capabilities directly using certain filters.
    capability
    As you can see in the pic, you just need to go to the Devices page in pCloudy and click on the Capabilities tab. Under the capabilities tab, you can choose the parameters like OS, Automation type, Device location, Device name, etc., and the capabilities generated can be copied and pasted directly into the Appium script.

    More updates

    iOS is a close operating system, and there are many restrictions for testing apps on Apple devices. In pCloudy 5.6, you will have two more functions on iOS devices; close app and lock/unlock the device. The close app function will enable pCloudy users to close any launched application on a connected iOS device using the soft button. The second function will allow any connected iOS device on pCloudy to be locked and unlocked. These two functions will enhance user interaction on iOS devices by saving time and effort. One more new update is that pCloudy now supports Appium 1.15.1 which is one of the latest stable versions of Appium. This will make it easier to perform test automation on iOS 13 devices in pCloudy.

    To sum up

    The new features added to the platform will help our users to perform live testing on remote devices. Features like the latest Appium support and better user interaction on the iOS devices will empower our users to speed up automation. We are working on some new features, based on our customer feedback, which will be included in the next release. So stay connected for more updates.

    Suyash Dubey | Posted on | 2 min Read

    Test automation is gradually taking over manual testing as it saves time and improves the quality of testing. Especially in the case of regression testing, automation can improve efficiency by manifold. It is a waste of time and resources to do repetitive testing manually. Also, there is a scope for an error as repetitive testing can be exhausting but the chances of errors occurring in automation are slim. But what is automation testing? In simple terms, automation testing is replacing the manual human activity with a system that performs test cases by repeating predefined actions. To get the most out of automation it is necessary to select the right automation testing tool.
     

    Types of Automation Testing Tools

     
    Record & Replay: Tools in this category provide a recording option for the automated script. Every interaction on the screen like tapping, scrolling or typing will be recorded and converted into automation steps. The recorded scripts can be replayed to perform the actions and to verify the AUT.
     
    Coordinate based recognition: Tools in this category interact with the application under test with the help of x/y coordinates, to automate and verify the application.
     
    Native object recognition: Tools that use the native object recognition detect the UI or control elements on their given element tree. This tree is built by XPATH, XML or CSS to identify the elements, verify and automate the script.
     
    Text recognition: Text recognition: Text recognition or (OCR) Optical Character Recognition tools identify elements based on their text. These tools use visible texts to drive automation and to verify the app.
     
    Image recognition: These tools take screenshots of the UI elements in the product to add them to the automation scripts. The AUT will be automated with the help of these screenshots.
     
    A lot of test automation tools support more than one recognition approach which can be useful to get more robust automation scripts. Now let us look at the factors to consider while selecting the automation testing tools.

    Platform Support

    Are you looking for automation testing tools for web apps or mobile apps? If mobile app then is it hybrid, native or PWA? The scripting language is also an important factor in many cases as mobile automation testing tools do not support all the languages. So if you developed your app using JavaScript then it’s not recommended to use UFT for test automation. In that case, go for a tool like Selenium that is flexible with the scripting language.
     
    There are a lot of test automation tools that support freeware frameworks like Selenium and Appium. Framework integration helps in reducing the effort put into the testing and increasing the collaboration between the teams. Tool integration forms an ecosystem of a collaborative effort that helps in object identification, bug logging, test case management, report sharing, and shared repository.

    Ease of Adoption

    Ease of adoption can be measured on the basis of the skills required to use the tool and the learning curve. If there are resources available easily on the internet to learn the functioning of the tool then it is a plus point. Also, the support should be good and it’s an advantage if there is an active community that will give a rough idea about the complexities of the tool. We have observed that the installation process is also lengthy for some tools and frameworks. But overall the team should be able to use the tool easily with the skill set or the experience they have.

    Ease of Scripting

    Most of the automation testing tools provide flexible scripting options. It allows testing teams to write test scripts in their preferred languages. A good test automation tool helps in getting better reusability of test components and provides flexibility in scripts that can be reused across projects. Scripting time with object identification, speed of script execution, support for CI/CD tools like Jenkins, usually these attributes are considered before selecting the tool.

    Integrations

    If the testing team is using a test case or a bug management tool then the team would prefer an automation tool that can be easily integrated into those tools. It helps in managing the app development lifecycle properly.
     
    The keyword-driven or data-driven automation framework should be easily integrated into any data source. It is convenient if the tool provides connectivity with different data sources easily.

    Test Analytics

    Analytical reports form the backbone of test automation as it provides insights into the success of the process. Automation tools must provide customizable dashboards to view different types of reports shared across various teams.
     
    Real-time reports traceability and sharing ensure feature validation and efficient test coverage. Quality metrics such as injection rate, defect density, and test execution status allow you to maintain quality at every stage of continuous testing. It’s convenient to analyze results to identify bugs, patterns, and do a root cause analysis with a good automation tool.

    License Type and Cost

    It’s better to go for open-source tools but not every open-source automation tool has all the attributes that you might be looking for. Even if you opt for the proprietary licensed tools, you need to figure out how much you need to spend on the maintenance.
     
    Some companies offer live support and training for premium plans. So you might want to upgrade to avail of these services. There is also something called runtime license if you just want to execute the automation on different machines. Apart from that, there are options for a node-locked license for a single machine and a concurrent floating license to use the tool on multiple systems.
     
    giphy-min

    To Sum Up

    If you are still confused and want to further simplify the tool selecting process, then use a tool comparison matrix. Jot down your preferred 2,3 or even 5 tools and then rate them on the basis of all the factors that we have discussed earlier in this blog. This way it will be easier to make a decision on the basis of your requirements.
     
    As far as mobile test automation tools are concerned, device coverage holds the key. It is recommended to use a cloud-based testing platform to run your automated test on multiple devices. Automation is the soul of any CI/CD pipeline and for the successful implementation of DevOps, it is crucial to choose the right automation testing tools.
     

    Varun Kumar | Posted on | 2 min Read

    Espresso is a testing framework for Android which automatically synchronizes your test actions with the UI of your app. It also lets the test wait until all the background activities have finished.
     
    The pCloudy utility uploads your Application APK, test APK and other APK files to pCloudy. It will then acquire a device to run your test on, execute those tests and report back the status and test report.
     
    In pCloudy you don’t have to set up the environment. Just follow these simple steps.
     
    After the test cases are run you can see the detailed reports. Once automation ends, the device under test will be released automatically so that you or other users can use it later.
     
    There are two ways you can execute espresso test cases. Here are the steps for both
     
    1. pCloudy UI Dashboard
    a. In pCloudy UI there is a section called Automation where you will find multiple automation frameworks integrated to the platform.
    b.Select the Espresso automation and to fill in the required details.

     
    Espresso-on-pCloudy
    c. pCloudy Support two TestRunners.
    i. AndroidJUnitRunner (android.support.test.runner.AndroidJUnitRunner)
    ii. AndroidXJUnitRunner (androidx.test.runner.AndroidJUnitRunner)
     
    Espresso-on-pCloudy
    d. Select your Application and Test Application.
     
    Espresso-on-pCloudy
    e. Select the required devices.
     
    Espresso-on-pCloudy
    f. Click on Next button and Schedule the automation by clicking on the Schedule button.
     
    Espresso-on-pCloudy
    g. You will get a dialog box. Click on Ok.
     
    Espresso-on-pCloudy
    Espresso-on-pCloudy
    h. You will get the active session details on the Dashboard and you will also be able to see the Live view.
     
    Espresso-on-pCloudy
    i. Liveview will help you to see the live execution on your test cases. Click on the Liveview icon
     
    Espresso-on-pCloudy
    2.pCloudy Utility
     
    a. Download the espresso starter pack from here and Unzip it
    b. You will find two files,
    i. pCloudy_Espresso.jar
    ii.Config.properties
    c. Place the config.properties file and pCloudy_Espresso.js file into the same directory.
    d. Update the config.properties file as guided in the comments in the file.
    e. In your workspace, run the command by typing the java -jar pCloudy_Espresso.jar
     
    Espresso-on-pCloudy
    f. This will upload the required files which you have mentioned inside the config.properties files.

    g. You will get the status of the Automation in pCloudy Dashboard ActiveSession.
     
    Espresso-on-pCloudy
     
    h. Even on the pCloudy dashboard, you can use the Liveview by which you can see the running test cases.
     
    Espresso-on-pCloudy
    i. Once the test execution is over, You will get a URL of the test execution report on the terminal.
     
    Espresso-on-pCloudy
    j. You can open the report from the dashboard Report section also. In the report, all the test details will be there as shown in the screenshots below.
     
    Espresso-on-pCloudy
     
    k. You will also get individual device reports. Inside the test case level details, you will find performance details, test case logs, device logs, and video files.
     
    Espresso-on-pCloudy
    Espresso-on-pCloudy

    Suyash Dubey | Posted on | 2 min Read

    In the previous blog in this series, we learned how to write the first Appium script for Android devices. Now in this blog, we will understand how to start the Appium server.

    The code structure is segregated into three parts, the first part of the code starts the Appium server, the second part stops the Appium server.

    The Code Structure

    The third part is the main method which is necessary to execute the class and from this main method, all the above method are called.

    Starting Appium from code requires a path of two files which are kept inside the Appium folder.

    • node.exe
    • js

     

    Starting Appium Server

    In the code highlighted in red, we have called process class which is a Java class and declared it static.

    Starting Appium Server

    We then created a start server variable and pass the path to node.exe and main.js, highlighted in yellow. In the code highlighted in green, we added both paths into the same variable with spaces and created a method called startAppiumServer() which takes care of the Appium server startup process. Next step is to create an object of Runtime class which is again a java class and call the method getRuntime(), highlighted in blue. We will pass the variable “STARTSERVER” into runtime.exec() method. It will start the Appium Server.

    Once the process is started, we have to store the current state of the process into this variable. We will give a sleep time of 5 seconds as it takes time to start the process. what we are trying to validate here is, if the process is not null; it means that the process is started as highlighted in the gray box.

    Stop Appium Server

    Here we need to create another method named stopAppiumServer(). We again verify the state of the process and if it is not null, call a method of process class destroy().

    Stop Appium Server

    So now we know how to start and stop the Appium server from code. Let’s try to install and launch an app on a mobile device. Let’s see how we can install a .apk file on an Android device and launch it automatically without providing the path in Appium Server.

    Installing and launching the application

    To begin with, we call the stopAppiumServer() method to verify if any instance of Appium Server is already running and if so, then it closes that instance, as highlighted in red. The method highlighted in yellow starts the Appium Server.

    Installing and launching the application

    Then store the apk path file into a variable of File class, which is a Java class as highlighted in blue. Create another variable of File class app. It takes two parameters:

    • The absolute path of the apk file.
    • Name of the apk file.

    Now the absolute path is stored into appDir variable and the name of apk file in the second part as highlighted in the green box. The rest of the things are the same as discussed in the earlier sessions.

    The complete code is shown here.

    Package QA campus

    Appium server installation

    Appium server installation

    Now that you have learned how to launch app from code, let’s move on to the next blog where we will learn how to set up Appium on a Mac machine.

    Suyash Dubey | Posted on | 2 min Read

    In the last blog in this series, we learned how to set up Appium for test automation on a Windows machine. Now we will handle the application by writing the Java code. We will require the following software:

    Eclipse

    Selenium standalone Jar

    Appium Java Client
     

    This process consists of five steps as depicted in the picture below.

    Test Automation

    First, we need to collect the Selenium Standalone JAR and Appium Java Client Libraries. To download the Selenium standalone JAR file, go to seleniumhq.org/download then click on the Download version.

    Test Automation SeleniumHQ

    For Appium Java client you need to go to appium.io/downloads and download the libraries for the selected language.

    Test Automation Downloads

    Now to create a java project download Eclipse from eclipse.org/downloads. Launch Eclipse and select the workspace location.

    Test Automation Eclipse

    To set the Android SDK path into Eclipse, click on the Windows tab in the menu bar and select Preferences in the drop-down list.

    Test Automation

    Then select the Android option and browse your Android SDK location and click on Apply.

    Test Automation Preference

    Launch Eclipse and right-click on Package Explorer. Then select a new Java Project.

    Test Automation Java ADT

    Enter a project name and click finish.

    Once the project is created, add a folder to the project, for the project dependency files [Selenium Standalone] and [Appium Client Library] which you have downloaded.

    Test Automation Package Explorer

    Copy the downloaded file into the newly created project dependencies folder. Select both files and right-click. Then select the add to Build path option and then Add to Build Path. Thus both classes have been added to your project reference libraries.

    Test Automation Package Exp

    Create a class and import the required packages. Now right click on the src folder, hover the mouse over the new option and select the Class option.

    Test Automation Class

    Provide a package name, the name of class then select the main method checkbox.

    Test Automation Java Class

    A .java file within the package is created in Package Explorer panel. An auto-generated the main method is generated on the right panel.

    Test Automation Lauchapp

    You need to define a public class variable of AndroidDriver as your first line of code. You will now see an error for an android driver. Now hover your mouse over the error, you will get a list of quick fixes. Select Import ‘AndroidDriver’. Once you select the import package option, the android driver package will be imported and the error will be removed.Test Automation Android Driver

    Create an object of DesiredCapabilities. Again you will get an error for DesiredCapabilities. Hover the mouse over DesiredCapabilities and then select Import ‘DesiredCapabilities’ from the quick fix list.

    Test Automation Emulator

    Once you select the import package option, the DesiredCapabilities package will be imported and the error will be removed.

    Now set the DesiredCapabilities and also provide the package of an application and the name of application launcher activity.

    Now you need to instantiate the Android driver.

    To do this we need to have two parameters, the first is the Appium server address with the port number which it is running and the Capabilities.

    Test Automation capabilities

    Hover mouse on URL and import URL from Java.net. You will get an error for complete new URL section, hover mouse on the error and select Add throws declaration. By doing so exception has been added into your main method.

    Test Automation Driver

    Now start the Appium server and connect a device to the system. Now return to Eclipse and execute the code. This will launch the app in the device.

    In the code, we have declared the class as public so that we can access it anywhere inside out test.

    Test Automation Diving Deep

    The java main method is highlighted in yellow. We created an object of desired capabilities class which you can see in the green box. In method setCapability, there are two parameters. First is the capability name and second is the capability value as highlighted in blue. Then we created an object of AndroidDriver class highlighted in the black box. The code highlighted in gray is the findElement(By) method which can locate an element on the screen.

    Now that you have learned how to write a mobile test automation script using Appium test automation framework, we can move on to the next blog where we will learn how to start the Appium server and launch the app from code.

     
    Related Articles:

  • How to use Appium Inspector for Test Automation
  • Basics of Appium Mobile Testing
  • Inspecting Element on pCloudy Devices Using Appium Desktop
  • Help Guide for pCloudy-java-Connector for Appium
  • Appium vs Espresso: The Most Popular Automation Testing Framework in 2019