Category:

8 Tips to Optimize Your Continuous Testing Strategy

July 13th, 2020 by

In this competitive digital era, there has always been a constant demand for continuous digital improvement. Even if automation is a prominent keyword nowadays, a recent report says that only 30% of the organizations have adopted automation testing. Although the organizations invest a considerable amount of time and money in transforming their development processes, they fail to reap the benefits of the shift in the right direction by choosing only a few aspects of testing.
 
Change is inevitable in the software development lifecycle and in order to go hand in hand with the ever-changing dynamics and produce high-quality software, it is important to rely on implementing an effective Continuous testing Strategy. The development teams have to be focused on optimizing the delivery cycles and provide uninterrupted business experience.

What is continuous testing?

 
Continuous testing is a type of testing in which involves evaluating the quality of software often at multiple stages of the continuous delivery process for early detection of errors ultimately providing quality software and business continuity. It involves testing in multiple developments and QA phases to achieve continuous quality and improvement. Continuous testing plays a key role in DevOps using automation tools that help developers in getting quick feedback on the defects and make changes accordingly.
 
Continuous Testing Strategy

Challenges to an effective continuous testing

 
Organizations want to frequent deployments but the majority are unable to even reduce their software development cycles because of a lack of effective continuous testing delivery strategy. The success comes when there happens to be a mix of both manual and automated testing consistently throughout the delivery process. It becomes really important to automate every step to move fast in aspects like test design, test orchestration, execution, etc.
 
Test automation strategy is the backbone of the final product that a business delivers. It should be strong and flexible enough to bear the impact of any change and every variable of the testing process. Let’s discuss how optimizing continuous delivery testing strategy helps improving the testing process and to increase the software quality:

1. Thorough Test Automation Plan

 
Research says that the majority of the organizations mostly focus on specific forms of testing. For example, few businesses focus on API integration testing and UI testing and a few focus on both individually. But what they lack is a detail oriented automated continuous integration testing strategy that involves all of the above types of testing. Choosing a specific type of testing is right in some business scenarios but a business should always be equipped with a plan that can accommodate any type of testing which can be used whenever the need arises. The real comparison of continuous testing vs test automation will always persist considering the scale and requirements of the business.

2. Initial Trial and Testing

 
Introducing unreliable and lengthy test cases in the process leads to a huge wastage of time and effort putting the QA teams to get indulged in an ineffectual analysis of each code commit. Instead, the QA teams can do a bit of testing by adding nominal test suites in order to avoid big risks and check its impact along the way. Gradually adding stable test cases. It is important to have more stable test cases than counting how many were executed especially in DevOps.

3. Choosing the best test automation tool

 
Building a comprehensive continuous testing strategy is a mandate but choosing the right test automation tool is of critical importance for achieving complete stability. It is insignificant to even think of a failure caused merely because the test automation tool couldn’t perform on the assigned UI control on the application being tested. The acceptable reason for a failed test could be either bug or design change but not because of an automation tool. A good automation tool would always add value to the organization by activating continuous release and deployment, by reducing time and maintenance costs, by increasing the reusability of the code and by providing a higher return on investment.

4. Modularizing tests

 
In order to handle complex integrated applications, introducing modular testing is the best way to breakdown application functionality. Breaking down the app functionality into modules provides a road map and makes it easier to rearrange the functionality into testing representing different workflows. Amorphous tests can be easily redressed by introducing modular testing and decoupling test data, test flow and element locator mappings.

5. Automated Screenshots

 
When analyzing the test results to find out the causes of test failure, screenshots of the test execution is the best reference in order to dig deeper into the cause of the problem. A reliable automation tool automatically generates screenshots of all important interactions happening during the test run. Screenshot recording is not supported by all the automation tools; In that case, one should reconsider selecting another test automation solution to reap the maximum benefit of this feature.

6. Measuring Test Automation ROI

 
Major trends like Continuous testing, automation and DevOps and demand for faster continuous delivery have raised the standards for quality software so high that everyone is trying hard to stay competitive by optimizing their continuous testing and development strategy, that too while maintaining low costs. In this situation, balancing the test execution costs cannot be ignored. At some point in time, test coverage needs to be scaled up, teams have to opt for cloud usage which is expensive and acts as a major cost burden. The best idea would be to separate high and low level test requirements i.e performing more advanced integration and compatibility tests on cloud and executing the low level tests on-premise to maintain cost balance and getting the best automation ROI.

7. Track and Monitor

 
When the development and operations teams work together, they face many operational challenges. Developers are focused on pushing a new update and testers- on maintaining the reliability of the service delivery. Connecting these two points is challenging and these challenges are overcome by introducing continuous testing and DevOps. In the long run, a business should always invest in a system monitoring tool that will help to detect the causes of the gap and back outs more efficiently.

8. Outsource

 
Outsourcing the QA process extends an additional support system, giving an extra edge in delivering quality service. The veto in the world of continuous testing is that it gives and extra nudge by providing an exposure to the more specialized areas of testing like impact analysis, performance and security testing etc. It acts as a source of energy given to a focused area of the development life cycle. As a result, it becomes a win-win situation for both developers and testers.
 
Continuous Testing and Delivery is not a luxury but a necessity in today’s era. A well designed continuous testing strategy is the foundation of continuous testing and quality service delivery. Although the journey is full of challenges if planned meticulously, the businesses would be able to streamline and reap the maximum benefits of test automation and continuous testing.
 

Want to test your Mobile App?

Join pCloudy Platform!

Test Local And Internal Servers Before Deployment Using Wildnet

April 28th, 2020 by

Some testing teams set up their own staging environment to test internal servers but as there is no public access which makes it difficult to perform local testing on remote devices. So how would you test your app which can be accessed through your internal network? We launched a feature called Wildnet, that enables you to test private or internal servers using pCloudy. You can test your privately hosted apps on any Android or iOS devices and implement continuous testing to ensure reduced release cycle.
 
Some times testers want to test their internal servers to protecting unsecured data and credentials, hide new features, and general security. But if an app hasn’t been made public, it should not delay testing across environments. When the Wildnet feature is enabled, a secure connection is created between your machine and pCloudy remote devices so you can perform testing even if the devices are connected remotely.

Let’s see how you can test local or private servers using Wildnet.

Prerequisite:
 
User should be registered on pCloudy platform
Executables can be downloaded from below links
 
Linux :
https://content.pcloudy.com/executables/utilities/WildNet_Executables/pCloudyWildNetLinux_v1
 
Windows :
https://content.pcloudy.com/executables/utilities/WildNet_Executables/pCloudyWildNetWindows_v1.exe
 
OSX :
https://content.pcloudy.com/executables/utilities/WildNet_Executables/pCloudyWildNetMac_v1
 
Manual Process to enable Wildnet
 
1. Run the below mentioned executable command on the terminal:
 
For Windows:
.\pCloudyWildNetWindows -i Emailadress –k accesskey -u URL
 
For Mac:
./pCloudyWildNetOSX -i Emailadress –k accesskey -u URL
 
For Linux
./pCloudyWildNetLinux -i Emailadress –k accesskey -u URL
 
Example:
./Win-Wildnet -i sample@test.com -k pass123 -u https://app.pcloudy.com
 
Note: Please use below mentioned URL for-
 
Public cloud(India): https://device.pcloudy.com
 
Public cloud(US) : https://us.pcloudy.com
 
Private cloud: https://private-cloud.pcloudy.com
 
Wildnet Example

Note:

  • This feature will work on all platforms.
  • Once “Wildnet” is enabled, the user can connect more than one device.
  • User will get authenticated as shown in the below screenshot
     
    Test Local Wildnet enabled

    This is how users can initiate Wildnet.
    2. Go to the Device page in pCloudy platform, connect the device and click on Enable Wildnet as shown in the screenshot:
     
    Device page

    Open any local URL that you wish to test and enjoy local testing on pCloudy platform.
     
    Local testing

    3. To disable the service simply run ctrl+c to exit
     
    Wildnet disable

    Activating Wildnet for Appium Automation
     
    1. Run the below mentioned executable command on the terminal:
    For Windows:
    .\pCloudyWildNetWindows -i Emailadress –k accesskey -u URL
     
    For Mac:
    ./pCloudyWildNetOSX -i Emailadress –k accesskey -u URL
     
    For Linux
    ./pCloudyWildNetLinux -i Emailadress –k accesskey -u URL
     
    Example:
    ./Win-Wildnet -i sample@test.com -k pass123 -u https://app.pcloudy.com
     
    Note: Please use below mentioned URL for
     
    Public cloud(India): https://device.pcloudy.com
     
    Public cloud(US): https://us.pcloudy.com
     
    Private cloud: https://private-cloud.pcloudy.com
     
    Wildnet Example

    Note:

  • This feature will work on all platforms.
  • Once “Wildnet” is enabled ,you can connect more than one device.
  • User will get authenticated as shown in the below screenshot
     

    Test Local Wildnet enabled

    2. Once the user is successfully authenticated, they need to add the capability mentioned below, in the Appium automation script.
     
    capabilities.setCapability( “pCloudy_WildNet”, true);
     
    3. Once the capability is added, users can perform local testing on pCloudy devices and verify the execution in live view page.
     
    The most important advantage of Wildnet is that you can be confident about the application as you can rectify all the defects prior to delivery. Try it yourself. Use Wildnet to find more bugs and defects in your internal server before deploying it to production.

    5 Ways To Create Better App Experience For Your Users With Remote Testing

    April 2nd, 2020 by

    As the world battles with turbulent, uncertain times, most of the workforce across the globe is working remotely. Organizations have acknowledged the importance of remote working as it helps in maintaining business continuity. But in some scenarios, it is difficult to maintain business continuity or distribute resources within the teams while the team is working remotely.
     
    For instance, if you have some physical device infrastructure to test your app on multiple mobile devices, how would you do it? How would you share the devices with other testers and developers in your team working from different locations? Most importantly, how will you make sure that the app works smoothly on all the popular devices? We will address these issues in this blog, so buckle up for some interesting insights into the remote testing advantages that can ensure a better app experience for your users.

    1. Abate device fragmentation and ensure better app compatibility with remote testing

     
    Device fragmentation is any testers Achilles heel as it limits their potential of extensive testing. Testing from a physical device lab at this global lockdown situation is not feasible, and testing on a few devices won’t yield good results. But this issue can be rectified by testing on a device cloud. In pCloudy, users can test on multiple devices based on the popularity of devices in a particular region and its penetration to get the optimum device coverage.
     
    Both manual and automation testing can be performed with unlimited parallel test runs remotely on hundreds of real devices. This is also convenient for globally distributed teams, as the users won’t have to wait for the devices to be available for testing.

    2. Deliver Better Quality App with Rapid Automation

     
    Enterprises can ensure better quality apps without missing out on any deliveries by leveraging remote devices for automation testing. pCloudy helps in speeding up automation testing with codeless scripting and test orchestration using integrated tools like Jenkins. Capability configurator is a feature in pCloudy that generates the desired capability based on a set of filters, which saves time and effort while performing test automation. Integration with popular automation and collaboration tools like Appium, Espresso, Jira, etc., makes it convenient for users to perform automated testing on remote devices.
     
    Mobile device lab

    3. Better collaboration and continuous feedback

     
    In pCloudy, users can manage teams and distribute credits among themselves. The user management feature allows managers to become the system administrator and create teams to allocate the credits to the members according to the task assigned. This helps in user and task as the hierarchy is maintained to distribute workload systematically.
     
    Once the tests are complete, detailed test reports are generated automatically, which can be easily shared across the team. The progressive reports also show the tests failed, passed, and those with errors. This helps in focusing only on the tests that failed and doing a root cause analysis to rectify the issues. Continuous access to a range of devices available for remote testing will provide stability to your CI/CD pipeline.

    4. Assured data privacy and security

     
    Enterprise-grade security gives assurance to our users that their data is safe on the cloud platform. Our data centers comply with internationally recognized security standards like ISO27001, SOC2, and SSAE-16. Keeping your security issues in concern, we have another useful feature called Wildnet. This feature enables you to test your internal sites or apps on your local network, keeping all your data and information secure.

    5. Advanced features to improve manual testing

     
    Take advantage of next-gen features like Certifaya, an AI-powered autonomous testing bot to save time and effort. FollowMe is another feature that enables the user to run a test on multiple devices in parallel. This will save your resources while reducing the testing time by multifold. Apart from this, there are many features in pCloudy, like taking screenshots, recording the test video, cross-browser testing, etc. that will make manual app testing a piece of cake.

    In a Nutshell

     
    Remote testing is convenient, and it will help you save big bucks while you deliver a better quality app in less time. Continuous access to numerous devices helps in accelerating automation testing, as the app can be tested on multiple devices in parallel. All these advantages of remote testing make it the optimum choice for enterprises.

    Ensure Continuous Productivity By Leveraging Remote Devices For Mobile App Testing

    March 17th, 2020 by

    Remote working is going to be a new normal and remote access tools and platforms will play a key role in maintaining productivity. Working from home has many advantages both for enterprises and teams. There are many tools that help teams to collaborate and share all the information conveniently over the net. pCloudy is one such tool that gives you remote access to hundreds of real devices to test your app.
     
    So let’s look at the 10 ways how pCloudy could help in maintaining business continuity while working remotely.
     
    1. Remote access to devices: Test your apps from home by accessing remote devices on pCloudy. You don’t have to worry about touching or managing the devices.
     
    Remote access to devices
     
    2. Device access to geographically distributed teams: Since testing professionals are working remotely, the problem is sharing the limited number of devices for app testing among the team members. With pCloudy offering a range of devices over cloud, members across different locations can access hundreds of real devices on the cloud for their app testing needs.
     
    Device access to geographically distributed teams
    3. All in one: Manual and automation testing can be performed remotely on real devices. The platform is well suited for continuous testing and optimizes your DevOps process.
     
    4. Autonomous testing bot: use our AI power autonomous testing bot to minimize the effort and time spent on testing.
     
    Autonomous testing bot
    5. Team management: Manage platform access to the team members and share detailed app test reports.
     
    Team management
    6. Parallel testing: Run automation tests in parallel on multiple devices to increase the testing efficiency.
     


    7. Latest devices: No need to wait for the delayed shipments of the new devices as all the latest devices are available on pCloudy.
     
    8. Fully integrated: pCloudy is integrated with most of the popular tools and frameworks used for task management, version control, test automation, etc.
     
    Fully integrated
    9. Device tunnel: Developers can use features like Device tunnel to access devices in their Dev environments like Android Studio or Xcode
     
    Device tunnel
    10. Quality support: Achieve quality@speed in mobile app testing while working from home. Our technical support team will be available to help you 24/7.
     
    11. Device management: Hassle-free app testing for our users and our cloud infra team will take care of the continuous availability and maintenance of the devices.
     
    It has been speculated that around 30% of the service sector workforce across the globe will be working from home by 2030. You can stay ahead of others in this initiative and adopt flexible work preferences. Contact us if you need more information on how pCloudy could help in enhancing productivity while working from home.
     

    5 Best Practices To Avoid Test Automation Failure

    February 24th, 2020 by

    Automation testing forms the core of any CI/CD pipeline and enterprises are keen to practice test automation to enhance the efficiency of the development process. Test automation saves resources and reduces the cost of any project in the long run. But there are some important points to keep in mind while testing to avoid automation failure. Let’s have a look at these salient points.

    Leverage Parallel Execution

    Once you are done automating the test cases, the challenge will be the complex test suites taking a long time to get executed. It affects the quality of the test queue in the test automation framework or IDE. This leads to queue timeout issues and test cases being halted abruptly due to the sequential execution of the test cases.
     
    Parallel execution in different test environments is preferred over sequential execution as it saves a lot of time. Although in automated testing, unintended code interactions could happen. This is why you need a thorough reporting mechanism to debug the causes of test failure.
     
    Leverage Parallel Execution

    Pick The Right Tools

    Choosing the right tool for test automation is critical to the success of automation testing. There has to be a set of clear requirements/parameters on the basis of which the tools have to be selected. Some important points that are to be kept in mind while selecting the tool are:
     

  • The team should be clear about the test tool requirements.
  • The testing requirements of the application under test (AUT) should be analyzed thoroughly.
  • The team’s skill set should be accessed accurately.
  • The cost-benefit analysis should be performed to calculate the return on investment.
  • Tool vendor and capability should be evaluated as technical support might be required while using the tools.
  •  
    One tool might not be enough to meet any organization’s automation needs. Also, test automation engineers have to be a part of the tool evaluation process so that they can help in selecting the right set of tools. For example, you can use Appium for test automation but you need pCloudy to perform automation testing on multiple real devices in parallel.

    Analyze The Test Reports

    Test reports provide insights into the underlying issues that are to be resolved. A detailed test report gives an idea of the efficiency of the test automation and the automation team can analyze the report to look for the scope of improvement. While selecting an automation tool you need to make sure that the tool generates test reports to be analyzed by the test automation engineer. There will always be some tests that will fail to execute and it is necessary to analyze the test report to get an understanding of the scenario.

    Test Automation Metrics

    Test automation metrics will help you gauge the quality of the tests performed based on some essential parameters like test duration, unit test coverage, path coverage, number of defects found, percentage of broken builds, etc. The test metrics will give you a clear picture of how well the code is tested. In an agile process, there are frequent iterations to the builds and it becomes important to track the quality of each build. With test automation metrics you can figure out what is the percentage of your tests that passed and what was the reason behind the failed tests.

    Optimum Device Coverage

    Test automation is effective when the tests are executed on multiple devices in parallel. Device coverage is the most prevalent challenge as we have witnessed growing device fragmentation across the world. To ensure the smooth functioning of mobile apps on all the devices, you need to perform automation testing on hundreds of device-OS combinations.
     
    Test automation should be designed to make the app compatible with most of the popular devices. The frequent release of new versions of OS from both Android and iOS is a major factor that drives device fragmentation. The only way to overcome this challenge is by testing the app on a cloud-based testing platform. In pCloudy, you will get the benefit of testing the app on more than 5000 device browser combinations in parallel ensuring optimum device coverage.

    Summing It Up

    Test automation has many benefits like better test coverage, faster feedback, and accelerated results which reduces the time to market of any application. Using the practices mentioned above you can ensure coherent test automation and increased productivity. Apart from these points, there are many other things you can do like writing original code and not copying it as the code taken from other sources might not work in your test environment. But you will always find new roadblocks which you will have to deal with spontaneously.

    The Scope Of Automation Testing In The Intelligent Digital Mesh

    February 17th, 2020 by

    Intelligent Digital Mesh is the entwining of people, devices, content, and services enabled by digital models, business platforms and a rich, intelligent set of services to support digital business. We have witnessed the implementation of AI in every technology to leverage the benefits of autonomous systems. Enterprises are now focusing on using AI with technologies like blockchain and immersive technology which will create new categories of apps. In this type of environment, attaining optimum device coverage will be essential to ensure quality services. Now let’s understand the fundamentals of the intelligent digital mesh.

    Intelligent

    In the near future, most of the mobile applications and services will use artificial intelligence or machine learning at some level. AI will be the inconspicuous force of most of the popular app categories while creating some new ones. Intelligent apps also create a new intelligent layer between people and systems as seen in enterprise advisors and virtual user assistants. Augmented analytics is also gaining ground and helping enterprises in enhancing business intelligence and data analytics using ML and NLP. Another use of AI and ML is in intelligent things like smart vacuums, drones, autonomous farming vehicles. Intelligent devices are getting smarter to serve better and reduce human dependency to a minimum.
     
    Top 10 Strategic Technology Trends for 2019

    Source: Gartner.com

    Digital

    When we talk about digital, we mean digital twins, cloud to the edge, conversational platforms, and Immersive Experience. A digital twin is a digital representation of real-world objects. It offers information on the state of the counterparts, improves operations and adds value to the operations by responding to the changes. In the near future, all the aspects of human life and the real world will be interconnected with their digital representation capable of advance simulation, analysis, and operation. This combined with immersive technologies like AR, VR, and MR will take extended reality to a new level.

    Mesh

    Mesh is the connection between devices, people, businesses, services, and content to build a digital ecosystem that yields high-quality results. Here mesh refers to technologies like Blockchain, Event-driven, and continuous adaptive risk and trust (CARTA). Enterprises are keen to find new ways to sense the new business events to get the most out of it. A business event can be a change in the status of the deal like finalizing a deal. Using new technologies like AI, it will be easier to detect a business event and analyze it in greater detail.
     
    Security is one of the most important and ever-evolving processes in digital businesses. There is a need to think beyond infrastructure and parameter protection. Continuous adaptive risk and trust assessment is a people-centric security approach that allows for real-time risk and trust-based decision making. New methodologies like DevSecOps and adaptive honeypots should be implemented to strengthen the security of digital businesses.

    Automation Testing For Intelligent Apps

    Intelligent apps are at the core of the intelligent digital mesh. Nowadays most of the apps use artificial intelligence, machine learning or predictive analysis to make suggestions to the customers. The apps use real-time and historical data from user interactions and other sources to predict the needs of their users.
     
    To ensure the quality of apps it is important to test the apps using futuristic tools. Manual testing is just enough and even automation needs to be scalable to get better results. Testing the app on a cloud-based app testing platform is the best choice as you can use as many devices as you want to test your app. Also, parallel testing increases app testing efficiency by multifold.
     
    pCloudy’s AI-powered autonomous testing bot steals the show when it comes to testing intelligent apps. The bot tests the app on real devices with just a single click and generates a detailed report based on the test result.

    Conclusion

    Mobile devices, by and large, are the focal point of most of the innovations that are happening around the intelligent digital mesh. Whether it is Ai driven development, autonomous things or immersive experience, mobile apps still used as a foundation to provide the technology to the masses. But the growing complexities of intelligent apps makes it crucial to implement new methods of app testing. A cloud-based app testing platform like pCloudy is suitable to ensure quality at speed in mobile app testing. The freedom of accessing hundreds of real devices from anywhere at any time and perform manual or automation testing using futuristic features is the correct way to test intelligent apps.

    Important Takeaways From The Software Testing Conference 2019

    December 10th, 2019 by

    Asia’s leading software testing conference, QAI STC 2019 concluded on December 6th, 2019. The theme of this year’s STC was architecting continuous quality: think, transform, and thrive in an Intelligent future. The STC is a platform for experts to display ideas, experiments, and experiences to explore challenges and suggest techniques and innovations to overcome common problems.
     
    Avinash’s keynote focused on the future of testing which was lauded by the top minds and became the highlight of the event.
     

     
    pCloudy was the title sponsor for the event and so we got a bigger platform to showcase our contribution in shaping the future of mobile app testing.
     
    pCloudy -STC2019
    More than 500 experts from around 130 software companies gathered to learn new trends in testing, share their ideas and grow their network. There were 14 keynotes from the industry leaders and 50 professionals got a chance to take the stage and share their views about the emerging technologies.
     
    pCloudy -STC 2019
    Let’s have a look at the major learnings from this event.

    Future of Testing


    The STC started with a keynote by Avinash which provided insights about the future of work, applications, and testing. How “We Working” will be the primary organizational model and algorithmic management will take over the middle management to some extent. He also talked about how people will have to constantly upskill to work with the ever-changing technology and the work-life challenges will increase in the future. A digital mesh architecture will allow enterprises to build an agile, flexible, and cloud-ready ecosystem. This will enable real-time connectivity of employees, business processes, business data, and services to help address high volumes of traffic and become cloud-native and mobile-first.
     

    Quality Engineering and Digital Transformation

     
    There was a great emphasis on quality engineering at the STC as most of the organizations are trying to take QE to the next level. Quality engineering focuses on the end-to-end management and the basic principle is the all the teams should bear the responsibility of maintaining the quality in the process. Software QE is the assurance of high standards in the software development life cycle while implementing DevOps and Agile.
     
    pCloudy - STC 2019
    The main role is played by testers who create, implement and maintain systems used to control the quality of production processes. These people need to have a deep understanding of all the technological activities and evaluation principles.
     
    pCloudy - STC 2019
    Quality engineering methodology is even bigger in scale than the traditional QA approach and that’s the reason that QEs cannot work in silos. In quality system engineering, people in multiple roles like IT architects, designers, test engineers, project managers, business architects, etc., must cooperate to meet customer expectations. Quality engineering is driven by emerging technologies such as AI and Big Data analytics. Automation is the driving force behind turning the traditional testing into a more effective quality support model.
     
    pCloudy - STC 2019
    The quality engineering team usually partners along with the business users and the product managers for having a better understanding of the required product details to match up the problems since the starting of the product to the last stage.
     
    pCloudy - STC 2019

    Artificial intelligence, Machine learning, and IoT

     
    The recent development in AI, ML and, IoT were the buzz creators. Experts elaborated on how augmented analytics will be utilized for creating, developing, and consuming analytics by combining these technologies. An augmented analytics engine can identify, filter, and analyze data, and then recommend what needs to be done next without the need of an IT team. These technologies will make data-driven insights accessible to a much larger set of workers.
     
    pCloudy STC2019
    The conference turned out to be really productive with good insights about emerging technologies and tools. It was a great opportunity to connect with software testing experts and professionals from around the globe. To say the least, it was a remarkable event where we got a great response and positive feedback from the crowd.

    How to Choose The Right Automation Testing Tool

    November 15th, 2019 by

    Test automation is gradually taking over manual testing as it saves time and improves the quality of testing. Especially in the case of regression testing, automation can improve efficiency by manifold. It is a waste of time and resources to do repetitive testing manually. Also, there is a scope for an error as repetitive testing can be exhausting but the chances of errors occurring in automation are slim. But what is automation testing? In simple terms, automation testing is replacing the manual human activity with a system that performs test cases by repeating predefined actions. To get the most out of automation it is necessary to select the right automation testing tool.
     

    Types of Automation Testing Tools

     
    Record & Replay: Tools in this category provide a recording option for the automated script. Every interaction on the screen like tapping, scrolling or typing will be recorded and converted into automation steps. The recorded scripts can be replayed to perform the actions and to verify the AUT.
     
    Coordinate based recognition: Tools in this category interact with the application under test with the help of x/y coordinates, to automate and verify the application.
     
    Native object recognition: Tools that use the native object recognition detect the UI or control elements on their given element tree. This tree is built by XPATH, XML or CSS to identify the elements, verify and automate the script.
     
    Text recognition: Text recognition: Text recognition or (OCR) Optical Character Recognition tools identify elements based on their text. These tools use visible texts to drive automation and to verify the app.
     
    Image recognition: These tools take screenshots of the UI elements in the product to add them to the automation scripts. The AUT will be automated with the help of these screenshots.
     
    A lot of test automation tools support more than one recognition approach which can be useful to get more robust automation scripts. Now let us look at the factors to consider while selecting the automation testing tools.

    Platform Support

    Are you looking for automation testing tools for web apps or mobile apps? If mobile app then is it hybrid, native or PWA? The scripting language is also an important factor in many cases as mobile automation testing tools do not support all the languages. So if you developed your app using JavaScript then it’s not recommended to use UFT for test automation. In that case, go for a tool like Selenium that is flexible with the scripting language.
     
    There are a lot of test automation tools that support freeware frameworks like Selenium and Appium. Framework integration helps in reducing the effort put into the testing and increasing the collaboration between the teams. Tool integration forms an ecosystem of a collaborative effort that helps in object identification, bug logging, test case management, report sharing, and shared repository.

    Ease of Adoption

    Ease of adoption can be measured on the basis of the skills required to use the tool and the learning curve. If there are resources available easily on the internet to learn the functioning of the tool then it is a plus point. Also, the support should be good and it’s an advantage if there is an active community that will give a rough idea about the complexities of the tool. We have observed that the installation process is also lengthy for some tools and frameworks. But overall the team should be able to use the tool easily with the skill set or the experience they have.

    Ease of Scripting

    Most of the automation testing tools provide flexible scripting options. It allows testing teams to write test scripts in their preferred languages. A good test automation tool helps in getting better reusability of test components and provides flexibility in scripts that can be reused across projects. Scripting time with object identification, speed of script execution, support for CI/CD tools like Jenkins, usually these attributes are considered before selecting the tool.

    Integrations

    If the testing team is using a test case or a bug management tool then the team would prefer an automation tool that can be easily integrated into those tools. It helps in managing the app development lifecycle properly.
     
    The keyword-driven or data-driven automation framework should be easily integrated into any data source. It is convenient if the tool provides connectivity with different data sources easily.

    Test Analytics

    Analytical reports form the backbone of test automation as it provides insights into the success of the process. Automation tools must provide customizable dashboards to view different types of reports shared across various teams.
     
    Real-time reports traceability and sharing ensure feature validation and efficient test coverage. Quality metrics such as injection rate, defect density, and test execution status allow you to maintain quality at every stage of continuous testing. It’s convenient to analyze results to identify bugs, patterns, and do a root cause analysis with a good automation tool.

    License Type and Cost

    It’s better to go for open-source tools but not every open-source automation tool has all the attributes that you might be looking for. Even if you opt for the proprietary licensed tools, you need to figure out how much you need to spend on the maintenance.
     
    Some companies offer live support and training for premium plans. So you might want to upgrade to avail of these services. There is also something called runtime license if you just want to execute the automation on different machines. Apart from that, there are options for a node-locked license for a single machine and a concurrent floating license to use the tool on multiple systems.
     
    giphy-min

    To Sum Up

    If you are still confused and want to further simplify the tool selecting process, then use a tool comparison matrix. Jot down your preferred 2,3 or even 5 tools and then rate them on the basis of all the factors that we have discussed earlier in this blog. This way it will be easier to make a decision on the basis of your requirements.
     
    As far as mobile test automation tools are concerned, device coverage holds the key. It is recommended to use a cloud-based testing platform to run your automated test on multiple devices. Automation is the soul of any CI/CD pipeline and for the successful implementation of DevOps, it is crucial to choose the right automation testing tools.
     

    Flutter App Automation Using Appium Flutter Driver

    November 4th, 2019 by

    Flutter has been creating waves in the app development space since its launch. The reason is simple, it solves the problem of creating two different codebases for a single app to run on Android and iOS devices. Some experts say that Flutter which was launched by Google is to counter React Native which is used by Facebook. Nevertheless, this is a much-needed relief for enterprises as Flutter will help them save a lot of their time and resources. So let’s first understand what is Flutter.

    What is Flutter

    Flutter is an opensource mobile UI framework that can create Android and iOS apps from a single codebase. The Flutter framework includes the Flutter engine, Dart platform, the Foundation library, and design-specific widgets. Flutter apps are written in Dart programming language and runs on dart virtual machine.

    How to Create a Flutter App

    Before we start the process of app development it is important to know that there are two types of widgets, stateless and stateful. A stateless widget describes part of the UI by building a structure of other widgets that describe the UI more definitive. Stateful widgets are the widgets that can change their state multiple times and can be redrawn on to the screen any number of times while the app is in action.
     
    To create a Flutter app you need to install Flutter SDK and other tools required to support the framework. We will create the app in Intelli J IDE and for that, you need to install two plugins. The first one is Dart and the second plugin is Flutter. You can install the plugins by clicking on File in the menu bar of the IDE and then click on Settings.
     
    1

    Then you need to create a new project and you will get a basic code layout to start with your app development. In the first line, we have the main method where we will enable the flutter driver extension and there is a run app method with MyApp class instance, to run the app.
     
    Flutter stateless

    In this sample code, we have used the title as the stateless widget and we have set the title as the key and for the home screen, we are using one more instance which is MyStatefulWidget(). So in this class, we have a small piece of code, where we have set values for elements like appBar, background color, and in the body, we have placed a floating action button for a counter.
     
    flutter statefull

    After adding the code you can run the program and you will be able to see the changed state of the counter with every click on the action button.
     
    Flutter automation

    Appium Flutter Driver

    Appium Flutter driver is a test automation tool for flutter apps on multiple platforms. It is a part of the Appium mobile test automation tool. But how does the Appium Flutter driver work? It uses webSocket communication to carry WDIO script to instantiate the flutter driver and the request will be sent to the particular application. Then the AUT will send the response to the script through the Appium Flutter driver.

    Appium Flutter driver
    Appium Flutter driver

    You can also build Appium in your machine through the source code. Once you have cloned the source code, run npm install then npm run built and type a command node to start the Appium server. You can download the package from GitHub. The package contains Appium Flutter Driver and Appium Flutter Finder. You can clone the sample project to get the WDIO code.
     
    Flutter sample

    In the terminal, you can see that to automate flutter application we have used Flutter as an automation name for both Android and iOS apps. You can see that port is 4723 and in the capabilities, we have mentioned osSpecificOps, automation name: ‘Flutter’. We have mentioned the app path at the top with the device name and platform name and we created an object.
     
    Flutter Terminal

    Appium==OS environment is ‘android’ which will pick all the information and specify in the capabilities and it will pass a post request to the server.
     
    Now here we are trying to find the element using CounterTestFinder and a ButtonFinder to click on the button. In the next line, we have created a driver using a remote method.
     
    Then we have switched the Native App context to Flutter. You can switch this if you are using the Flutter app. Once the app is launched, we are validating that the counter text finder has value as zero by calling Flutter APIs internally. So here we are using two actions, one is clicking and the other is touch.
     
    Flutter code

    The button is to be clicked two times and thus we have used strictEqual function. In the last line of code, we are deleting the session.
     
    Now to run the test we need to pass the environment variable to connect the device.
     
    Flutter demo

    Then we will check if our Appium server is running.
     
    Flutter Appium

    Then once you run the code you will be able to see the actions performed on the device and the counter will change to two.
     
    Flutter Appium automation

    This is how you can automate your Android and iOS app testing using Appium Flutter driver. Although there are many benefits of using Flutter, there are some limitations as well. The first thing is that you need to learn Dart language to develop and automate the Flutter app.
     
    Suppose you need to run automated tests on multiple devices, then you will have to fire multiple commands in the terminal. Appium flutter driver depends on Flutter Driver APIs and as it’s still not matured enough, some actions cannot be performed.

    Audio Interaction Testing On Remote Devices

    October 10th, 2019 by

    In the past couple of years, there has been a tremendous increase in voice-enabled apps in the market. This is because business is looking to make its app more user-friendly and easy to use. If the same function can be performed using just your voice then why would you waste your time entering text? Voice recognition is being used by many apps from all sectors like food delivery, eCommerce, gaming, weather, navigation, social media, etc.

    The Rise of Voice-Enabled Apps and Intelligent Virtual Assistant

    Voice-enabled apps are quite common these days as enterprises are trying to make it easier for users to control the app. Google was the first widely used app that utilized voice reorganization to make it simple for users to look for anything on the search engine. But when it comes to Intelligent Virtual Assistant (IVA), Apple does lead the way with Siri.
     
    The idea of IVAs is not new and we have seen it in many Sci-Fi movies too. Take the example of Weebo from the movie Flubber. It was a flying robot assistant who recognized voice commands and helps perform daily tasks. It’s quite possible that we will see something like that soon in the market leveraging AI and drone technology.
     

    Graphical User Interface Vs Voice User Interface

    Apart from GUI, a new trend is shaping up and it’s called VUI. In the world of voice apps, the primary interaction between human and machine is done through the Voice User Interface (VUI). It is a natural form of interaction where the user can speak to the device and the device will reply or perform the task that it is asked to do. For instance, If you ask the Google assistant “How is the weather in Palo Alto?” the app will convert the analog command into binary and using AI it will reply with an audio output. Here the testers need to make sure that the app is generating the correct reply to the questions. Exploratory testing plays a key role in identifying the bugs in audio-based testing.
     
    Audio Testing
     
    When it comes to GUI, it is designed in a way to make it easier for the user to navigate through the business process. The graphical interface has controls such as checkboxes, dropdown lists, radio buttons, etc, through which all the features are controlled. Therefore the tester needs to test if all the control elements are working fine in all the pages.
     
    The fundamental difference between testing the GUI and the VUI is the input type. There is limited data input in GUI testing but in VUI testing, it can create an indefinite number of test cases. When it comes to voice apps, pairwise testing can help to get better test coverage with fewer test cases.

    The Easiest way to do Audio Interaction Testing

    The best way to test your app for audio functions is by using pCloudy. To perform audio testing, users need to upload their app on the platform and install it on any device of their choice. Once the device is booked and the app is installed on the device, click on the “More” option at the bottom left corner of the screen. Then you will get the audio in and audio out options in the device interaction page to do audio testing. Users can also use their voice as an input for testing using the “Audio In” feature.
     
    This feature can be used for audio out testing as well and it is available only for devices on which audio icon is present. To perform audio output testing, just book a device on the pCloudy platform, click on the “Audio Out” icon and play any audio/video file to get the sound output on your system.
     
    audio testing

    The audio testing feature is not available on any other cloud-based testing platform and this gives a big advantage to pCloudy. Testers will be able to use this feature to test their apps with voice commands. They will also be able to check if the app is responding with sound output for a particular function.

    Conclusion

    Looking at the popularity of voice apps, It is estimated that in the US, 111.8 million people will use a voice assistant at least once in a month in 2019. This makes up about 39.4% of internet users and 33.8% of the total population. Voice apps and assistants are gaining popularity due to its ease of use.
     
    A lot of organizations are working on using AI and ML to make customer support automated. This means that there will be no more humans answering the phones or replying to chats. A basic version is already in practice which is the IVR. In the near future, there will be smart voice support applications that will listen to the customers and resolve their issues without human interference. This will also reduce the waiting time to talk to a support executive from the organizations. Again a great example of efficiency and speed.
     
    For more information check out this video:

     
    Related Articles:

  • Test For Voice/Audio Commands on pCloudy Devices
  • How to Run Espresso Test on Remote Devices
  • Tips To Use Devices More Effectively
  • 7 Types Of Mobile App Testing
  • Mobile Devops+Agile – Challenges and Keys to Success