As the world battles with turbulent, uncertain times, most of the workforce across the globe is working remotely. Organizations have acknowledged the importance of remote working as it helps in maintaining business continuity. But in some scenarios, it is difficult to maintain business continuity or distribute resources within the teams while the team is working remotely.
For instance, if you have some physical device infrastructure to test your app on multiple mobile devices, how would you do it? How would you share the devices with other testers and developers in your team working from different locations? Most importantly, how will you make sure that the app works smoothly on all the popular devices? We will address these issues in this blog, so buckle up for some interesting insights into the remote testing advantages that can ensure a better app experience for your users.
1. Abate device fragmentation and ensure better app compatibility with remote testing
Device fragmentation is any testers Achilles heel as it limits their potential of extensive testing. Testing from a physical device lab at this global lockdown situation is not feasible, and testing on a few devices won’t yield good results. But this issue can be rectified by testing on a device cloud. In pCloudy, users can test on multiple devices based on the popularity of devices in a particular region and its penetration to get the optimum device coverage.
Both manual and automation testing can be performed with unlimited parallel test runs remotely on hundreds of real devices. This is also convenient for globally distributed teams, as the users won’t have to wait for the devices to be available for testing.
2. Deliver Better Quality App with Rapid Automation
Enterprises can ensure better quality apps without missing out on any deliveries by leveraging remote devices for automation testing. pCloudy helps in speeding up automation testing with codeless scripting and test orchestration using integrated tools like Jenkins. Capability configurator is a feature in pCloudy that generates the desired capability based on a set of filters, which saves time and effort while performing test automation. Integration with popular automation and collaboration tools like Appium, Espresso, Jira, etc., makes it convenient for users to perform automated testing on remote devices.
3. Better collaboration and continuous feedback
In pCloudy, users can manage teams and distribute credits among themselves. The user management feature allows managers to become the system administrator and create teams to allocate the credits to the members according to the task assigned. This helps in user and task as the hierarchy is maintained to distribute workload systematically.
Once the tests are complete, detailed test reports are generated automatically, which can be easily shared across the team. The progressive reports also show the tests failed, passed, and those with errors. This helps in focusing only on the tests that failed and doing a root cause analysis to rectify the issues. Continuous access to a range of devices available for remote testing will provide stability to your CI/CD pipeline.
4. Assured data privacy and security
Enterprise-grade security gives assurance to our users that their data is safe on the cloud platform. Our data centers comply with internationally recognized security standards like ISO27001, SOC2, and SSAE-16. Keeping your security issues in concern, we have another useful feature called Wildnet. This feature enables you to test your internal sites or apps on your local network, keeping all your data and information secure.
5. Advanced features to improve manual testing
Take advantage of next-gen features like Certifaya, an AI-powered autonomous testing bot to save time and effort. FollowMe is another feature that enables the user to run a test on multiple devices in parallel. This will save your resources while reducing the testing time by multifold. Apart from this, there are many features in pCloudy, like taking screenshots, recording the test video, cross-browser testing, etc. that will make manual app testing a piece of cake.
In a Nutshell
Remote testing is convenient, and it will help you save big bucks while you deliver a better quality app in less time. Continuous access to numerous devices helps in accelerating automation testing, as the app can be tested on multiple devices in parallel. All these advantages of remote testing make it the optimum choice for enterprises.
Automation testing forms the core of any CI/CD pipeline and enterprises are keen to practice test automation to enhance the efficiency of the development process. Test automation saves resources and reduces the cost of any project in the long run. But there are some important points to keep in mind while testing to avoid automation failure. Let’s have a look at these salient points.
Leverage Parallel Execution
Once you are done automating the test cases, the challenge will be the complex test suites taking a long time to get executed. It affects the quality of the test queue in the test automation framework or IDE. This leads to queue timeout issues and test cases being halted abruptly due to the sequential execution of the test cases.
Parallel execution in different test environments is preferred over sequential execution as it saves a lot of time. Although in automated testing, unintended code interactions could happen. This is why you need a thorough reporting mechanism to debug the causes of test failure.
Pick The Right Tools
Choosing the right tool for test automation is critical to the success of automation testing. There has to be a set of clear requirements/parameters on the basis of which the tools have to be selected. Some important points that are to be kept in mind while selecting the tool are:
The team should be clear about the test tool requirements.
The testing requirements of the application under test (AUT) should be analyzed thoroughly.
The team’s skill set should be accessed accurately.
The cost-benefit analysis should be performed to calculate the return on investment.
Tool vendor and capability should be evaluated as technical support might be required while using the tools.
One tool might not be enough to meet any organization’s automation needs. Also, test automation engineers have to be a part of the tool evaluation process so that they can help in selecting the right set of tools. For example, you can use Appium for test automation but you need pCloudy to perform automation testing on multiple real devices in parallel.
Analyze The Test Reports
Test reports provide insights into the underlying issues that are to be resolved. A detailed test report gives an idea of the efficiency of the test automation and the automation team can analyze the report to look for the scope of improvement. While selecting an automation tool you need to make sure that the tool generates test reports to be analyzed by the test automation engineer. There will always be some tests that will fail to execute and it is necessary to analyze the test report to get an understanding of the scenario.
Test Automation Metrics
Test automation metrics will help you gauge the quality of the tests performed based on some essential parameters like test duration, unit test coverage, path coverage, number of defects found, percentage of broken builds, etc. The test metrics will give you a clear picture of how well the code is tested. In an agile process, there are frequent iterations to the builds and it becomes important to track the quality of each build. With test automation metrics you can figure out what is the percentage of your tests that passed and what was the reason behind the failed tests.
Optimum Device Coverage
Test automation is effective when the tests are executed on multiple devices in parallel. Device coverage is the most prevalent challenge as we have witnessed growing device fragmentation across the world. To ensure the smooth functioning of mobile apps on all the devices, you need to perform automation testing on hundreds of device-OS combinations.
Test automation should be designed to make the app compatible with most of the popular devices. The frequent release of new versions of OS from both Android and iOS is a major factor that drives device fragmentation. The only way to overcome this challenge is by testing the app on a cloud-based testing platform. In pCloudy, you will get the benefit of testing the app on more than 5000 device browser combinations in parallel ensuring optimum device coverage.
Summing It Up
Test automation has many benefits like better test coverage, faster feedback, and accelerated results which reduces the time to market of any application. Using the practices mentioned above you can ensure coherent test automation and increased productivity. Apart from these points, there are many other things you can do like writing original code and not copying it as the code taken from other sources might not work in your test environment. But you will always find new roadblocks which you will have to deal with spontaneously.
Intelligent Digital Mesh is the entwining of people, devices, content, and services enabled by digital models, business platforms and a rich, intelligent set of services to support digital business. We have witnessed the implementation of AI in every technology to leverage the benefits of autonomous systems. Enterprises are now focusing on using AI with technologies like blockchain and immersive technology which will create new categories of apps. In this type of environment, attaining optimum device coverage will be essential to ensure quality services. Now let’s understand the fundamentals of the intelligent digital mesh.
Intelligent
In the near future, most of the mobile applications and services will use artificial intelligence or machine learning at some level. AI will be the inconspicuous force of most of the popular app categories while creating some new ones. Intelligent apps also create a new intelligent layer between people and systems as seen in enterprise advisors and virtual user assistants. Augmented analytics is also gaining ground and helping enterprises in enhancing business intelligence and data analytics using ML and NLP. Another use of AI and ML is in intelligent things like smart vacuums, drones, autonomous farming vehicles. Intelligent devices are getting smarter to serve better and reduce human dependency to a minimum.
Source: Gartner.com
Digital
When we talk about digital, we mean digital twins, cloud to the edge, conversational platforms, and Immersive Experience. A digital twin is a digital representation of real-world objects. It offers information on the state of the counterparts, improves operations and adds value to the operations by responding to the changes. In the near future, all the aspects of human life and the real world will be interconnected with their digital representation capable of advance simulation, analysis, and operation. This combined with immersive technologies like AR, VR, and MR will take extended reality to a new level.
Mesh
Mesh is the connection between devices, people, businesses, services, and content to build a digital ecosystem that yields high-quality results. Here mesh refers to technologies like Blockchain, Event-driven, and continuous adaptive risk and trust (CARTA). Enterprises are keen to find new ways to sense the new business events to get the most out of it. A business event can be a change in the status of the deal like finalizing a deal. Using new technologies like AI, it will be easier to detect a business event and analyze it in greater detail.
Security is one of the most important and ever-evolving processes in digital businesses. There is a need to think beyond infrastructure and parameter protection. Continuous adaptive risk and trust assessment is a people-centric security approach that allows for real-time risk and trust-based decision making. New methodologies like DevSecOps and adaptive honeypots should be implemented to strengthen the security of digital businesses.
Automation Testing For Intelligent Apps
Intelligent apps are at the core of the intelligent digital mesh. Nowadays most of the apps use artificial intelligence, machine learning or predictive analysis to make suggestions to the customers. The apps use real-time and historical data from user interactions and other sources to predict the needs of their users.
To ensure the quality of apps it is important to test the apps using futuristic tools. Manual testing is just enough and even automation needs to be scalable to get better results. Testing the app on a cloud-based app testing platform is the best choice as you can use as many devices as you want to test your app. Also, parallel testing increases app testing efficiency by multifold.
pCloudy’s AI-powered autonomous testing bot steals the show when it comes to testing intelligent apps. The bot tests the app on real devices with just a single click and generates a detailed report based on the test result.
Conclusion
Mobile devices, by and large, are the focal point of most of the innovations that are happening around the intelligent digital mesh. Whether it is Ai driven development, autonomous things or immersive experience, mobile apps still used as a foundation to provide the technology to the masses. But the growing complexities of intelligent apps makes it crucial to implement new methods of app testing. A cloud-based app testing platform like pCloudy is suitable to ensure quality at speed in mobile app testing. The freedom of accessing hundreds of real devices from anywhere at any time and perform manual or automation testing using futuristic features is the correct way to test intelligent apps.
Asia’s leading software testing conference, QAI STC 2019 concluded on December 6th, 2019. The theme of this year’s STC was architecting continuous quality: think, transform, and thrive in an Intelligent future. The STC is a platform for experts to display ideas, experiments, and experiences to explore challenges and suggest techniques and innovations to overcome common problems.
Avinash’s keynote focused on the future of testing which was lauded by the top minds and became the highlight of the event.
pCloudy was the title sponsor for the event and so we got a bigger platform to showcase our contribution in shaping the future of mobile app testing.
More than 500 experts from around 130 software companies gathered to learn new trends in testing, share their ideas and grow their network. There were 14 keynotes from the industry leaders and 50 professionals got a chance to take the stage and share their views about the emerging technologies.
Let’s have a look at the major learnings from this event.
Future of Testing
The STC started with a keynote by Avinash which provided insights about the future of work, applications, and testing. How “We Working” will be the primary organizational model and algorithmic management will take over the middle management to some extent. He also talked about how people will have to constantly upskill to work with the ever-changing technology and the work-life challenges will increase in the future. A digital mesh architecture will allow enterprises to build an agile, flexible, and cloud-ready ecosystem. This will enable real-time connectivity of employees, business processes, business data, and services to help address high volumes of traffic and become cloud-native and mobile-first.
Quality Engineering and Digital Transformation
There was a great emphasis on quality engineering at the STC as most of the organizations are trying to take QE to the next level. Quality engineering focuses on the end-to-end management and the basic principle is the all the teams should bear the responsibility of maintaining the quality in the process. Software QE is the assurance of high standards in the software development life cycle while implementing DevOps and Agile.
The main role is played by testers who create, implement and maintain systems used to control the quality of production processes. These people need to have a deep understanding of all the technological activities and evaluation principles.
Quality engineering methodology is even bigger in scale than the traditional QA approach and that’s the reason that QEs cannot work in silos. In quality system engineering, people in multiple roles like IT architects, designers, test engineers, project managers, business architects, etc., must cooperate to meet customer expectations. Quality engineering is driven by emerging technologies such as AI and Big Data analytics. Automation is the driving force behind turning the traditional testing into a more effective quality support model.
The quality engineering team usually partners along with the business users and the product managers for having a better understanding of the required product details to match up the problems since the starting of the product to the last stage.
Artificial intelligence, Machine learning, and IoT
The recent development in AI, ML and, IoT were the buzz creators. Experts elaborated on how augmented analytics will be utilized for creating, developing, and consuming analytics by combining these technologies. An augmented analytics engine can identify, filter, and analyze data, and then recommend what needs to be done next without the need of an IT team. These technologies will make data-driven insights accessible to a much larger set of workers.
The conference turned out to be really productive with good insights about emerging technologies and tools. It was a great opportunity to connect with software testing experts and professionals from around the globe. To say the least, it was a remarkable event where we got a great response and positive feedback from the crowd.
In our previous chapter on Android, we learned about UI Automator Viewer, Which is available on Android SDK, to get the properties of the application object. In the case of iOS, Appium itself provides an Inspector which helps users to locate those elements in the application.
First, open the simulator by clicking on the dock option.
Now in the Device/Simulators window, select the simulator. Open the Appium Desktop and keep the simulator side by side.
Once the inspector is started, select any of the objects on the screen. It will show you the complete hierarchy and properties of that object.
At the top of the window, you can see the Record button which is used to record all the actions taken and record the script.
To select any object, click on the Select Element button and then you can use Tap button to click on an object, Send Keys to enter text and clear to undo the action.
As soon as you perform an action on an object, it is recorded in the form of a script.
Once you are done with the recording you can copy the script and paste in eclipse editor.
In the next blog, we will learn how to write the first appium script for iOS.
DevOps helps enterprises to build software at a fast pace and with minimal issues. The time to market is accelerated and the bugs are fixed faster in continuous deployment with the help of automated tools. AI is much in line with DevOps as the main focus is on automating the process and with AI the system can identify patterns, anticipate issues and provide solutions. The proactive approach improves the overall efficiency of the software development life cycle. So let’s have a look at how AI is transforming DevOps.
Feedback Loop and Correlate Data
The main role of DevOps is to take continuous feedback at every stage of the process. often people use performance monitoring tools to get feedback on running applications. These tools gather much information in the form of log files, data sheets, performance matrix, and other types. The monitoring tools use machine learning to identify the issues early and make suggestions. The DevOps teams use these suggestions to make the necessary improvements to the application. Many times teams use two or more tools to monitor the health of the app and the data from all the platforms can be correlated by the help of machine learning to get a more deep understanding of the app functioning.
Software Testing
AI is changing DevOps for good by enhancing the software development process and making testing more efficient. Whether it is regression testing, user acceptance testing or functional testing, these all produce a large amount of data. AI can figure out patterns in the data collected in the form of results and identify poor coding practices which produce a lot of errors. This information can be used by the DevOps teams to increase their efficiency.
Anomaly Detection
DevSecOps is one of the essential aspects of software development as security is the key to any successful software implementation. Distribution denial of service attacks are increasing and the business needs to prepare themselves to protect their security systems from hackers. DevSecOps can be augmented using artificial intelligence to enhance security by central logging architecture to record threats and running machine learning based anomaly detection. This will help businesses proactively attenuate the attack from hackers and DDOS.
Alerts
DevOps approach might create scenarios where the team receive an overwhelming amount of alerts without any priority tag. This will create ruckus in the teams as it will be very difficult to handle all the alerts in the continuous development environment. AI can help in this scenario by tagging the alerts and prioritizing them so that the urgent ones can be worked upon immediately.
Root Cause Analysis
To fix an issue permanently, a root cause analysis is necessary. Although it might take time to do it compared to fixing the issue with a patch which will provide the instant solution. In order to find the root cause of an issue, the developers will have to spend time which will delay the release of the product. AI can speed up the process by finding patterns in the data collected and implement to fix the root cause.
The collected data can be used by implementing AI to find a pattern and speeding up the development process. The organized data is more useful and makes prediction possible. The best practice is to use machine learning to automate the tasks which are time-consuming which will ensure the smooth and effective functioning of the DevOps teams.
Testing of Mobile Apps in quite cumbersome because of sheer magnitude of testing required on variety of devices. Moreover, Mobile Apps require changes faster than other kind of Applications (Web or Desktop). That’s the reason, more and more organizations have started realizing the need of using automation testing over manual testing as much as possible.
Mobile App Automation Testing can be a massive undertaking, and if unaware, one can end up complicating the process by selecting a bad tool. With a major trending shift to open-source mobile test automation tools, there have been a plethora of tools available in most open-source software markets.
So how do you know which are the best software testing tool available in the market? Which tools will give you the most efficient solution to fulfill your enterprise’s need for speed and integration? Will manual testing suffice your app testing needs?
You would need a set of criteria to fulfil when assessing your selection of the right open-source automation tool. Here are a crucial few questions to ask:
Do you have the required skilled resource for automation tasks?
Is there ease of script development to support agile processes and shorter iteration cycles?
Does the tool support cross team collaboration for seamless use by QA and Dev?
Can it match app platform with test development language?
Will it have performance capabilities gaps while testing?
Will it support both real devices and emulators?
Does the app support multiple platforms — Mobile and Web?
Does it have multi device execution capability
How easily can it integrated with external Device cloud platforms?
Best Open-Source Mobile Testing Frameworks to use:
To take the final call, testers must have a strong awareness of the tool’s strong and weak aspects, what it can do and what it cannot, and find a balance between cost and benefit.
These are top highly adopted open source test automation frameworks available in the market. Each of these frameworks are backed by different communities due to their unique offerings to the target audiences and relevant platforms. The overall benefits are that they cover a wide range of devices. However, for technical clarity it’s important to know the pros and cons of the framework based on your mobile and web testing needs:
1. Appium: Widely adopted, it is the leading open-source test framework for mobile app (Android, iOS) test automation.
Pros:
Best suited for QA teams to test the functionality of mobile web, native and mobile hybrid apps across iOS and Android.
Its reports are limited from debugging and fast feedback loop.
Supports development tools using any WebDriver compatible language including Java, C#, Ruby etc.
Cross Browser Support and cross platform capabilities
Con: It is less suitable for performing and developing unit testing.
2. Calabash: It is a Behavior-driven development (BDD) test framework based on Ruby development language.
Pros:
Has a large community support
Cross platform development support (Android and iOS)
Provides solid reports and insights to QA and Dev teams
Easy path to both develop and test features in parallel
Simple and easy-to-read test statements
Con: It is not friendly to languages other than Ruby.
3. Espresso & XCTest UI: Both are very similar tools as they were designed for the target users. Espresso for Android and XCTest for iOS are fully maintained by Google and Apple, assuring the latest features for respective platforms.
Pros:
Latest feature integrations assure lead in market curve for developers and testers
Easy to develop techniques including test recorders
Support both types of unit testing and functional UI
Con: Both are app context only, which means limited ability to test for user condition scenarios
4. Selendroid: An open source automation framework which drives off the UI of android native, hybrid and mobile web application. A powerful testing tool that can be used on emulators and real devices. And because it still reuses the existing infrastructure for web, you can write tests using the Selenium 2 client APIs.
Pros:
Can interact with multiple Android devices and simulators simultaneously
Can simulate human actions like touch, swipe, drag etc. on devices
Supports development tools using any WebDriver compatible language including Java, C#, Ruby etc.
5. Robotium: Widely adopted open source Android test Automation framework.
Pros:
Easy to write powerful test scenarios
Full support for native and hybrid Android Apps
Easy to use recorder
Handles multiple Android routines automatically
6. EarlGrey: EarlGrey is a native iOS UI automation test framework that enables you to write clear, concise tests. It integrates with Xcode’s Test Navigator so you can run tests directly from Xcode or the command line.
Pros:
Works directly from XCode
Full support for native and hybrid Android Apps
Synchronization features which automatically synchronizes with the UI and network requests.
