Category Page

Author: Avinash Tiwari

Avinash Tiwari is a thought leader, a recognized keynote speaker, and the co-founder of pCloudy.com. He has 15+ experience in Product development and Testing. He brings with him a passion for emerging technology and quick adoption, both of which have solidified his reputation as a leader in the mobile app testing field.

Avinash Tiwari | Posted on | 2 min Read

App Testing during your Dev Cycle – Developer Tool to Access Real Devices on Cloud

In today’s fast-paced development landscape, quality assurance is no longer the sole responsibility of QA teams. Developers and testers are increasingly working together as part of a DevOps-first approach, ensuring that quality is embedded throughout the development lifecycle. Nowhere is this collaboration more critical than in mobile app testing, where continuous iterations demand faster feedback and efficient testing strategies. 

To keep pace with this, developers need tools that provide real-time testing capabilities on real devices. This is where cloud-based solutions bridge the gap between development and quality, empowering teams to test and debug seamlessly. 

The Importance of App Testing in DevOps

In the world of DevOps, where speed and agility define success, continuous testing is a crucial pillar. App testing in DevOps goes beyond finding bugs; it plays an integral role in ensuring seamless collaboration, faster releases, and enhanced product quality. Here’s an in-depth look at why app testing is indispensable in DevOps. 

1. Enables Faster Feedback Loops

In DevOps, the objective is to deliver high-quality software quickly and continuously. Testing provides instant feedback on the quality of the code after every change, helping developers address issues early. This prevents defects from compounding and becoming costlier to fix later. 

Key Benefits: 

    • Developers receive immediate insights after each code check-in. 
    • Bugs are detected and fixed in real-time, avoiding production delays. 
    • Faster resolution times improve overall efficiency and team productivity. 

2. Ensures Continuous Integration and Delivery (CI/CD)

Continuous testing forms the backbone of CI/CD pipelines, ensuring that automated tests are executed with every build. This practice ensures that each integration of new code doesn’t break existing functionality. 

Key Benefits: 

    • Reduces the chances of last-minute surprises before deployment. 
    • Ensures new features and patches are stable before they are merged. 
    • Automates regression tests to validate large portions of the codebase efficiently. 

3. Supports Shift-Left and Shift-Right Testing Approaches

DevOps encourages a shift-left testing approach where testing is conducted early in the development lifecycle. Additionally, shift-right testing focuses on post-deployment monitoring to gather user feedback and ensure stability in production. 

Key Benefits: 

    • Early testing (shift-left) identifies issues before they become deeply embedded. 
    • Post-release testing (shift-right) enhances user experience and reliability in real-world environments. 
    • Combines pre-release and post-release insights to drive continuous improvement.

4. Reduces Time-to-Market

App testing in DevOps accelerates the release cycles by automating repetitive test cases and integrating them into the CI/CD process. Automated tests enable teams to quickly validate code changes and release builds faster. 

Key Benefits: 

    • Speeds up regression and performance testing. 
    • Allows frequent releases without compromising quality. 
    • Reduces bottlenecks in the testing process, helping teams meet tight timelines. 

 

5. Enhances Collaboration and Transparency

DevOps fosters a culture of collaboration between development, QA, and operations teams. Automated test reports and dashboards improve visibility, ensuring that all stakeholders are aware of the application’s health. 

Key Benefits: 

    • Improves cross-functional communication through shared dashboards. 
    • Helps developers, testers, and operations teams work in sync toward a common goal. 
    • Reduces miscommunication and silos by providing a single source of truth for quality metrics. 

6. Increases Test Coverage and Reliability

Manual testing is often limited by time and resources, while automated testing within a DevOps framework ensures comprehensive test coverage. This includes functional, performance, security, and compatibility tests. 

Key Benefits: 

    • Executes thousands of test cases across various environments. 
    • Validates edge cases, boundary conditions, and stress tests in parallel. 
    • Ensures that all areas of the application are consistently validated. 

7. Reduces Post-Release Defects

One of the goals of DevOps is to minimize post-release issues. Automated testing detects vulnerabilities and inconsistencies in pre-production stages, reducing the risk of defects in live environments. 

Key Benefits: 

    • Enhances customer satisfaction by reducing app crashes and bugs. 
    • Improves product stability, even after frequent updates. 
    • Reduces the costs associated with post-release hotfixes and patches. 

8. Enables Scalability of Testing Across Devices and Environments

For mobile app testing, DevOps workflows require testing across different devices, operating systems, and screen sizes. Cloud-based testing platforms ensure that teams can scale their testing efforts without maintaining physical device labs. 

Key Benefits: 

    • Provides access to hundreds of real devices for testing. 
    • Ensures compatibility across various OS versions and device models. 
    • Supports parallel testing, significantly reducing execution time. 

9. Facilitates Performance and Load Testing

In a DevOps environment, testing isn’t limited to functional validation. Performance and load testing are essential to ensure that the application can handle real-world usage patterns and peak traffic. 

Key Benefits: 

    • Identifies potential performance bottlenecks early. 
    • Ensures the application remains responsive under heavy user loads. 
    • Prevents service disruptions by validating server and API performance. 

10. Promotes Continuous Improvement

DevOps is centered around the concept of continuous improvement, and testing plays a key role in gathering insights for refinement. Test results, post-deployment feedback, and analytics help teams fine-tune their processes. 

Key Benefits: 

    • Provides actionable insights for improving the testing process. 
    • Helps in refining test cases based on past failures and new scenarios. 
    • Encourages experimentation and learning from each release cycle. 

Why App Testing is Critical in DevOps

Without continuous testing, DevOps practices cannot achieve their full potential. Here’s why app testing is a critical component in DevOps: 

    • Prevents regression issues with each code iteration. 
    • Improves release confidence, ensuring builds are stable and reliable. 
    • Optimizes workflows, enabling teams to focus on innovation rather than firefighting. 


By incorporating robust testing tools and strategies, teams can automate mundane tasks, improve coverage, and release high-quality software faster. Platforms like Pcloudy empower teams by providing cloud-based device access, real-time debugging, and seamless integration with CI/CD pipelines. 

Challenges Faced by Developers

Despite advancements in DevOps, mobile app developers continue to face unique challenges when testing apps: 

 

  1. Access to a Variety of Devices and OS Versions

Developers need to test their apps on different device models, screen sizes, and operating system versions to ensure compatibility and performance. However, maintaining an in-house library of physical devices is expensive and difficult to manage due to: 

    • Frequent OS updates and new device launches. 
    • The need for both legacy and latest devices to cover all user scenarios. 
    • Limited access to rare or region-specific device models. 

This results in insufficient device coverage, leading to undetected bugs in production when users with different device configurations encounter issues. 

 

2. Need for Robust Debugging Capabilities on Real Devices

Even when organizations provide device clouds, developers often lack the level of control they need for comprehensive debugging. Many platforms only allow basic interactions, such as installing and uninstalling apps, without: 

    • Access to system-level logs and detailed runtime data. 
    • Support for issuing custom ADB (Android Debug Bridge) commands. 
    • Advanced capabilities like port forwarding and shell access for running diagnostic commands. 

Without robust debugging, identifying the root cause of complex issues becomes time-consuming, impacting development speed and overall productivity. 

 

  1. Access to Specific Device Models for Debugging Production Issues

When an issue is reported in production, developers need access to the exact device model and OS version where the bug occurred. Unfortunately, this is not always feasible due to: 

    • Limited physical devices in-house that may not match the reported configuration. 
    • Remote or distributed teams that cannot easily share physical devices. 
    • High costs and logistics associated with procuring new devices for debugging one-off issues. 

In such cases, developers often resort to inefficient workarounds or face delays in issue resolution, which can affect customer experience and app ratings. 

 

  1. Cumbersome Physical Device Setup

In many workspaces, developers juggle multiple mobile devices connected to their computers via USB cables. This setup often leads to: 

    • Tangled cables and overcrowded workspaces. 
    • Devices frequently disconnecting during testing. 
    • Lost time switching between different devices for testing and debugging. 


These inefficiencies slow down development cycles and create unnecessary frustration, especially when working on apps with frequent updates and tight release timelines. 

Cloud-Based Solutions for Developers

To address the challenge of device access, many organizations have adopted device clouds that provide remote access to a library of real devices hosted in data centers. These platforms allow developers and testers to: 

    • Run tests on various device models and OS versions without managing physical devices. 
    • Scale their testing efforts without maintaining an expensive in-house device lab. 


However, most device clouds are designed primarily for QA teams and automated testing, offering limited support for hands-on debugging by developers. Key limitations include: 

    • No direct integration with popular IDEs like Android Studio or Eclipse. 
    • Restricted access to ADB commands, making real-time debugging difficult. 
    • Limited file transfer options for testing specific scenarios. 


Due to these gaps, developers often still rely on physical devices, undermining the benefits of cloud-based solutions. 

Introducing DeviceTunnel by Pcloudy

Pcloudy’s DeviceTunnel bridges the gap between device clouds and developer needs by enabling complete control of cloud-hosted devices directly from the developer’s local environment. 

How DeviceTunnel Works 

    • Connects to Pcloudy’s cloud servers over the internet. 
    • Appears to local tools like Android Studio or Eclipse as if it’s physically connected via USB. 
    • Developers can interact with the cloud-based device exactly as they would with a physical device. 

Capabilities of DeviceTunnel

Once connected, developers can perform a wide range of actions to debug and test their apps: 

    • Issue ADB Commands: Execute commands for debugging, shell creation, port forwarding, and retrieving system information. 
    • File Transfer: Push and pull files between the local system and cloud-based devices for testing various use cases. 
    • App Management: Install and uninstall applications seamlessly. 
    • Advanced Debugging: Add breakpoints, inspect variables, and analyze runtime metrics to identify performance bottlenecks and optimize the app. 
    • Run Tests Directly from the IDE: Execute tests within the development environment without needing to switch to external platforms or tools. 


For a step-by-step guide on connecting to cloud devices using DeviceTunnel, click here. 

Why DeviceTunnel is a Game-Changer

DeviceTunnel transforms cloud-based mobile app testing by providing developers with real-time, hands-on access to cloud-hosted devices. 

  1. Eliminates Physical Device Setup 

a. No need to maintain an in-house library of physical devices. 
b. Reduces clutter and time spent managing physical connections. 

2. Real-Time Debugging 
a. Debug apps on real devices as if they were connected locally. 
b. Identify and resolve issues faster with direct ADB access and advanced debugging features. 

3. Seamless Integration with IDEs 

a.Works seamlessly with Android Studio, Eclipse, and command-line tools. 

b. Enables uninterrupted workflows for faster development cycles. 

4.Improves Collaboration Across Teams 

a. QA and development teams can share device logs and testing results. 

b. Ensures smoother handoffs between testing and debugging stages. 

In an era where speed and quality are non-negotiable, DeviceTunnel helps teams bridge the gap between local development and cloud testing. By empowering developers with real-time device access, Pcloudy enables faster debugging, fewer production defects, and accelerated release cycles—ultimately enhancing the overall app experience. 

Automate with Selenium

Get a free demo

Avinash Tiwari | Posted on | 2 min Read

13 Advantages of Automation Testing

Automation Testing Advantages: A Comprehensive Guide

Automation testing has transformed how software is developed, tested, and delivered. By leveraging tools and frameworks to execute predefined test cases, automation testing offers speed, precision, and scalability that manual testing cannot match. Below, we delve into the advantages of automation testing, highlighting how it addresses key challenges and accelerates software development lifecycles. 

1. Faster Feedback Loops

Automation testing enables rapid feedback during various stages of the software development lifecycle. It enhances communication between developers, testers, product owners, and designers, ensuring issues are identified and resolved promptly. 

Benefits: 

    • Speeds up development cycles by enabling faster iterations. 
    • Minimizes miscommunication between teams through immediate defect reporting. 
    • Helps maintain project timelines without compromising quality. 

2. Significant Time Savings

Writing test cases manually is time-consuming, particularly for repetitive tasks like regression testing. Automation testing eliminates the need for repetitive manual efforts, reducing the overall testing time. 

Benefits: 

    • Accelerates regression, smoke, and performance testing. 
    • Shortens delivery cycles for time-sensitive projects. 
    • Frees testers to focus on exploratory and high-priority testing. 

3. Early Bug Detection

Automation testing allows bugs to be identified during the initial stages of development. This proactive approach prevents issues from escalating into larger problems. 

Benefits: 

    • Saves time and resources by reducing debugging efforts in later stages. 
    • Improves overall code quality. 
    • Enhances collaboration between developers and QA teams by integrating defect tracking into the CI/CD pipeline. 

4. Reusability of Test Scripts

Test scripts in automation can be reused across multiple projects with minimal modifications. This reusability is particularly beneficial for applications with frequent updates or releases. 

Benefits: 

    • Reduces the effort required to adapt tests for different versions. 
    • Ensures consistency in test execution across environments. 
    • Saves significant time in setting up test cases for new projects. 

5. Running tests anytime, anywhere:

Automated testing help test engineers to run their tests 24/7. If the test engineer has to leave early for the day, he can easily schedule the tests and leave the office. The test results will be ready by the time he logs in again next morning. 

6. Distributed Test Execution

Automation testing cuts down complexities with its distributed test execution feature. It helps the testers to run a test script on more than one computer or shared network or servers simultaneously. So, only an automation testing tool is the requirement rather than multiple tools. 

7. Robust and simpler reporting:

Automation testing gives us the benefit of tracking each test script. All the test scripts executed will be visible in visual logs. The reports generated can evidently show the number of test scripts already executed, scheduled, their reported bugs or issues and the ways in which they have been fixed. 

8. Testing Capabilities:

Automated testing offers an unmatched and huge testing capability. The mobile app needs to be tested on multiple devices, OS versions, screen sizes etc. which can be efficiently done through automated testing and not by manual testing. In fact, it is almost impossible to get perfect results through manual testing.

9. Better Test Coverage

Test automation can easily execute thousands of different complex test cases during every test run providing coverage that is impossible with manual tests. It can only be possible through automated testing as it can run test scripts on multiple computers with varied configurations. It can look inside an application and see memory contents, data tables, file contents, and internal program states to determine if the app is functioning as expected. 

10. Less Manual Effort:

Automation testing significantly reduces dependency on manual testers for repetitive and time–consuming tasks. This allows QA teams to focus on higher-value activities. 

Benefits: 

    • Minimizes human error during testing. 
    • Optimizes resource allocation for other critical tasks. 
    • Increases overall team productivity. 

11. Improves Accuracy:

We have been saying this time and again that automation testing overcomes the shortcomings of manual testing. It has improved the accuracy to a great deal by giving error-free results unlike manual testing where testing is error-prone, delays the delivery and increases the cost. It is especially a boon in stress testing where getting error free results in manual testing is almost impossible. 

12. Return on Investment:

One of the most important advantages of automation testing is the return on investment to the organization. Every enterprise analyses the return it would get out of its investment and then would go for creating a test automation framework. Automated testing offers immense returns in terms of faster testing, error-free results, less manual effort. If the enterprise has automated testing tool, then the testing would be more efficient and accurate and easy. 

13. Volume:

You can run your tests on more than 500 devices in automation testing which is impossible in manual testing. 

One of the most important business advantages of automation testing is the return on investment to the organization. There are many advantages of automation testing tools as they are a crucial component in the DevOps toolchain. The importance of automation testing comes into picture when you have to run the test cases with different data and conditions several times. 

Conclusion

Automation testing not only saves time and improves accuracy but also adds significant value to all stakeholders involved in the app development lifecycle. Despite its many benefits, some organizations hesitate to adopt automation due to initial configuration costs. However, with Agile and DevOps methodologies becoming the industry norm, automating testing processes is no longer optional—it’s a necessity. 

By investing in renowned automation testing tools and frameworks, organizations can ensure faster releases, error-free applications, and satisfied customers. Whether you’re new to automation or scaling your testing efforts, platforms like Pcloudy can help you achieve unparalleled efficiency and coverage. 

Automate with Selenium

Get a free demo

Avinash Tiwari | Posted on | 2 min Read

Application Performance Monitoring

What is Application Performance Monitoring (APM)?

 

Application Performance Monitoring (APM) involves employing various strategies and tools to ensure the efficient operation of business applications. These apps need to meet the performance, reliability, and user experience requirements set by stakeholders, such as employees, clients, and partners.

 

Mobile Application Performance Monitoring is a part of the broader term Application Performance Management, which encompasses all aspects of managing an application’s performance, while APM focuses on monitoring an application’s performance. In simple terms, monitoring is an integral part of management.

 

What is the purpose of APM?

The main goal of Application Performance Monitoring (APM) is to maintain an application’s optimal performance and smooth operation. It enables linking application performance to business results, detecting and correcting mistakes before they impact end-users, and minimizing mean time to repair, aiding organizations in maintaining business continuity and enhancing customer satisfaction

 

Why is APM crucial?

 

Application Performance Monitoring (APM) is important because of all the advantages it offers to a business in terms of operational effectiveness, brand recognition, and long-term cost savings.

 

By enabling front-end monitoring, managing the user experience, back-end monitoring of the services and dependencies used by the app, and infrastructure monitoring, APM offers improved collaboration and coordination in the application delivery process. This promotes seamless and transparent application delivery and the elimination of silos.

 

APM safeguards the organization’s reputation and brand image by ensuring applications are reliable and accessible. Poor application performance can lead to user dissatisfaction, lower application utilization, slower adoption, and decreased revenue. APM is essential to maintaining a good reputation and ensuring application success.

 

App Performance while conducting Manual Tests

The performance of an app can be viewed real-time while conducting functional tests. You can simply select the application installed on the device from the Tools section to view a graph of Memory and CPU usage, with the packets/bytes consumed in the Data Usage.

application performance monitoring
application performance monitoring

All information captured during a app testing session is gathered by the platform and stored in a secure location to be used later. When a tester completes his testing session using a device, the performance related information can be found in the Cloud Drive.

 

Battery Consumption

The Battery Consumption graph indicates Battery consumption by Device, Battery consumption by Application and Battery consumption in %.

Perf_Battery_Consumption

 CPU Consumption

The CPU consumption of an App is plotted as CPU consumption in percentage against Time Duration in seconds.

Perf_CPU_Consumption

 

Memory Consumption

The CPU consumption of an App is plotted as Memory consumption in ‘Mb’ against Time Duration in seconds.

Perf_Mem_Consumption

How pCloudy can help you?

 

Mobile application testing does not stop when all the functional tests pass. Testing the application performance monitoring is a critical step before releasing it.

 

pCloudy helps you with performance profiling of your mobile apps. You can view the CPU usage, Memory usage, Data usage and Battery consumption while performing a series of actions on the app. The Real time app performance monitoring feature helps you to keep track of the app’s performance while conducting manual tests. You can even check the performance consistency of the app by running automation test suites in parallel on multiple devices and by also simulating different network environments.

 

Avinash Tiwari | Posted on | 2 min Read

Navigating through Uncertain times – My journey as an Entrepreneur

As I write this today, it’s been almost two months of full-scale disruption across the world due to Corona Virus pandemic.
 
As humans, we always have this tendency to believe that we can control the impact of events around us, until one day something exceptional happens and it breaks every assumption and norms, we are familiar with.
 
With no time to realize or sync in with the change, the world was struck with a sudden lockdown and crisis. We all were only surrounded by uncertainty and were panic stricken.
 
On the positive side, this period has forced everyone to slow down and focus on what’s really important at the moment. The word “essential” is clearly itched in our minds for some time to come.
 
At individual level, our mindless pursuit of material well-being has been replaced by our inner well-being. We are celebrating the return of fresh breathing air, clear blue skies, a fresh tinge of green in plants and trees around today.
 
After staying in a never thought of lockdown for more than 50 days, I thought of sharing my experiences and feelings and how I, as an entrepreneur am coping up with these uncertain times.
 
So, I would break this entire situation into 3 phases, which I am sure most of you would be able to relate to.
 

Phase 1: Chaos

 
This crisis created a chaos which forced us to think about what the essential elements for our business are. Does it matter how fancy our offices are, or how big the lunch or dinner is for a client? So, we started looking at three core essentials.
 
a) Our Product – What problem are you solving as a business and is it relevant in today’s time? At the same time will it be relevant for the coming future?
 
We started looking at it by asking the basic fundamental question – If we have to start the business all over again today, what part of business we would like to build? This analysis really helped us figure out what’s essential for us as a product/offering and business model? Apart from above, some open-ended questions helped us reimagine the future roadmap of product.
 
1. What are we currently not doing for our customers?

2. What are the needs of the customers for which there are no solutions?

3. Which areas of our existing customers are growing?
 
b) Our people – Removing all the fringe around people we had built over the years, it’s really “them” which has become the most important aspect now. If organisations want employees to do and deliver more, then employees must feel valued as people. That’s especially critical when health concerns intersect with work responsibilities. Guarding my most critical asset—my employees—both physically and mentally is my top priority now.
 
c) Our Customer – What is essential with the customer right now? Empathy, ability to listen and be a 2 AM friend to your customer. Flexibility to find ways to sail together in these difficult times.
 
We realised that hard selling can boomerang. Companies are already facing tough time. It was important to take care of the loyal clients and invest in them more. We decided to use this opportunity to strengthen your relationship with them and build more trust.
 

Phase 2: Stabilisation

 
With the teams settling to the new working conditions and adapting to virtual collaboration, we focused on the things that we as a businesses can and can’t do. We knew this is going to be the new normal and we had to develop well thought out contingencies and new approaches.
 
a) Not to keep head in the sand:
 
Financially, we immediately prepared ourselves for the plan A (worst case) and Plan B (optimistic case) and Plan C ( aggressive) . We started with Plan A. Whatever cost optimisation was needed was done quickly. We decided the focus areas based on whatever signals we were getting. Almost daily monitoring of situation helped us move switch from Plan A to Plan B and finally to Plan C quickly.
 
b) Spark the team spirit and keep moving together:
 
We created focused group of people handling different areas of organisation and had discussions with them on daily/weekly basis. We have started a lot of initiatives for learning new skills which can be anything apart from work. Learning a new language, cooking, photography anything…that you craved to do all this while. Trust me, I have seen a tremendous improvement in my team’s productivity and efficiency. They are more accountable and responsible towards their work and that has led us to witness some significant success even in these times.
 
c) Be human. Be helpful. Show solidarity. Mindful marketing: It is high time that we come out of the notion of business as usual. This is the time when we are doing “business unusual”, and it’s likely to be that way for some time. We reoriented our sales team not to be 2AM friends to our customer, not to hard sell but empathise. We checked with our clients and partner if they need any help. Help them in whatever way you can. It’s was and still is important to keep your name out there, but make sure you’re not being tone-deaf.
 

Phase 3: Realisation

 
Now, this is the current phase I am in, and it is truly an exciting phase for me as an entrepreneur and for us a company. With the help of our customers, partners and my team, we have got a fair idea of where the businesses will begin to focus. Ours being a enterprise SAAS product over cloud, has done wonders for companies adapting to the new working culture. We have a huge growth in the month of April and it’s continuing. We have been able to clear significant product backlog with some really exciting new features. We are upgrading our data centres to be ready for increasing load. At the same time, we have started our new DC in Singapore. We have on-boarded middle management in key areas of pre-sales and customer support. Overall, lots happening.
 
We are also mindful of the fact that crisis is not over. Situation across the world is still fluid. It’s difficult to predict the future at this moment. So, we have to be on our toes for some time to come. We have to be agile and adapt to the situation. But last two months have given immense learning as a business and personally as an entrepreneur.
 
I read this quote somewhere, and it’s truly what I believe in now – ‘Out of constraints comes creativity, out of chaos comes growth, out of fear comes success’.

Avinash Tiwari | Posted on | 2 min Read

How to Analyze Data to Predict Your Optimum Device & Test Coverage?

It’s time to predict

 

Imagine that you developed a mobile app but it does not work properly on some devices as the device fragmentation is getting more complex with time. This will be a tester's nightmare as the users will drop the ratings of the app which will lead to a decrease in the downloads.

 

Teams need to consider the device usage patterns across geographies to test their apps. This will lead to smooth app functionality across all digital channels. This is the need of the hour.

 

We can optimize the device and test coverage by using data analytics in app testing instead of relying on guesswork. You can also watch the webinar video where we have covered this topic extensively.

 
Before we move on, let’s understand some of the terms that will be used in this article.

 

Terminology

 

Term Description
Stock Android A basic and unmodified version of OS developed by Google.
Android Custom ROM Read Only Memory modified to provide better functionality than that in Stock Android.
Manifest file The manifest file describes essentialinformation about your app to the Android
build tools, the Android operating system, and Google Play.
Compile SDK version The version of Android that the build tools uses to compile and build the application in
order to release, run, or debug
Minimum SDK version The oldest version that you want your app to support.
Target SDK version The version of Android that your app was created to run on.

 

The growing complexities of Device Fragmentation

While 70% of the Apple devices are running on the latest iOS 12, only 0.1% of the Android devices run on the latest Android 9.0. According to statista.com, Android 7.0 is still being used by 19% of the Android users, followed by Android 6.0 with 21.6% Android user share. Around 7.8% of the Android users are still using Android 4.4 which was released in 2013. In 2018, 88% of the mobile devices were running on Android operating system compared to 12% iOS devices.

 

It is not feasible to run a test on all the devices as the permutations will be huge. There are 16 device display categories, 20 different resolutions, 15 android versions, 400 android device manufacturers and multiple cellular networks. I am sure you will be able to relate to it. So, there are two traditional ways to create an optimum device matrix. First is based on market penetration and the second is based on device diversity. Here is a brief overview of both the ways we have been using on our convenience.

 

To get a better understanding of the device fragmentation based on operating system, we need to understand the basics of Android architecture. So let's get on with it.

 

Android Architecture

Android is an open source operating system based on the Linux kernel created for an array of devices. Android has a five-layer architecture as mentioned below:

 

  • Linux Kernel: It is the base of the Android operating system. It gets the
    work done from the hardware for e.g. camera, display, Wi-Fi etc.
  • Hardware Abstraction Layer: It provides standard interfaces that expose device hardware capabilities to the higher-level Java API framework. The HAL consists of multiple library modules, each of which implements an interface for a specific type of hardware components, such as the camera or Bluetooth module.
  • Native C/C++ libraries: It consists of some logical instructions that are sent to the kernel to perform an action.
  • Android Runtime: It compiles the applications from java code to byte code which Linux can understand.
  • JAVA API Framework: The entire feature-set of the Android OS is available to you through APIs are written in the Java language. These APIs form the building blocks you need to create Android apps.
  • System Apps: These are apps built in the device like calculator, clock, calendar etc.

 
android-stack_2x

Source: https://developer.android.com/guide/platform/

 

In some cases, mobile device manufacturers make customizations in the Android architecture to enhance the user experience. That is done by using custom ROM.

 

Why bugs occur

One of the major reasons for bugs is the fast evolving Android platform. If you have built an application then you need to ensure that the app works on various API levels that are still used widely. Every new API level launch comes with changes in the guidelines and this raises the chances of app failure. Many brands like Xiaomi, One Plus, Huawei etc, keep releasing new phones and as an update, they customize the ROM. All their devices have diverse specifications like screen size, RAM, processor, camera etc. In the table below you will see the layers of android app stack and how they are divided into two sections the higher level and the lower level.

Why bugs occur?

The OEMs make changes in the lower level system to add a device specific functionality by creating a supporting driver at the hardware abstraction layer. So if the driver has a bug than the higher level system will have a bug too as the APIs communicating with those drivers will also have the bugs. The OEMs also change the higher level system to enhance their UI and these changes also create compatibility issues.

 

OEMs customizations and its impact

According to Statcounter, this year's data shows that Samsung and Apple have almost 32% and 23% of the market share in mobile sales worldwide. Chinese companies like Xiaomi, Huawei, and Oppo has around 20% market share combined. There are several local OEM's in many countries adding to the diversity in mobile devices. Many OEM uses custom ROMs to provide added features but custom ROMs comes with some duplicate apps.

 

Most of the custom ROMs like MIUI and Oxygen OS has its own calendar app, music app, and their own browser. UI skins are a significant source of fragmentation in OEMs. This means that we need to consider custom ROMs while creating a strategy for device coverage.

 
Device Fragmentation
 

How Android evolved and its impact

Android has released many versions and API levels since its inception. API level is a unique identification code which is given to every framework API revision offered by a version of Android. For e.g., Android 8.0 is API level 26, Android 8.1 is API level 27 and the next version Android 9.0 is API level 28. Developers can specify the compile SDK version, target SDK version and a minimum SDK version in the manifest file of an app as it tells you which app is supported by a particular device.

 
API level
 

What we have seen so far

We have observed a trend in the occurrence of compatibility issues and their root causes. We can divide the compatibility issues into two groups, device specific and non-device specific. In device specific issues the first and the most common root cause is problematic driver implementation. For e.g., there was an issue in some devices related to the proximity sensor. If you bring the device near your year the touch screen should get disabled but that didn't happen in those devices. The second root cause is OS customization. Some devices brought the multi-window support feature where you can open multiple application in a single screen. That
feature created a lot of bugs in most of the applications. The third issue is peculiar hardware composition.

 

Type Root Cause Percentage of Issues Issue Examples
Device-Specific Problematic Driver Implementation 23% Proximity Sensor issue in Samsung devices
OS Customization 10% Multi window support
Peculiar hardware composition 7% Amazon Fire: OMX Codec issue
Non-device Specific Android Platform API evolution 47% API level 26: 8.0: Internationalization
Original Android System bugs 13% Android 8.1 Bluetooth issue

 

The majority of compatibility issues falls under non-device specific issues. Most common of these are due to changes in the Android platform APIs. In Android 8.0 API level 26, there was an internationalization failure. That means if your app is built to work with different languages than it will show a bug. The second root cause in non-device specific issues is the original Android system bug. These issues are resolved quickly as soon as these bugs are reported to Google it releases a patch to fix it.

 

How to predict

Android regularly publishes release notes on its forums which talks about the behavior changes for all apps. These changes include power management, security behavior changes, Android test changes etc. They also inform about the behavior changes for new apps targeting the particular API level. This information will help you to predict and plan your test cases. You can select the devices for which you need to plan your testing and the functionalities that you need to focus on where there are maximum chances of failure.

 

For example, if your app uses the current location of the user, then Android 8.0 will have an impact because android has restricted the number of location update when the app uses the background service. You must have observed a lag in the map location in all the delivery service apps like Ola, Swiggy, Uber eats, etc. Another example can be the background execution limit. Apps running in the background consumes device resources and result in unpleasant user experience. That's why Android 8.0 has limited the background execution limit.

 

We have created a cheat code by analyzing the data that you can refer to perform testing on Android 8.0 and Android 9.0.
Click on this link to download the cheatcode

 

Conclusion

It's high time that we say testing is just not enough. It's time to predict. Data has a lot to say. You can utilize it to help you in various ways. When predicted accurately, it can take your testing to another level.

 
Related Articles:

  • Why Choose Automation for Cross Browser Testing
  • A Brief Overview Of Regression Testing
  • How to Measure the Success of End to End Testing
  • Mobile Responsive Testing and Debugging on Real Devices
  • Mobile Devops+Agile – Challenges and Keys to Success

    • Avinash Tiwari | Posted on | 2 min Read

    Welcome to pCloudy 5.0

    pCloudy 5.0

     

    pCloudy is a unified mobile app testing platform which is trusted by its users for its excellent performance. We have more than 30,000 users across the globe, and we have received an overwhelming response for our platform. From bot testing to crowd testing, pCloudy has always been a front runner in the market. And with the launch of pCloudy 5.0, we are up for the challenge once again. It is a re-imagined and re-worked upon platform created in-line with intelligence to meet the market demands and accelerate delivery.

     

    Let us have a look at what’s new here:

     

     

     

    Features

     

    a. iOS Connect
    b. Automation Studio
    c. Follow Me

     

    a) iOS Connect: One of the very powerful features that we added is the iOS Connect. The major challenge iOS developers had been facing was the debugging of their iOS apps because of the difficult Apple guidelines regarding building any utility . We are proud to say that we have been successful in cracking that.
    With iOS Connect, you can connect to an iOS Device which is present remotely and access the device as if the device is connected to your computer. With iOS Connect we are bridging the gap for the iOS development life cycle.

     

    iOS Connect

     

    b) Automation Studio: It is another promising feature that can revolutionize mobile app automation. We now have an in-built automation studio in pCloudy platform where you can access the devices to record your test scripts and execute it in parallel on multiple devices without writing any code.

     

    There’s something more to it! If you already have your projects on Appium you can record your new workflows and export it to Appium Java format. This new integration would not only enable every user to start creating Automation scripts but it will also help Appium users to speed up their Automation. This makes pCloudy truly a Unified app testing platform.

     

    Automation Studio Launch

     

    c) Follow me: It is again one of the ‘wow’ features of pCloudy 5.0. This feature can speed up your App Testing exponentially. Follow me allows you to test your apps on 3 unique device while performing your test on only 1 device, thus saving your testing time multi-folds.

     

    Follow Me

     

    Re-Imagined UI

     

    It has been designed to give a leaner, simpler and faster user experience. The look and feel of the pCloudy platform has been transformed keeping in mind the requirements, usability and ease of our users. All the pages have been made more intuitive. To simplify, we have grouped the actions together to save your time in figuring out the next step.

     

    a. True Collaboration
    b. Global File Storage
    c. Comprehensive reports at a single stop
    d. Test Scheduler

     

    a) True Collaboration: We have introduced a powerful feature on our revamped Device Interaction page. Now, you can take the screenshot of the action you are performing and see the screenshot next to the device. But, the icing on the cake is that you can edit the screenshot, save it and log a bug directly to the JIRA enabling true collaboration between the developers and testers.

     

    UI Design

     

    b) Global File Storage: Heeding to the demands of our users we have reduced the clicks to perform any action on the File browsing /storage page. The file uploaded once will be available across all the different locations we have. It will also allow sharing of files across teams.

     

    Global File Storage

     

    c) Comprehensive reports at a single stop: Now, to make the user experience simpler and faster, we have placed all your reports at single stop be it manual or automation across all locations. Not just that, you can see all reports in the same format with much detailed information. Raising bugs after analyzing reports is also possible from here.

     

    comprehensive-reports-at-a-single-stop

     

    d) Test Scheduler: We have re-designed this page based on a Guided interface where the interactions are more thoughtful, straightforward, and guide you to the next step. It needs zero learning to run automation scripts on multiple devices.

     

    Test Scheduler

     

    pCloudy 5.0 is another breakthrough for us. After launching pCloudy 5.0, it is time to get feedback on a larger scale. We’re going to continue improving the UI, the navigation, and add some futuristic features to make your app testing simpler and faster. While we do that, we’d love to continue getting feedback.

    Avinash Tiwari | Posted on | 2 min Read

    Year 2017 at a Glance!

    pCloudy 2017

     
    2018 is around the corner and it’s just the time to recap on the major events and milestones of pCloudy in 2017. We thank each one of you for your immense support and trust you bestow on us that keeps us working towards improving pCloudy to meet all your testing needs.
     
    Let’s recap:
     
    GROWTH OUTPACING THE MARKET: Mobile Application Testing Market is on a rapid trajectory and is currently a 3.2 Bn USD industry and is estimated to reach 6 Billion USD by 2021. The growth is not getting stagnant anytime soon. Technavio’s analysts forecast the global Mobile App testing services market to grow at a CAGR of 18.31% during the period 2017-2021. With an incessant growth in the app testing market, pCloudy made a remarkable progress outpacing the market growth and making more than a 100% jump in its user base. More than 17,000 users registered on the pCloudy platform this year.

     

    pCloudy Map

     

    GEOGRAPHICAL EXPANSION: pCloudy added another feather in its cap and launching its 4th data center in Melbourne, Australia. The launch of the data center is aimed at tapping the Australian app market helping our customers increase their app coverage to ANZ region.

     

    pCloudy world-template

     

    PARTNERSHIP WITH IT STALWARTS: 2017 was indeed a great year for us as we partnered with some of the IT stalwarts like Infosys, Wipro, Hexaware Technologies, UST Global and many more.

    logo Banner

    MAJOR PRODUCT BREAKTHROUGHS: pCloudy in its incessant effort to make faster and effective, we got two major breakthroughs this year. Let’s have a look at them:

    certifya_2812
    CERTIFAYA: Understanding the needs and analyzing the market trends, pCloudy created a milestone by introducing Certifaya – AI powered bot testing to make testing faster and efficient multi-folds.
    crowd4test_2812
    CROWD4TEST: Another major breakthrough for pCloudy this year has been the launch of crowd testing. This feature can help app testers get feedback from the real users before its market launch.

    INDUSTRY RECOGNITIONS: We are honored to get our hard work recognized by major industry analysts this year. Let’s peep through our recognitions:

    G2Crowd Test Automation Software
    G2 Crowd Test Automation Grid: Having received multiple valuable feedback from real users, pCLoudy has succeeded in becoming a part of G2 Crowd Test Automation Grid.
    et
    Lead Story in ET CIO: ET CIO covered a story on pCloudy and published it as their lead story in December this year putting an icing on the cake and making our efforts worth appreciation. You can have a look at the story here.(https://cio.economictimes.indiatimes.com/news/enterprise-services-and-applications/how-pcloudy-platform-is-solving-mobile-application-testing-challenge/61956020)

    PRODUCT ENHANCEMENT: We understand the importance of product enhancement in technology industry and that’s why we keep on upgrading our product to serve you better. This year we have upgraded it with 11 new features which makes this as one of the biggest product update ever for us. We have introduced features like –

    Calculate your test coverage
    Planner where you can get the latest stats of the market penetration of devices from across the globe. Some of the features will have a significant impact on the way you currently test your Apps.
    JIRA Task
    Integration With JIRA: pCloudy introduced this feature where a tester can now log bugs directly to JIRA while testing the application on pCloudy. All relevant data can be attached in a single click.
    JMeter Integration
    JMeter: As pCloudy has always been working towards making testing fast and efficient, it has recently launched a plugin for JMeter to bring you an insight about the Performance Testing of your application.

    A PART OF AGILE TESTING CONFERENCE, 2017: pCLoudy participated in the Agile Testing Conference, 2017 and showcased its products under two main divisions – mobile app testing platform and Robotics. It was a great show and we received phenomenal response from the audience.

     

    agile
     
    A BLEND OF WORK AND FUN: pCloudy is not just committed to its customers but also to its employees. Bringing a smile on every pCloudian is a reward and acts as a catalyst in improving our performance. Every pCloudian believes in working hard and partying even harder.

     

    pCloudy Christmas

    We are thankful to each one of you for your trust and support throughout. Embarking on such a journey would have been impossible without you. We are proud of our achievements even this year but will strive to have more exciting starts and remarkable successes in 2018. We will continue to improve our products and make your experience with pCloudy a remarkable one.

     

    Appy Holidays and have a great year ahead!!

    Avinash Tiwari | Posted on | 2 min Read

    Save 20% on All Flagship Packages at pCloudy!

    Special 20% discount

     
    Every special day is a reason to celebrate and be joyous about and to add more excitement to this holiday season; we are offering a special 20% discount on all our flagship products.

    pCloudy is growing with you and we leave no stone unturned to keep you happy. To make this holiday season business worthy, here is a chance to get any of our products (Click here to visit the packages) for 20% off.

    This discount is live & anyone can take advantage of this special offer until 31st December midnight. The 20% discount will be applicable to your order by applying a coupon code HOLIDAY17. This is a onetime offer and we would like each one of you to get the maximum benefit out of pCloudy.

    With this, we wish you all a super holiday season and enjoy this 20% discount from the pCloudy team.

    If you love your association with pCloudy, do let others know about our exclusive discounted sale by sharing it on Facebook & Twitter.

    Avinash Tiwari | Posted on | 2 min Read

    Biggest Update Ever

    pCloudy 4.6

     
    We have always believed that product enhancements need to reach the customer as quickly as possible. Their feedback is the most important part of the product evolution journey.

     

    So, we can’t wait to unveil what we have created in the last few months. We have upgraded it with 11 new features which makes this as one of the biggest product update ever for us.

     

    Some of the features will have a significant impact on the way you currently test your Apps.

     

    We bring to you the guide that will help you to get the most out of the Biggest Update Ever!…Here is what you can find new in the product.

     

    1. Check the Device matrix to be used before you start Testing

     

    Biggest Update Ever

     

    Are you always puzzled about what all devices should be a part of your test scope? We have made that decision easy for you by introducing Planner. Planner will help you to calculate your test coverage on an optimal set of devices.

     

    Planner calculate your test apps

     

    This feature will help you get the latest stats of the market penetration of devices from across the globe. Choose your market coverage and geography to get the list of devices you should be ideally opting for your test.

     

    Calculate your test coverage

     

    2. Log Bugs to JIRA directly from pCloudy Report page

     

    Log Bugs to JIRA

     

    We understand the pains a tester goes through and that’s why we have introduced this feature to bring an end to all your woes! Now you can log bugs directly to JIRA while you are testing your application on pCloudy. You can attach relevant data in one single click.

     

    Let’s see how this feature can help you log bugs in JIRA:

     

    Provide your credentials once and we store the authentication cookie provided by JIRA.

    (Note : we don’t store your credentials)

     

    JIRA Login to Test Apps

     

    After logging in you will come across a JIRA Task pop up where you can easily log a Bug or Task to JIRA along with relevant attachments (screenshots, logs or video links). No need to download a file and then upload it separately to JIRA.

     

    JIRA Task

     

    3. “Certifaya” –Our AI powered App Testing engine is smarter now

     

    certifaya ai powered app testing

     

    Our motto is to make your life easy and we always work towards making the testing quick and simple for our clients. Not only this, we provide more and more data about the App under test to improve the quality rapidly.

     

    We released a new version of Certifaya with the following changes-
    a) Certifaya coverage across devices has been increased.
    b) Moreover, you can provide App credentials or other sets of Test data.

     

    Now you can use Certifaya every time you make a change to your App. In less than 15 mins!!

     

    Watch our latest video for Certifaya!

     

     

    4. Speed up your Manual testing speed using Action Repeater

     

    Manual Testing Action Repeater

     

    Android Action Repeater (AAR) can speed up your Manual App Testing exponentially. AAR observes your manual test steps on a device, records it, and repeats it on one or multiple devices, thus saving your time to perform sanity checks, prerequisite setup or a full fledged functional test.

     

    App Launch Completed

     

    Watch the intro video of Android Action Repeater (AAR)

     

     

    5. Testing your Apps for Voice command

     

    Testing your apps for voice commands

     

    Now a days many apps can be operated by using voice commands like Google voice search, Evernote voice / audio note etc. pCloudy understands your requirement and gives you an option to send voice commands to pCloudy devices as well while testing such apps.
    Read More…

     

    inject audio

     

    6. It is easier to search a device

     

    It is easier to search a device

     

    Searching for a device is the most frequently used task in the platform. We have made searching simpler for you. Now you can use the free text search to find a device by the name of its OS version.

     

    Moreover, we have added quick search filters for most common searches.

     

    Instant Access

     

    7. Plan your Testing based on the availability of Devices

     

    Availability of devices

     

    When you are searching for the desired devices you may not see that device online if other user is using it at the same time. You may want to plan your test activity based on the next availability of the same device. We value your time and to make your testing efficient, we have made that information available and you can see the status of some devices as ‘Busy’ if it’s in use and the indicative time by when the device will be available next.

     

    Real Mobile Devices

     

    Moreover, you can even see which devices are ‘Offline’. You can now request for such devices to be made available.

     

    8. IOS App Resigning made easy

     

    iOS App Resigning made easy

     

    We have built IOS App resigning as a separate utility, so that you can resign your iPAs once and upload it on all our cloud locations.
    Read More…

     

    9. New App performance parameters

     

    App Performance

     

    New App Performance Parameters have been introduced in pCloudy to let you have better insights about your app and also to know how it is performing on different devices.
     
    We have added new parameters “Frame Rendering Time”.
     
    Application Name

     

    10. Improved Performance Graphs with New parameters

     

    App Performace Graphs

     

    Existing performance graphs have full break up of data for deeper analysis.

     

    CPU Data

    CPU Data

    		 Memory Data

    Memory Data
    Network Traffic data

    Network Traffic Data

    		 Battery Consumption Graph
    battery consumption graph

    11. pCloudy-Appium client for Nodejs

     

    We have released pCloudy-Appium client for the users who want to use nodejs for their test automation to run on multiple devices on pCloudy.

    Read More…