Mobile App Testing Challenges: A Comprehensive Guide
The mobile market has seen exponential growth over the past decade, largely driven by the mobile application industry. With more than 3.5 billion smartphone users worldwide, mobile apps have become an integral part of our daily lives. This booming sector is expected to generate over $189 billion in revenue by 2020 through app stores and in-app advertising, and the demand shows no signs of slowing down. As mobile apps become more ubiquitous, the competition to create unique, high-performing apps has intensified.
Yet, as apps grow more complex and diverse, so do the challenges faced in ensuring their quality. Mobile app testing plays a pivotal role in meeting these challenges, especially as the industry evolves at a rapid pace. Below, we explore the common issues in mobile app testing and offer solutions to ensure apps remain competitive, reliable, and user-friendly.
1. Device Fragmentation: The Complexity of Multiple Devices
One of the most significant challenges in mobile app testing is device fragmentation. With countless manufacturers, models, operating systems, and screen sizes, testing across all devices is a daunting task. Android alone has a wide range of OS versions, with older versions still in circulation despite new releases. This fragmentation means that an app may perform flawlessly on one device but crash on another.
The Challenge
Operating System Fragmentation: Apps must function across multiple operating systems, such as Android and iOS. Even within these operating systems, there are variations in performance and compatibility across different versions (e.g., Android 10 vs. Android 12).
Device Variability: Different devices have varying processing capabilities, screen sizes, and resolutions, all of which can affect app performance.
The Solution
Testing on a range of real devices is the best way to ensure compatibility. This is where cloud-based platforms like Pcloudy come in. Pcloudy offers access to real devices with varying OS versions and configurations, allowing testers to check how their app behaves across multiple devices without needing physical access to each one.
Pro Tip: It’s crucial to prioritize testing on the most popular devices and operating systems to maximize your app’s reach.
2. Network Conditions: Testing for Real-World Scenarios
In today’s connected world, apps must perform well across a variety of network conditions. However, this introduces another layer of complexity in mobile app testing. Network issues, such as low bandwidth or weak signal strength, can drastically affect an app’s performance, leading to poor user experiences. According to studies, 53% of users will uninstall an app if it crashes, freezes, or has performance issues.
The Challenge
Network Fluctuations: Users frequently switch between Wi-Fi and cellular networks, both of which offer different speeds and signal strengths.
Latency and Packet Loss: Poor network conditions can lead to latency issues, dropped packets, or complete loss of connectivity, which can make even the best-designed apps frustrating to use.
The Solution
Testing apps under real-world network conditions is essential to ensure that they perform smoothly, even under poor network environments. Pcloudy offers network simulation tools that enable testers to replicate varying network conditions, from weak 2G signals to high-speed 5G or Wi-Fi, on real devices.
Pro Tip: Regularly test your app’s performance in low-bandwidth scenarios to ensure seamless user experiences in all conditions.
3. Choosing the Right Tools: Making or Breaking Mobile App Testing
The choice of testing tools can significantly impact the efficiency and effectiveness of mobile app testing. There are numerous tools available in the market, each with its strengths and weaknesses. Selecting the right one based on your app type (native, hybrid, or web) and testing needs is critical.
The Challenge
Tool Overload: The number of tools available can be overwhelming. Each offers different features for automation, debugging, performance monitoring, and security testing.
Incompatibility: Not all tools are suitable for every app type. For instance, some may work well for native apps but not for hybrid or web-based apps.
The Solution
To navigate this landscape, it’s crucial to evaluate tools based on your specific app requirements. Pcloudy supports a wide range of automation tools, such as Appium, Espresso, and Selenium, making it easier for teams to test apps across multiple environments.
Consider the following when evaluating tools:
App Type: Your chosen tool should support native, hybrid, and web-based apps.
Cross-Platform Support: Ensure that the tool supports Android, iOS, and other potential operating systems like Windows.
Cloud Integration: Leveraging cloud platforms for test automation allows teams to access devices and results from any location, improving collaboration and efficiency.
4. Screen Size Variations: Ensuring Consistency Across Devices
Mobile devices come in various screen sizes, and ensuring that your app displays correctly on all of them can be challenging. Apps that look perfect on a large-screen phone may appear cluttered or disjointed on a smaller device.
The Challenge
Pixel Density: Different devices have varying pixel densities, which can affect the sharpness and clarity of app content.
Layout Adjustments: App elements need to adjust dynamically to fit various screen sizes without compromising user experience.
The Solution
Adopt responsive design principles to create adaptable UI layouts. This approach ensures that your app looks good on all screen sizes, from small smartphones to large tablets. Testing your app on real devices of varying screen sizes is crucial, and platforms like Pcloudy allow for testing on multiple screen configurations to ensure a seamless experience.
Pro Tip: Focus on adaptive designs rather than pixel-perfect layouts, as adaptive designs scale more effectively across different screen sizes.
Native Apps: Developed for specific platforms (iOS or Android), native apps tend to offer better performance but require separate testing for each platform.
Hybrid Apps: These apps combine elements of native and web apps. While easier to develop and maintain across platforms, they often face performance and compatibility issues.
Web Apps: Running in browsers, web apps must be tested across multiple browsers and operating systems, making compatibility a primary concern.
The Solution
Each app type comes with its own set of testing challenges, and it’s essential to customize your testing strategy accordingly. Pcloudy supports testing for all three app types, allowing teams to ensure that their apps meet the required standards of performance, usability, and functionality.
6. AI-Powered Test Automation: The Future of Mobile Testing
Artificial intelligence is revolutionizing mobile app testing by automating complex testing tasks, generating test cases, and predicting defects. AI-driven testing can significantly reduce time and effort, allowing testers to focus on more critical aspects of app development.
The Challenge
Resistance to Change: Many teams are still reliant on traditional testing methods, hesitant to adopt AI-powered testing solutions.
Implementation Complexity: Integrating AI testing into existing workflows can be challenging without the right expertise or tools.
The Solution
AI-powered test automation, like the solutions offered by Pcloudy, helps automate repetitive tasks such as regression testing, bug detection, and performance analysis. AI-driven bots can create test cases, execute tests, and analyze results, enabling faster releases and higher accuracy.
Pro Tip: Embrace AI-based testing early to stay ahead of the competition. Automating repetitive tests frees up resources for more creative problem-solving.
7. Security and Compliance Testing: Safeguarding Data and Trust
In an era where data privacy and security are of paramount importance, ensuring that your app is secure and compliant with regulations is vital. The increasing number of cyberattacks and data breaches highlights the need for robust security testing.
The Challenge
Security Vulnerabilities: Apps are often vulnerable to attacks such as data leaks, insecure storage, and unauthorized access.
Compliance Regulations: Apps must comply with regulations like GDPR, HIPAA, or PCI DSS, depending on the region and industry.
The Solution
Incorporate security and compliance testing into your QA process. Test for data encryption, authentication, and security vulnerabilities. Pcloudy offers features like biometric authentication testing and encrypted device communication to ensure that your app meets the highest security standards.
Pro Tip: Regularly update your app’s security protocols to keep up with emerging threats and regulations.
8. Usability Testing: Ensuring a Seamless User Experience
Usability testing focuses on how user-friendly your app is, evaluating its ease of navigation, intuitive design, and overall user experience.
The Challenge
User Expectations: As mobile users grow more tech-savvy, they expect apps to be easy to use and navigate.
Cross-Platform Usability: Usability can differ across iOS and Android devices due to interface design differences.
The Solution
Conduct usability testing on real devices to gather feedback from real users. Cloud-based platforms like Pcloudy allow for real-device usability testing, providing insights into the app’s user experience across different devices and operating systems.
9. Battery Usage: Avoiding Power-Hungry Apps
Battery consumption is a critical factor that can impact app usage and customer retention. An app that drains battery quickly is likely to be uninstalled by users.
The Challenge
Performance Optimization: Apps that use GPS, background processes, or frequent notifications can quickly drain battery power.
Device-Specific Impact: Battery usage can vary across devices, especially those with older hardware.
The Solution
Test for battery efficiency on various devices using real-device cloud testing environments like Pcloudy. Analyze how your app consumes battery power and optimize where needed to ensure it runs smoothly without excessive battery drain.
10. Memory Leaks: Preserving Device Performance
Memory leaks occur when an app uses excessive memory, causing performance issues like slowdowns or crashes. This can lead to a poor user experience, especially on devices with limited resources.
The Challenge
Resource Management: Apps need to manage memory effectively to avoid crashing or slowing down the device.
Device-Specific Issues: Memory management can vary depending on the device’s hardware.
The Solution
Implement memory profiling tools during your app’s development and testing phases. Regularly test your app on different devices using Pcloudy to identify and fix memory leaks.
11. Geolocation Testing: Apps that Depend on Location
For apps that rely on geolocation features, such as navigation or ride-hailing apps, ensuring that location services work across different regions is crucial.
The Challenge
Location Variability: GPS performance can vary based on the user’s location and the accuracy of their device’s GPS hardware.
Testing Across Regions: Simulating different geolocation scenarios can be difficult without access to real devices in those regions.
The Solution
Use cloud platforms like Pcloudy to simulate geolocation testing on real devices in different geographic regions. This ensures your app’s location services work accurately across the globe.
12. App Localization: Adapting for Global Markets
Apps often need to be localized to different languages, currencies, and cultural contexts. Ensuring proper localization is essential for expanding into global markets.
The Challenge
Text Expansion: Some languages, like German or Russian, take up more space than English, which can break layouts or text boxes.
Cultural Sensitivity: Localization isn’t just about language—it’s also about ensuring that the app’s design and functionality make sense in the target culture.
The Solution
Conduct thorough localization testing, focusing on the user interface, translations, and regional features. Pcloudy allows testing in real-world scenarios for apps localized into multiple languages and regions.
13. Accessibility Testing: Meeting User Needs
Accessibility testing ensures that your app is usable by people with disabilities, such as visual or hearing impairments. Ensuring your app meets accessibility standards is vital for inclusivity and can be a regulatory requirement in many regions.
The Challenge
Regulatory Compliance: Many countries have strict accessibility regulations, such as the Americans with Disabilities Act (ADA) in the U.S. or the Accessibility for Ontarians with Disabilities Act (AODA) in Canada.
Wide Range of Disabilities: Apps must be tested for a range of disabilities, including vision impairments, hearing impairments, and physical disabilities.
The Solution
Use accessibility testing tools to check your app’s compatibility with screen readers, voice commands, and other assistive technologies. Test your app on different devices using Pcloudy to ensure it meets accessibility guidelines.
Interruption testing evaluates how well an app handles interruptions like phone calls, text messages, or low battery alerts. These interruptions are common during real-world app usage.
The Challenge
App Stability: Apps must be able to handle interruptions without crashing or losing user progress.
Consistent Experience: Interruption handling should be seamless across different devices and operating systems.
The Solution
Perform interruption testing on real devices to evaluate how your app reacts to common disruptions. Cloud platforms like Pcloudy allow testers to replicate interruptions during active app sessions, ensuring smooth recovery and minimal disruption.
15. App Store Compliance: Ensuring Successful Submissions
Each app store (Google Play, Apple App Store) has specific guidelines for app submission. Failing to comply with these guidelines can result in rejection, delaying your app’s release.
The Challenge
Guideline Variations: App store guidelines differ between platforms, and ensuring compliance with both can be time-consuming.
Performance Criteria: Stores often have performance benchmarks that apps must meet to be approved.
The Solution
Before submitting your app, ensure it meets all necessary guidelines. Test your app’s experience, security, and overall quality on multiple devices and operating systems using Pcloudy to minimize the risk of rejection.
Conclusion: A Holistic Testing Strategy
Mobile app testing involves overcoming a wide range of challenges, from ensuring compatibility across numerous devices to handling network variability, memory leaks, and accessibility. A successful testing strategy combines real-device testing, cloud-based automation, AI-driven test automation, and comprehensive security testing to deliver a high-quality app experience.
Pcloudy provides a robust cloud-based platform for mobile app testing, offering access to real devices, network simulation, and AI-powered automation. By adopting a well-rounded approach to testing, you can ensure
Android 11 is scheduled to be released this year in September, but we will get the first Android 11 beta version on June 3, 2020. The new Android version contains some new features and important developer focused updates, as seen in the developer previews. Android 11 features include airdrop-style file sharing, motion sense gesture, screen recording and many more. The behavior changes in the latest Android version may affect your app regardless of the target SDK version. So let’s get familiar with the behavior changes that will help you make your app compatible with Android version 11.
Accessibility
In the new Android, your accessibility service cannot declare an association with the system’s accessibility button at runtime. You can declare your accessibility service’s association with the accessibility button using the flagRequestAccessibilityButton flag in your accessibility service metadata file, typically res/raw/accessibilityservice.xml. Earlier we used to add AccessibilityServiceInfo.FLAG_REQUEST_ACCESSIBILITY_BUTTON to the flags property of an AccessibilityServiceInfo object. But now the framework won’t pass accessibility button callback events to your service.
Open Mobile API changes
In Android 11, Open Mobile API has additional functionality:
1. Parse rules for carrier privileges.
2. Customize embedded Secure Element (eSE) access or provision an eSE using one or more of the following:
3. Provide readers a clear indicator for applications to filter device capability.
Security
The new Android stores each user’s app usage stats in credential encrypted storage. Therefore, neither the system nor any apps can access that data unless isUserUnlocked() returns true, which occurs if the user unlocks their device for the first time after a system startup or the user switches to their account on the device. If your app already binds to an instance of UsageStatsManager, check that you call methods on this object after the user unlocks their device. Otherwise, the API will return null or empty values.
Android 11 uses the Scudo Hardened Allocator internally to service heap allocations. Scudo is capable of detecting and mitigating some types of memory safety violations. Refer to the Scudo troubleshooting Android documentation if you see any Scudo-related crashes (for example, Scudo ERROR:) in native crash reports. Also, Android’s default SSLSocket implementation is internally built on top of ConscryptSSLEngine.
App Compatibility
Android 11 update includes lists of restricted non-SDK interfaces based on collaboration with Android developers and the latest internal testing. Whenever possible, we make sure that public alternatives are available before we restrict non-SDK interfaces.
If your app does not target Android 11, some of these changes might not immediately affect you. However, while you can currently use non-SDK interfaces that are part of the greylist (depending on your app’s target API level), using any non-SDK method or field always carries a high risk of breaking your app.
If you are unsure if your app uses non-SDK interfaces, you can test your app to find out. If your app relies on non-SDK interfaces, you should begin planning a migration to SDK alternatives. Nevertheless, we understand that some apps have valid use cases for using non-SDK interfaces. If you cannot find an alternative to using a non-SDK interface for a feature in your app, you should request a new public API. To learn more about the changes in this release of Android, refer to Updates to non-SDK interface restrictions in Android 11.
Debugging
Android 11 offers debugging support for apps to identify potential JobScheduler API invocations that have exceeded certain rate limits. Developers can use this facility to identify potential performance issues. For apps with the debuggable manifest attribute set to true, JobScheduler API invocations beyond the rate limits will return RESULT_FAILURE. Limits are set such that legitimate use cases should not be affected.
The default mode for fdsan is changing in Android 11. It detects mishandling of file descriptor ownership, such as double-close and use-after-close. Previously fdsan used to log a warning and continue but now it aborts upon detecting an error. If you’re seeing crashes due to fdsan in your application, refer to the fdsan documentation.
Privacy Changes
There are some crucial changes in the privacy control in Android 11 and developers should review the privacy features and test their apps for compatibility. The first change we will talk about is One-time permissions. Now users can grant temporary access to location, microphone, and camera through a one-time permission. This feature grants more control to the users who do not wish to share information while using any app that uses these features. The next change is in package visibility. Android 11 changes how apps query and interact with other installed apps on the same device. Add the element to your app’s manifest and this change will affect the apps that target Android 11. There is an update for the background location access which will affect the apps that need all time access to location. Request foreground (coarse or fine) and background location permissions incrementally in separate calls to the permission request method. Before each request, use a full screen view to explain the benefits that users receive for granting that permission.
Apart from this there are many UI changes like system_alert_window permission and manage_overlay permission. For more information you can visit the Android documentation and also check out the behaviour changes in apps targeting Android 11.
Good news for Android users!!! The wait for the new version of the Android is over as it is likely to be released in the first week of September. Yes, you read it right.
Finally, Google has ditched its sweet flavours that started with Android 1.5 naming as “Cupcake”. After that, the trend followed as — Donut, Eclair, Froyo, Gingerbread, Honeycomb, Ice Cream, Jellybean, KitKat, Lollipop, Marshmallow, Nougat, Oreo, and Pie.
For the first time in its history, Android released its OS without a dessert name. It has discontinued the tradition permanently. It is simply called Android Q or Android 10.
However, the previous version Android 9 or Pie was most focused to introduce the best of intelligence, simplicity, digital wellbeing and ultimate navigation.
The new version Android Q is filled with functions like Android Q desktop mode offering more control to apps and features to its users. Fans and tech junkies had lots of expectations from Android 10 as it comes with lots of interesting and innovative features.
Android Q Features
Let us look at major changes in the Android 10.
Fully Gesture-based Navigation
Putting forward the gesture-based navigation added to the Android Pie, the new version will have fully-gesture based navigation. The traditional back button of the Android OS will not be available. It’s one of the remarkable changes that users will find in the Android OS navigation.
Some of the gestures are:
Swipe up to access home;
Swipe up and hold to show the multitasking menu,
Swipe from the left or right edge of your screen to go back.
First, swipe to open a side menu and a double swipe to go back.
However, user can anytime switch between various system navigation options that include: Gesture navigation, 2-button navigation, and the traditional 3-button navigation.
System-Wide Dark Mode
The feature which every Android user was waiting for- the Dark mode. Google’s Android 10 will have a system-wide dark theme that can be turned on and off at any time via a Quick Settings toggle.
Adding to that, Google has also created APIs that third-party app developers can use to activate the dark theme mode within their apps.
Live caption transcription
Live Caption will provide real-time captions for just about anything on your phone where someone is talking or reading. The best part is that it happens locally on-device. In other words, no internet connection of any kind is required. Anyone can turn Live Caption on through Android 10’s accessibility settings.
This will be a godsent gift for users who have difficulty in hearing or partially deaf. They generally face difficulty to understand the media contents like video, games, podcasts and so on, they can turn on the “Live caption” feature to better understand the media being displayed.
Privacy
Privacy is the biggest concern for our generation. New steps have been taken by the Android 10 developers to give us a better understanding and more control over what exactly apps on your phone have access to like calendar, location, camera, contacts, and microphone.
Now the apps that ask for your location will now show a new pop-up asking you if you want to grant location access all the time, only when the app’s being used, or not at all. This is a major step where the user has more command over third-party apps.
Permission Usage
The Permissions Usage page in settings has been completely modified to show which permissions are being used by how many of your apps, the ability to filter by permissions to see which apps are using certain ones, and a new User Interface for the App info page.
Sharing Shortcut
The direct share is replaced by sharing shortcut. It allows apps to return lists of direct targets for sharing for use within share menus. Unlike its previous versions, apps publish their targets in advance and do not have to be polled at runtime, improving performance.
Wi-Fi QR Code
Using a password to connect to a Wi-Fi will no longer be required. Android 10 has a built-in feature called “Easy connect”, where the system lets us make a QR code that others can access to use your Wi-Fi.
Bubbles
Bubbles are just like you see chat heads in facebook messenger. It will be a circular floating head that shows the latest notification on the screen. The new notification bubbles will now pop up on your screens while using other applications. The bubble is created for multitasking purposes; however, the user has an option to disable it.
Audio Features
Android 10 lets us record or listen to audio on two different apps. Previously, if an app started to capture audio, all other apps had no way to access the audio input until the capturing app was stopped. But now, you will be able to get two applications to listen to audio input at the same time.
Settings Panel
User need not jump through multiple pages to get to the phone settings you want. Settings Panel on the screen let you access phone settings that are contextual and correlated to the app that users currently have open.
Foldable Devices
There is a high possibility that foldable smartphones will be introduced in the market by the end of the year. Android 10 will offer support for application developers to develop apps specifically for such devices, even with different folding patterns. The support addresses the app’s resizing, screen ratios, multi-window functions and aspect ratios.
Undo App Removal
This feature can be saviour for users, as it happens to most of us, we accidentally remove any app and then we realize, we shouldn’t do that.
Google has understand this issue and has given to undo the removal. After you have removed any app, you have a few seconds to undo it by pressing the “undo” button that you can find within the bottom of the screen.
Smart Battery Indicator
The new indicator will reveal the remaining battery capacity. Currently, we see a certain percentage showing on the screen. Now the OS will show a text like “Until 10 PM” instead of a percentage. This text will only appear when the battery decreases to a certain level.
Android 10 will give extra privacy to the smartphone users and with the help of innovative technologies, the Android OS users will be able to function more efficiently. All these features will surely make things easier and navigable.
When to Expect the Release
Google had released the first Android Q Beta version in early March this year for the Pixel phones and multiple Beta versions were released including Android Q Beta 6 which appeared on August 7. As of now the announcement of the Q3 has been made on August 22, so we can expect the final release of the Android 10 at any point of time when Q3 is over.
These Android Q features of the Beta versions are live on 20 different phones across multiple manufacturers. People can sign up and enroll to get the new updates on Android 10.
The current compatible Phone after launch with Android 10 is – Pixel 3, Pixel 3XL, Pixel 2, Pixel 2 XL, Pixel, Pixel XL, OnePlus 6T, Sony Xperia Z3, and LG G8, Nokia 7.1, Nokia 8.1.
We are committed to keeping you ahead of others. pCloudy is one of the fastest to release the support of Android Q beta devices on the cloud.
Google has launched the final Android Q beta and Android Q official release is just a few weeks away. Testers and developers can test their app against this final beta version. Android Q beta 6 includes the final API 29 SDK and updated build tools for Android Studio. It also supports all the features, developer APIs and system behaviors that will be in the final release.
Now you can test your mobile apps on devices loaded with Android Q beta version in pCloudy.
Mobile app automation testing has evolved as a crucial aspect of the mobile app development process to help deliver better quality solutions, under controlled time cycles and cost schedules. But for delivering bug-free app, choosing the best suitable automation testing framework for your app is very important. There are many automation testing frameworks available in the market with exceptional capacities. This blog is all about Appium vs Espresso and we will analyze which of these two most widely used Automation testing frameworks is preferable for your app testing.
Espresso was not preferred because of its flakiness and instability issues. But, from the time Google has brought Android Test Orchestrator, a Gradle test option, instability and unreliability of Android Espresso tests have vanished. This, in turn, is creating a serious problem for the most popular automation framework Appium.
Let’s find out in this blog if Espresso now comes with a power to kill Appium or Appium can hold its stand in this fiercely competitive market.
Let’s get into the details.
What is Appium?
It is an open source, cross-platform mobile app automation testing framework. Appium allows native, hybrid and web app testing and supports automation test on physical devices as well as emulators or simulators. The Appium server uses selenium web driver which permits platform independence and allows the user to use the same code for Android or iOS.
Advantages of using Appium
Facilitates test execution without server machines
Appium is developed using cross-platform runtime environment like NodeJs which enables programmers to write server-side code in javascript. It is designed as an HTTP server and you can run the test without requiring a server machine.
Does not require app code recompilation
Most of the automation testing tools require testers to alter app code. Some of the test automation frameworks require testing professionals to recompile the code according to the targeted mobile platforms. Appium enables testers to evaluate both cross-platform and native apps without recompiling and altering the code that often.
Automates various types of mobile apps
Testers can avoid using different automation tools for different types of apps as Appium can be used for web apps, hybrid, and native apps too. It facilitates the testing of hybrid and mobile web apps as a cross-platform test automation framework. At the same time, it enables testers to test native apps through web driver protocol.
Testers can use real devices, emulators, and simulators
Testers use real devices to evaluate mobile app’s usability and user experience more precisely. Although, to speed up the mobile app testing one needs to use emulators or simulators too. Appium helps testers to produce reliable test results and reduce testing time by supporting real devices, emulators and simulators.
Provides a record and playback tool
In Appium, testers can use the inspector to accelerate testing through record and playback functionality. Appium inspector can record the behavior of native apps by inspecting their document object model (DOM). Record and playback tool can produce test scripts in a number of programming languages.
Testers can automate apps without adding extra components
Testers can execute the same test across multiple mobile platforms without putting extra time and efforts or adding extra component. Appium simplifies automation by keeping complexities in Appium server.
Supports several web driver compatible languages
You can integrate Appium with many testing frameworks and WebDriver – compatible languages including PHP, Java, Ruby, Javascript, C# and Objective C. Hence, a tester has the option to write test scripts in his preferred programming language.
Disadvantages of using Appium
Common gestures
Appium lacks commonly used gestures like double-clicking in java-client libraries. It also does not support Android alert handling directly and the users cannot evaluate alert handling through native API. Testers have to put extra time and effort to test these gestures.
No script execution on multiple iOS simulators
Simulators make it easier for testers to mimic internal behavior of the underlying iOS devices. Although Appium does not allow users to run multiple test scripts on multiple simulators simultaneously.
Lacks the capability to recognize images
Appium cannot locate and recognize images automatically to evaluate games and apps precisely. The testers have to take help of screen coordinates to make Appium locate and recognize images.
Does not support older versions of android
Appium supports only Android 4.2 and later and does not supports older APIs for Android. There are still many people using devices which run on older versions of Android and developers find it difficult to test mobile apps developed targeting older Android API level.
What is Espresso?
Espresso is a tool developed by Google which is used for testing the UI of Android apps. It automatically synchronizes your test actions with the user interface of the mobile app and ensures that the activity is started before the tests run.
Although when you execute an Espresso test you will have shared state in separate tests and some flakiness. For this Google came up with a solution. Android Test Orchestrator is a Gradle test option that helps in testing and increases the reliability of our automated test suites.
If you use Gradle build tools in any version of Android Studio below 3.0 then you also have to update the dependency setup. Let’s take a look at the advantages of using Android Espresso.
Advantages of using Espresso
Integration with Gradle
The new Android Espresso now has the power of the Android Studio and Gradle that comes along with it. So now invoking your tests, running it or modifying it is just a matter of calling a Gradle command. This gives the full power of command line to the developer and makes testability much easier.
Test Orchestrator
The new Android Espresso comes with the power of Android Test Orchestrator that allows you to run each of your app’s tests within its own invocation of Instrumentor. It ensures that there is minimum shared state and crashes being isolated. It allows you to filter the tests that you want to run and also distribute tests across devices. This implies that you have finer control over how your tests run.
Less flakiness
The scalability of the test cycle in Android Espresso is high due to the synchronized method of execution. A built-in mechanism in Espresso that validates that the object is actually displayed on the screen. This saves test execution from breaking when confronted with “Objects not detected” and other errors.
It’s easy to develop Espresso test automation
Test automation is based on Java and JUnit which Android developers are familiar with. There is no setup or ramping up to implement quality in the in-cycle stage of the app SDLC.
Reliable and fast feedback
Android Espresso does not need any server to communicate with, instead, it runs side by side with the app and delivers fast results. It gives fast feedback to the code changes so that developers can move to the next bug fix.
Simple workflow
Espresso allows developers to build a test suite as a stand-alone APK that can be installed on the target mobile alongside the app under test and be executed quickly.
Disadvantages of using Espresso
It requires access to the application source code
Without the source code, you won’t be able to do anything. Also, There is a risk to get used to the in-built test synchronization and UI – then it might be hard to work with WebDriver.
Narrow focus
If UI tests are required for both Android and iOS, it will be necessary to write twice, for two different systems. If tests require to work with Android outside the application (for example, open a received notification with a text message), you’ll have to use additional tools, such as UIAutomator.
Knowledge of launching Android app on emulators required
It is desirable to have at least minimal experience of building and launching Android applications on emulators.
Conclusion
Appium and Espresso both can be used to perform UI testing on Android app but if you have to choose one of them then you need to decide on the bases of your requirements. What kind of app is it and what kind of testing you want to perform. Developers who want to perform UI testing for their native Android app should go for Android Espresso. Although, if the test needs to support iOS and Android both and you want to test at a functional level then you can use Appium.
The second blog in the series “Start to end guide for mobile app testing”
Basics and Comparison
In the previous blog in this series, we talked about the evolution of mobile technology. In this blog, you will know more about the two most popular mobile operating systems, Android and iOS. Here you can also learn about the Android architecture, concepts of Android SDK, emulators and iOS architecture and mobile cloud. Let’s start by getting familiar with Android versions.
What is Android?
Android is a software bunch comprising not just the operating system but also middleware and key applications. It is developed by Google and later by the Open Handset Alliance but it is not limited to only mobiles. In other words, it is a complete set of software required for the development of smart devices such as smartphones, tablets, notebooks, set-top boxes, TVs, smart watches, etc. Android is a Linux based open source software platform. The application development in Android is done in the Java language.
Versions of Android
The first version of Android was launched on the HTC Dream mobile in the year 2008. Since then Android has been evolving constantly and now it has the largest user base of around 88% global market share. Android OS versions are released with a name following the alphabetical order, such as Android 1.1, 1.5-Cupcake, 1.6-Donut, 2.0/2.1-Eclair,2.2-Froyo,2.3-Gingerbread,3.X-Honeycomb,4.0-Ice Cream Sandwich, 4.1/4.2/4.3-Jelly Bean, 4.4-KitKat, 5.0-Lollipop, 6.0-Marshmallow, 7.0-Nougat, 8.0-Oreo and 9.0- Pie being the latest of all the versions.
Why so buzz about Android?
Whenever we hear the word Android, we usually think about the ‘smart phones’. This is how Android is placed in our minds. It is one of the most successful mobile operating systems in the market today. Android apps are the most downloaded apps in the app stores. It runs on millions of mobile devices in more than 190 countries in the world. Around 1.5 billion apps and games are downloaded from Google play store in a month. It does not fail to impress its users by consistently introducing new features. It is open source, so any android variant can be developed using the source code. It supports wireless communication including 3G, 4G, WiFi, and Bluetooth. Android keeps introducing its new and upgraded versions, often. Due to its popularity, around 1million new Android devices are activated worldwide in a day. Google play is an open marketplace for developers to sell and distribute their mobile apps. It has already entered the field of Artificial Intelligence enabling the apps to be more intuitive and user-friendly.
Android Architecture
Android is architected in the form of a software stack comprising applications, an operating system, run-time environment, middleware, services, and libraries. The following figure is the visual outline of the elements integrated layer by layer. These all elements are the prerequisites of the mobile app development and to make the app environment ready. Android Architecture is categorized as Linux kernel, native libraries(middleware), Android runtime, Application framework, and applications.
Linux Kernel- It exists at the root of the Android architecture. It contains all drivers for hardware components, battery and memory management, resource access and device management. Android only uses the Linux Kernel.
Libraries – It is the layer above the Linux Kernel, including native libraries such as WebKit, OpenGL, FreeType, SQLite, Media framework, C runtime library(Libc), etc. Webkit library supports the web browsing engine, SQLite is used for sharing and storing application data, Media to play record audio/video, etc. FreeType is for processing fonts, SSL libraries are for internet security, OpenGL and SGL are responsible for rendering 3D,2D graphics, respectively, the Surface manager is responsible for rendering windows and drawing surfaces of apps on the screen. Libraries also contain C++ libraries used by android system components.
Android Runtime (ART)- these have the core libraries also known as Dalvik Libraries (DVM) which are responsible for running an android application. Android Runtime is built to run apps in a restricted environment where there is limiter power in terms of battery, processing, and memory. ART uses DEX files, a type of byte code designed for Android to manage memory more efficiently.
Android Framework- On the top of Android runtime is Android Framework. It includes a collection of Android APIs written in Java. Enables and simplifies the reuse of core components and services such as Window, view, Activity, telephony, resources, locations, Content Providers (data) and package managers. It provides access to Android feature set fir developers to build a mobile app for Android OS.
Applications- Over the Android Framework lies the application layer covering system and other apps that the users can download from the Google Play Store. The core apps like email, SMS, calendar, maps, browser, contacts, etc are pre-packed in the mobile device. This layer uses all other layers for enhancing the performance of these mobile apps.
Concepts of Android SDK and .apk file and emulators
Android SDK is a Software Development Kit which allows the developers to develop an application for the Android platform. The Android SDK comprises of software programs with the sample source codes, developer tools, documentation, tutorials, an emulator and essential libraries to build, test and debug mobile apps for Android. Apps are written in Java language and are run on Dalvik(DVM) that runs on Linux Kernel.
APK stands for Android Application Package. It is a package file format used by Android OS for distribution and installation of mobile apps and middleware. For installing any mobile app/games, we require APK files with an extension .apk. These can be downloaded from the play store. Apk files are just like .exe files for windows. Apk file is in zip format and contains all necessary files required for app installation. The Apk archive usually contains META-INF directory:
MANIFEST.MF: the Manifest file
CERT.RSA: The certificate of the application.
CERT.SF
Android Emulators-
Android emulator or Android Virtual Device (AVD) is a device that is a functional replica of an Android device that can be used to run and test the Android applications on the PC even before they are published in the market for final use. Android emulator comes as part of the Android SDK. It is a virtual device that lets the developer develop the apps without using a physical device. Android emulator requires JRE –Java Runtime Environment and Android SDK to function. The applications can be either downloaded or installed directly on the device from the Google play store or if the application is available in ‘.apk’ format, it can be installed using the “add” command.
iOS and its versions
iOS is a mobile operating system developed by Apple Inc. It was originated in 2007 for iPhone and later extended its support to other Apple devices like iPad and iPod touch. It is the second most popular mobile device in the world after Android. The iOS mobile apps can be downloaded from Apple’s App Store. The App store contains more than 2 million iOS apps today. The iOS apps are programmed in Objective C, C, and C++ languages. Version updates for iOS are released through iTunes software until the introduction of iOS 5 in 2011. Now, the software updates and data sync can happen wirelessly through Apple iCloud service. iOS has expanded its market by introducing new products powered by Apple like iWatch and AppleTV.
It was formerly known as iPhone OS and the name was used for its other 3 subsequent versions until 2010 when Apple released iOS4. In 2011, iOS5 was released providing access to around 500000 iOS apps and some additional features. iOS 6, 7, 9 were released in the succeeding years with more advanced features and performance. The latest versions iOS 10,11 and 12 are released in 2017 and 2018 respectively.
iOS Architecture
iOS Architecture is also a layered one. Each layer is built with a variety of frameworks which can be assimilated in the iOS apps. The layers communicate with the hardware with the help of clearly described system interfaces that make it easy for the developer to build the app that is ready for different devices. Let us discuss each layer below:
Core OS- This layer is the foundation layer of the OS on which other layers are dependent. This layer is responsible for managing memory, system and OS tasks, networking and also interacts directly with the hardware. This layer comprises of frameworks like accelerate, external accessory, core Bluetooth, security and local authentication.
Core Services Layer- It consists of technologies that provide certain services to the app but are not directly related to the UI of the app. It contains high-level features like iCloud storage. The core services include address book framework(provides access to contacts and user database), CloudKit (medium of transferring the data between app and cloud), Core Data (to manage the data model of a model view controller app), Core Foundation( Technologies to provide Data management services to IOS), Core Location(gives location info to apps),Core Motion(ton access motion-based info on the device),Foundation(Using Objective C), Healthkit (handles health-related info of the user),Homekit(controlling connected devices of the user at home),Social( to access user’s social media accounts) and Storekit Framework( supports in making in-app purchases from iOS apps).
Media Layer- Media layer in iOS architecture enables the Graphics, Audio, Video technologies. Graphic Technologies like UIKit Graphics, Core Graphics framework, Core Animation, Core Images, OpenGl ES which handles 2D vector and animating views and 2D and 3D figures, GLKit and Metal. Audio Framework supports rich Audio experience and includes- Media Player Framework, AV Foundation, OpenAL.
Video Framework includes AV Kit, AV Foundation, Core Media, Also the iOS support
for the playback of movie files with the .mov, .mp4, .m4v. and .3gp filename
extensions.
Cocoa Touch Layer – The layer defines the basic application and support for key technologies such as multitasking, touch-based input, push notifications, and many high-level system services. It includes EventKit, GameKit, iAd, MapKit, PushKit, Twitter and UIKit frameworks.
IPA stands for iOS App Store Package. Any file with .ipa extension is an iOS application. It is an archive like ZIP that contains software sets used to develop the iOS app. Each .ipk file can be opened with Apple’s iTunes program. An IPA file has a binary for ARM architecture and can only be installed on an iOS device. IPA files cannot be installed on the iPhone Simulator. To run applications on the simulator, original project files which can be opened using the Xcode SDK are required.
iOS Simulators – These are again programs to test and run the iOS applications without having any physical or the ‘real’ device. The iOS Simulator allows you to rapidly prototype and test builds of your app during the development process. Installed as part of the Xcode tools along with the iOS SDK, iOS Simulator runs on Mac and behaves like a standard Mac app while simulating an iPhone or the iPad environment. iOS simulators require MAC Environment and Xcode to function. To start the iOS simulator, firstly launch the Xcode and then do one of the following:
1. Choose Xcode > Open Developer Tool > iOS Simulator.
2. Control-click the Xcode icon in the Dock, and choose Open Developer Tool > iOS
Conclusion
Operating systems are being revamped using AI and connectivity to the Internet of Things. These technologies are still evolving and both Android community and Apple are trying to lead the way by enhancing the user experience. Android had an upper hand in the past as it has a very active open community to support the development. Although, Apple in the recent past has taken a new approach in getting ahead with technology for e.g. Air Pods and their own coding language Swift. We can be sure that both Android and iOS will be more convenient and interactive as both Google and Apple are the torch bearers of the future.
In the next blog, we will talk about the types of mobile applications.
AndroidX is an improved version of the android support libraries that the android team uses to develop, test, package, version and release libraries within the jetpack. AndroidX fully replaces the support library by providing feature parity and new libraries. In addition, AndroidX includes the following features:
All packages in AndroidX are in consistent namespace starting with the string AndroidX. The support library packages have been mapped into androidx.* packages. For a full mapping of all the old classes and built artifacts to the new ones.
Unlike the support libraries, AndroidX packages are separately maintained and updated. The AndroidX uses strict semantic versioning.
All new android development will occur in the AndroidX library. This includes maintenance of the original support library artifacts and introduction of new jetpack components.
Android Jetpack
Android jetpack is a set of components and tools along with architecture guidance designed to help you accelerate your android development. It gives a template to write production ready android code. Jetpack is made up of components in four categories, foundation architecture behaviour and UI. Each component is individually adaptable and build to maintain backwards compatibility. Android architecture components are very modular, so we are allowed to choose what feature sets we want that are compatible to our app.
Espresso is now a part of the AndroidX family
Espresso is a testing framework designed to provide a fluent API for writing concise and reliable UI test. Writing reliable UI test is difficult as user interfaces are asynchronous driven by events, transitions and data loaded from background threats. Coding around that without any help from UI testing framework would require a lot of boilerplate. Espresso takes care of any UI events, so that in most cases you don’t have to worry about any view state transition and implementation details. The basic UI test flow when using Espresso includes:
View Matchers: To find view in the current view hierarchy for e.g. to find UI elements like buttons, textbox etc.
View Action: To perform action on the view, e.g. to click on a button, double click, scrolling etc.
View Assertions: Allows to assert state of a view.
Application of Espresso test recorder
Allows us to create effective UI test cases with user interactions.
We can capture assertions and interactions without accessing app structure directly which increases execution speed and optimizes test case.
Saves time searching for locators and then writing the test cases.
It supports multiple assertions making more reliable test cases.
Pcloudy supports androidX instrumentation with Espresso
Now you can write test cases in espresso and test the APIs in pCloudy using androidX Junit instrumentation. Here are the steps for running your Test scripts on multiple android devices:
Login over https://device.pcloudy.com with your registered Email ID & Password.
To schedule “Espresso” over pCloudy, follow the below mentioned steps-
Go to the “Automation” page.
Select the Automation tool as “Espresso”.
Select “Instrumentation Type” based on your Test Scripts you’ve written.
Note: pCloudy provides support for Instrumentation Type (InstrumentationTestRunner, AndroidJUnitRunner and AndroidXJUnitRunner) for Android.
Select the Application APK and Test APK that you must have uploaded in the MY APP/DATA section.
Select the single device execution time and assign a name to your test cycle.
In the next step, Click on “ADD” to add the device for testing and click on “
Next”.
Click on “Schedule” to start the test.
Go to your mailbox and open pCloudy Automation Alert mail.
Click on the given link “Click to view Report”.
Now you have the result of your scheduled test automation.
How to migrate to AndroidX?
To migrate from support libraries to AndroidX the Google has provided a refractor tool in Android Studio. Projects can be migrated to AndroidX by clicking on ‘Refractor’ in the menu bar and then clicking on ‘Refactor to AndroidX’. Then it will search for the usage and show the result. To refactor click ‘Do Refactor’.
pCloudy is leading the way in the field of automated mobile testing solutions.
It is a new publishing format by Google which is a more efficient way to develop and release app. App bundle helps to reduce your app size and deliver features on demand. Earlier, android operating system used android packaging kit (APK) to distribute and install applications on a device. These applications are downloaded by users across the world on various devices. These devices have different configurations and language inputs. To meet all the users demands, the application becomes bulky as all the features are to be downloaded.
Android App Bundle is a zip archive with .aab extension. It contains codes and resources for all the devices that the app supports. Google Play handles signing and generation, once it is uploaded for publishing. In app bundle, dynamic delivery is used to generate an optimized APK for users, based on their device configuration.
Benefits of .aab
The key benefit of android app bundle is that it the developers need to write less code to push the app in Play store. The users save space in their device by saving a small size APK. App bundles can use uncompressed native libraries in android 6.0 and up, that are stored in the APK instead of the users device. This lowers the download size and the size on disk. It serves users with functions they need on demand, instead of installing all the functions at one go. We don’t need to build and publish multiple APKs, therefore, app bundle also simplify the built and release management.
How Android App Bundle works
Android delivers APKs with the required resources using split APK mechanism. Google Play uses this mechanism to split large apps into smaller APKs, as per the device requirements.
According the Google, there are 3 types of APKs:
a) Base APK: This is the first mandatory APK to be installed. It contains the basic requirements for the application. This APK contains codes and resources that other split APKs can provide. Only the base APK’s contains full declaration of your app’s services, permissions, platform version providers and dependencies of system features. It is important that all codes and resources included in this module are included in the base APK.
b) Configuration APK: It contains specific data, based on the device requirements. Configuration APK is generated by Google Play from the app bundle that is uploaded to the store. Each of these APKs includes native libraries and resources for a specific screen density, CPU architecture or language. When a user downloads the app, their device downloads only the specific APKs for that device. You don’t create separate module for configuration APKs. If you use standard practices to organize alternative, configuration specific resources for your base and dynamic modules, Google Play automatically generates configuration APKs for you.
c) Dynamic Feature APK: These are the optional features installed required by the user. Each of these APKs contains code and resources for a feature of your app that is not needed when your app is first installed. Using the play core library, dynamic APKs may be installed on demand after the base APK is installed on the device to provide additional functionality.
Android Application Bundle Format
An Android App Bundle is a file with .aab extension which you can upload to Google Play to support dynamic delivery. App bundles are signed binaries that organize your apps resources into modules. Each of these modules may be generated as separate APKs. Google Play uses the app bundles to generate various APKs that are served to users.
App Bundle’s files and directories: Base/, feature 1/ and feature 2/: Top level folders that contain different modules of your app. The base directory contains base module of the app. The directory for dynamic feature module is given the name specified by the split attribute in the module’s manifest.
Bundle-Metadata: Metadata files include complete list of the app’s DEX files and Proguard Mappings. Files in this directory are not packed into the app’s APKs.
Module Protocol Buffer files (*.pb): Provides metadata that describe the content of each module to the play store. For example, native.pb and resource.pb describe the code and resources in each module, which is used when Google Play optimizes APKs for different device configurations.
Manifest/,DEX/: Unlike APKs, app bundles stores the androidmanifest.xml and DEX files for each module in a separate directory.
res/, libs/and assets/: These directories are used in the same ways as APK, except that for an app bundle, they are used by Google Play to package only the files that satisfy the target device configuration.
root/: This directory stores files that are later relocated to the root of any APK including corresponding module.
How to deploy App Bundle
Unlike APKs, App Bundles cannot be installed on a device. It is an uploaded format which contains compiled code and resources in a single build framework. Once we upload out signed app bundle, Google Play builds and signs the apps APKs and serve them to users through dynamic delivery.
Testing your app bundle with Google Play Internal Test Track
You need to generate signed in app bundle before you can upload your app bundle to the play console. Proceed with these steps to generate a signed app bundle.
Select Build then select Generate Signed Bundle/APK from the menu bar. In the Generate Signed Bundle/APK dialogue, select Android app bundle and click on Next.
In the Module dropdown menu, select the base module for the app you want to generate app bundle for.
Provide information for an existing key and keystore, or create a new. This is the same type of key and keystore information you provide when building a signed APK.
I you want Android Studio to also save your signing key as an encrypted file, check the box next to Export encrypted key. To be able to upload your app bundle and take advantage of dynamic delivery, you need to upload this encrypted file to the play console and enrol in app signing by Google Play.
Click Next and provide a Destination Folder for your app bundle. Select the Build Type and flavours that you want to generate app bundles for.
Click Finish.
Now you have generated a signed bundle, you can upload your app bundle to the play console.
Testing your .aab file on pCloudy pCloudy supports .aab format and the user can upload the App Bundle instead of “.apk” to test their app on the device cloud.
Conclusion
Android Application Bundles is a big step forward in the area of application publishing and uploading. It has reduced the size of APK of your application which leads to more download of the application.
With no access to internal or private servers, complexity in the process of VPN connections, or the problems to test a software, website or service in a production-similar environment, testing your local or a private website or url on a cloud platform has always been a challenge for testing teams.
For all the users who have been asking for a solution to test their private servers on pCloudy devices before deploying them on production, we have a good news for you.
Now, you can access your site behind a firewall, on a staging server, or locally with pCloudy before it hits production.
Since private servers are internal to your network, they have no public access which makes it difficult to test on a device on cloud. Local Site emulation Testing provides a platform to test private or internal servers on any Android device present on pCloudy.
Let us see how to set up the connection for Local Site Testing:
2. Select desired device from the Device page and click on “Local Site Testing” icon present at top on right hand side as shown in below screenshot.
3. Download the Local Site Testing jar file as shown in snapshot below.
Note: Ignore if jar is already downloaded.
4. Copy the command by clicking on icon as shown in the snapshot.
5. After the file download is complete, Open your command prompt or terminal
Let us see how to open Command Prompt in different Operating system.
You can follow any of them among three mentioned ways
For Mac-OS users:
a. Open “Launch pad”: It’s the silver icon in the Dock that looks like a rocket. Type “terminal” in search bar and hit enter.
b. Hold command button and hit space bar. Pop will appear and type terminal and hit enter.
c. Open Launch pad and click the “Other” folder: It’s a square icon containing several smaller icons. Then look for terminal.
For Windows users:
a. Open Start and type in command prompt or cmd then hit enter.
b. Click the Windows logo in the bottom-left corner of the screen, or press the ⊞ Win key. You can search for Command Prompt on all supported versions of Windows.
c. Open the Run program. Hold the Win key and press the R key to open the Run window. Then enter “cmd” or “Command prompt” and press enter.
For Ubuntu users:
a. Using Keyboard Shortcuts: Press Ctrl+Alt+T This will launch the Terminal.
b. Press Alt+F2 and type gnome-terminal. This will also launch the Terminal.
c. Press Win+T (Xubuntu only). This Xubuntu-specific shortcut will also launch Terminal.
6. Execute the command on same location where the local site testing jar file was downloaded.
7. After the command is successfully executed you will able to see a success message appearing as “Local Site Testing is enabled in your system”. This confirms that now you are ready to test your private server on the device connected on pCloudy.
Note: Don’t close the command prompt until session ends.
8. Go to the connected device and using the browser open any private URL to test. That’s all you are all set.
You can now find more bugs and defects on your local sites before deploying it to production with this breakthrough feature.
Here’s testimony to it by one of our customers from TagIT mobile.
Why don’t you try it yourself and let us know your feedback in comments about how this feature is useful to you!
The much awaited bonanza for Appium users is here. We are up with a newer and simpler Appium integration architecture which can simplify your Appium test automation execution on Android and iOS devices with pCloudy. The newer architecture make developers’ life simpler with lesser changes in the code. The older architecture demanded using APIs and also changes in your code that required some level of expertise. We addressed this issue and have come up with a simpler architecture where you need to mention some desired capabilities instead of calling APIs or doing changes in the code to run the Appium scripts. And wonder what! it makes you save 50% of your app testing time.
Watch the video to know more about the new simpler and faster pCloudy Appium architecture.
Pre-requisites
Appium Script
APK or IPA file
pCloudy Account
The basic steps to replace your local Appium driver to pCloudy Appium Driver are given below:
Upload the apk/ipa from your local system to pCloudy. Check this link to know steps to upload an app for test.
Set pCloudy capabilities
pCloudy_Username: Enter the email ID with which you have registered on pCloudy. For reference, check this link
pCloudy_ApiKey: API key is important for user’s verification. You can generate the API key from Settings page on device.pcloudy.com. Check this link to get your API key.
Note: If you don’t know the Device full name then you can enter pCloudy_DeviceManufacturer and pCloudy_DeviceVersion in the code and it will automatically run command on those devices.
