7 Types Of Mobile App Testing

Suyash Dubey
By Suyash Dubey
April 9, 2019
17 min Read
Share This Article
7 Types Of Mobile App Testing

Quick Links:

Types of Testing


Since there are many players in the mobile app market, the competition is endless. To beat the heat and to keep providing an enhanced mobile app experience to the users, the app needs to be constantly updated with the changes happening and for doing this, there are different types of testing techniques. Let us have an insight into the types of testing methods.

1. Compatibility Testing

Compatibility testing is a critical QA task. It assures that a given application works as intended with selected operating systems, selected devices with different screen sizes resolutions, and internal hardware (memory size, processor speed, and button/input differences). It defines the feasible compatibility combinations of devices and interfaces for a specific testing assignment, in concurrence with the customer’s requirement.

Want to test your Mobile App?

Join pCloudy Platform!

Compatibility Factors:

There are different factors that play a very significant role in mobile compatibility testing strategy. Mobile operating systems and their supported versions, mobile devices developed by different manufacturers, different device screen sizes with different resolutions and internal hardware including different input types, processor speeds and memory size of devices.

4
Let’s develop a test strategy for compatibility testing. The first step is to identify the Operating systems and devices based on key factors described in the previous slide. Once you identified the devices based on different compatibility factors you need to set up your environment for testing. You can either set up your internal test lab or you can go for external cloud providers like pCloudy. Now, the next step is to execute your test cases on a defined set of environment. Report the defects to appear while testing and report these defects to your development team.

The main purpose of compatibility testing is to ensure whether different software is compatible with different configurations and whether the application is compatible with the client’s environment.

2. Installation Testing

Mobile phones come with different types of mobile apps like Native and Web and Hybrid. Installation testing is a type of testing that is done at the initial stage of mobile app life span or maybe it is the first impression on the users, in other words. Installation testing checks whether the mobile app installs, uninstalls and updates properly without any interruption.

Want to test your Mobile App?

Join pCloudy Platform!

Installation Testing Factors:

There are some of the factors that play a significant role in installation testing of an application. Factors like which platforms and operating systems supported by the application and how they will be distributed whether it is directly available in the app store or are going to get the installable files on mail itself or is it available on an app distribution platform and can be accessed through public URL. The user behavior regarding installation changes from person to person. For eg: the developer may install the app (By using ADB install command) and a different user might directly install it from the app store or public URL.

Important points to be noted in Installation Testing:

Some of the common checkpoints which can be converted to standard test cases are:

  • Installing the app in phone memory.
  • Installing the app in phone memory is full.
  • Installing the app on an external SD card.
  • Installing the app on an external SD card is full.
  • Check the memory space after installing. It should not occupy more than promised.
  • Check if it is releasing the space after removing

Another part of the installation and uninstallation testing is the testing of patches/updates
These are the common situation when we need to test app after updates.

Check for updates:

  • Can I download an update?
  • Can I update the app when there are multiple updates available?
  • What happens if I don’t update?

When App is upgraded:

  • Check that the user is still logged in
  • Check that data is still the same
  • Backward compatibility:-Check that database changes will not affect the current data.
  • Test the update from an older version to the current version

When OS is upgraded:

  • Check the app compatibility.
  • App updated gracefully without a crash.

3. Interruption Testing

Interrupt testing is a process to replicate abrupt (Unexpected) interrupt to the application. It is done to understand how the app behaves under certain interruptions before resuming to its original state. This can be achieved in various ways and techniques depending on the application under test.

Following are some Common Interrupts:

1. A phone call when the application is running or is in background
2. Battery removal when the application is running or is in the background, battery high, battery low, incoming call or SMS
3. Plugged in or out of charging
4. Device shut down
5. OS upgrade
6. Network loss and restoration, etc.

Some specific scenarios for an application using a network:

1. Connect to the network but remove LAN connection from the router so the device can sense wifi state on the device but cannot connect to the internet
2. Connection via VPN and VPN disconnected

Scenario for Application using services:

1. Kill service by clicking on the recent button and swiping the application right to kill app and services
2. Kill app using third-party App killer
3. Kill specific services from Settings->Manage Applications
4. Scenarios for Application Linked to account Manager
Remove account from Settings->Account Manager

Incoming and Outgoing SMS and MMS:

An application is running in your mobile and you send an SMS/MMS from another mobile to your mobile. In this situation, SMS/MMS notification should display for a while and then close and the application should continue to run.

Incoming and Outgoing calls:

You are in a login page trying to log in and you have just pressed the submit button that will take you to the application home page. This is a transition period because loading is happening.If we receive a call during this time then the application should not end abruptly or crash.

Battery Removal:

If the application is running, remove the battery of your mobile and it will shut down. After rebooting the mobile, run the application again, it should run smoothly and without a crash.

Data Cable Insertion and Removal:

When the application is running, connect the USB Cable, the system should display “USB Connection Screen” and if the user returns to the application then the application should not crash or end abruptly

Interruption Testing Strategy

Let’s prepare a test strategy to test interruption testing:
The first step in this process is to decide the critical interrupts based on your application components and architecture. For example, if your application is using GPS include common interrupts related to GPS. Let’s Include two type of interrupts:

Application specific interrupts:

Prepare your test cases and try to fit theses interruption test cases with state of the application. For example, interrupt your Network while performing payment.

Interrupt testing process:


1

List of common Interrupts:

Phone calls, Text messages, App notifications, Battery Warning, Forced updates, Voicemail, Shaking the Phone, Different Gestures, Switching between apps, Locking and unlocking the screen, Changing the orientation, Music playing while using the app, Out of memory (general performance interruptions), Data app interruptions (What Sapp, Viber, Tango), Audio interrupts from multiple sources (iPod, Media player, Other audio apps), phone in standby mode, switch the network to plane mode.

Tools to test interruption Testing:

Use tools like Joule Unit (Android), On iOS use the Energy Usage Function of Instruments, Monkey for Android to stress test your app and see the handling of the interrupts, UI Auto Monkey for iOS

Scenario

A) Once the user connects the mobile earphone jack with Nokia Lumia and starts music Player.
B) Now when the user resumes the application the cart gets empty.

4. Localization Testing

This type of testing is a technique in which we check whether the mobile app adheres to the local cultural settings, customizing the apps according to target country and language and also according to the linguistic aspects. Localization is also known as “L10N” as there are 10 characters between L & N. For eg: Chinese language-search engine offered by Baidu beats Google in China. Baidu does better than Google because it looks and feels fully native to the Chinese speaking audience.

Also, regional language speakers can’t get many games in their own language, so they turn to race games, which are easy to understand. We must have noticed in these examples, language is the common issue. Apart from Translation there are different other elements in the localization, like Usage of proper Time zone, local formats of dates, addresses and phone numbers, Design and layout to fit translated text, Converting to local requirements (such as currencies and units of measure), Adapting graphics to target markets, Modifying content to suit the tastes and consumption habits of other markets, Addressing local regulations and legal requirements.

Why should we localize?

Localization is important in order to understand the cultural and linguistic aspects of a particular region. While testing, testers keep repeating the tests to check typographical, linguistic errors, cultural appropriateness of the UI, etc. Research says:
1.56.2 percent of consumers say that the ability to obtain information in their own language is more important than price. (Common Sense Advisory, Can’t Read, Won’t Buy: Why Language Matters on Global Websites, 2006)
2.74 percent of multinational enterprises believe it is either important or most important to achieve increased revenues from global operations. (California State University at Chico, 2007.
3.95 percent of Chinese online consumers indicate a greater comfort level with websites in their language. (Forrester Research, Translation and Localization of Retail Web Sites, 2009)
4. A critical success factor for cross-border merger and acquisition deals is the ability to communicate information clearly and accurately in multiple languages. (Merrill Corporation, How to do Better Multinational M&A Deals, 2008)

Types of Localization Testing:

In the localization testing, validating whether a mobile app is capable of performing given a geographical location, in particular, is found out. It includes four major types of testing like linguistic, culture, cosmetic, and functional tests.
1. Linguistic testing– ensures that the UI text appears in the congregation with the language and is not mistranslated or misapplied. This requires language skill and knowledge of the product. Mistranslation, irrelevant usage of the language are few of the errors in this.

2. Cultural Testing– Few words may imply different meanings in different cultures. So, this has to be translated properly in relation to the target culture.

3. Cosmetic Testing– For easy understanding, let us use this scenario. For eg. In the apps used in Afghanistan, the text will be in Arabic and will start from right to left unlike other languages like English, French that are read from Left to right.

4. Functional Testing– In this testing, testers follow prescribed test scripts to run through all aspects of the product to make sure it functions as designed in a localized environment. Often functional testing includes verification that the localized product is compatible with various operating systems and third-party products. Functional testing generally requires a good knowledge of the target language and total familiarity with the product. Some of the major Functional testing problems may include– Incorrect or missing UI elements, graphics, and windows, Incorrect error messages, Localization bugs that may cause software crashes, Errors generated by installing the localized software on a localized OS.

5. Performance Testing

Performance testing is to test the performance of the mobile application in expected workload scenarios and to eliminate the performance hurdles. It checks whether the response of the app is quick, the ability of the app to tale load, and app stability in those load situations. Performance is very critical because if the app is malfunctioning, it is more possible that the user might uninstall the app and might shift to a competitor’s app which is better.

The three main pillars of Mobile app performance testing are:

Device Performance

App start-up time is the most critical performance parameter from the user’s point of view. It shouldn’t take more than 2 seconds for the app’s first screen to pop up after the user taps on the app’s icon. Battery time is the second most important issue. Some apps consume a high amount of battery life. Excessive resource usage creates a burden on the processor and the phone heats up. In some cases, it is observed that the new app installed uses the same amount of battery as the OS. If more features are added in the app its memory consumption also increases. This is why memory consumption should be checked while testing the app.

Network Performance

The app needs to be tested on different mobile networks and network properties. If there is packet loss then the app should generate alerts or resend the request for the information. Jitters or delay in receiving information

Server/API Performance

Performance is directly proportional to the response time from the Server. In this, tests are run to check how efficient the app is to handle data transfers from the server and how quickly and in what format the data is transported by the app. The number of API calls generated by apps should be lesser and how and where the data is saved in the ‘server down’ case is all that is tested here.

Mobile app testing tools make it easier for the testers to identify and rectify any error that comes up. For different OS there are a different set of Mobile app tools. For Android, we can use Robotium and Monkey Runner and for IOS we can use Automator.

6. Usability Testing

Usability testing is a type of testing which is performed to check how user-friendly the mobile application is in terms of the navigation, ease to use the app, flexible app controls, etc. It is also known as user experience testing.
What matters in Usability Testing?

Workflow: To understand, below image represents the workflow of how the user reaches its goal while navigating through the app controls.
3
We must understand what the user needs to do in order to use the app’s functionality. If the workflow was overlong and included repetitive actions It may annoy the user while performing the task.
Design and Layout: Good layout and design allow a user to easily complete tasks. An app need not offer so many features or content that the app is difficult to navigate. A friendly app has a design that allows less user text entry, helps differentiation between the selected items, finger-sized targets, should provide intuitive hints to texts, etc.

The response time of the application: It is one of the key factors. This means how long the user takes to complete a task. Often long delays while the app interacts with the server hinders the user experience standards.

The emotional state of users: Emotional state of user measures the user’s motivation to use the app. The apps should be smart enough to understand what the users want and how to engage them, this can be achieved by usability testing.

Usability Testing Strategy:

2
The first step is to identify users who match your expected audience. Watch and listen to them as they use your products. You can use video and audio recording equipment for archiving test sessions, in case later review is needed. However, our primary tool is taking notes with paper and pencil! This is still the fastest way we have found to get information quickly to developers and designers. You have your data. You now need to organize and analyze it. How the results are absorbed as a report in order to be useful for product changes.

Tools and techniques for usability testing

1. Mr. Tappy: It is a tool set up to capture how users interact with the apps naturally. The user data so generated could be used for better user experience.

2. Paper Prototyping: It is a technique that consists of creating hand drawings of user interfaces in order to enable them to be rapidly designed, simulated and tested. As simple as this technique may look like, it has been effectively used since the 1980s and is likely to continue to be used with a guaranteed degree of success for many more years to come.

3. Magitest: It allows you to perform mobile user testing of native apps and websites. Captures the participant’s voice and facial expressions during the session. Here the users complete the tasks you specify and allows to listen to them speak their thoughts aloud as they interact with your mobile app.

4. Reflector: This app allows us to see the iPad, iPod, iPhone screens on MAC or PC. The devices connect wirelessly to proceed with further functions.

7. Conformance Testing

Conformance/compliance testing is a type of testing which is performed to validate if the application is meeting the Market place and Enterprise policy guidelines. It is like an audit done to check whether certain standards are being met or not. Conformance Testing is focused on two main areas- App store guidelines and Enterprise policy compliance. Each Market place has their own guidelines. We need to validate if the application is following the guidelines of each of these app stores.

1. App store Guidelines pertaining to User Interface, Media Content, Privacy, Spyware and Phishing, nudity, religion, culture, violence, etc.

2. Enterprise Policy Compliance pertaining to a different set of industrial standards. For eg: For the pharmaceutical industry, the Food and Drug Administration (FDA) passed rule comes into the picture, For the pharmaceutical industry, the Food and Drug Administration (FDA) passed rule comes into the picture, In healthcare, the Health Insurance Portability and Accountability Act (HIPAA) includes an administrative simplification section, which mandates protecting the information assets of patients. So, it is a dire need for the testing organizations to follow the necessary guidelines and compliances to limit the risk that non-compliance might follow.

Reap the benefits of different types of testing

In this day and age, mobile app quality matters the most as the market is very competitive. We need to focus on user experience as well as the performance of the app. If the app contains bugs the app ratings will go down and so will the number of app downloads. If we can make our app bug free than the chances of app gaining popularity will increase. This can be achieved by performing different types of testing on the mobile application. In the next blog in this series, we will talk about mobile app testing strategies.

Want to test your Mobile App?

Join pCloudy Platform!


Watch this video for more information:


Related Articles:

  • Mobile App Testing Strategies
  • 6 Most Common Problems in Mobile App Testing
  • Accelerated Mobile App Testing powered by pCloudy
  • Traditional Mobile App Testing Vs New Way of Mobile App Testing with pCloudy
  • Mobile Application Testing Trends in 2019
  • Types of Mobile Apps: Native, Hybrid, Web and Progressive Web apps
  • Suyash Dubey
    Suyash Dubey

    Suyash is a content strategist at pCloudy. He is a frequent contributor to the world's leading mobile technology blogs and tech forums. In his spare time, you will find him reading detective novels, watching a documentary or exploring a new destination.

    Related Articles

    April 19, 2020
    What’s New In pCloudy 5.6?

    pCloudy is committed to delivering the best solutions in mobile app testing and therefore we come up with product updates at regular intervals. This time we are thrilled to announce the release of pCloudy 5.6 with exciting new features to…

    Learn More Arrow

    March 17, 2020
    Ensure Continuous Productivity By Leveraging Remote Devices For Mobile App Testing

    Remote working is going to be a new normal and remote access tools and platforms will play a key role in maintaining productivity. Working from home has many advantages both for enterprises and teams. There are many tools that help…

    Learn More Arrow

    January 31, 2020
    5 Best Python Frameworks For Test Automation In 2020

    Testing framework plays a crucial role in the success of any automated testing process. Choosing the right test automation framework is important as it will maximize the test coverage and improve test efficiency which means a better return on investment.…

    Learn More Arrow

    Mobile App Performance Monitoring Basics to Advanced

    shivani
    By shivani
    August 24, 2018
    3 min Read
    Share This Article
    Mobile App Performance Monitoring Basics to Advanced

    Monitoring Mobile App Performance


    Mobile Device App Testing
    Source: Pixabay

    With the myriad of apps cluttering our smartphone screens nowadays, it has become all the more important to optimize app performance. While going through your daily routine, you certainly don’t want useful apps hanging up or worse still, crashing abysmally.

    In this post, you’ll know how app monitoring performance works so that you can get the best out of your smartphone software.

    What is application performance management? 10 APM features that we can tick right off our fingers are:

    • Troubleshooting of disruption of frequent speed and uptime
    • Monitoring of the speed at which transactions are performed by end-users and systems
    • Use of a single integrated suite of software tools
    • Load-testing
    • Synthetic monitoring
    • Real-use monitoring
    • Root-cause analysis
    • Web-performance monitoring, to gauge app speed and uptime
    • An end-to-end overview of bottlenecks
    • Review of service interruptions

    APM is integrated through the lifecycle of an application including the pre-deployment and post-deployment stages. It is therefore important for developers, testers, and business teams.
    A slow load can be prevalent due to a host of issues related to APIs, servers, or browsers. APM techniques are what help pinpoint the actual problem.

    Basic availability monitoring involves testing IP protocols and network services, but a user needs to establish some optimal performance thresholds and real-time alerts to monitor and manage both speed and reliability.

    Tips for monitoring app performance

    Tips for monitoring app performance
    Source: Pixabay

    Performance monitoring must take into account certain things in order to optimize user experience. These include measuring from the user’s point of view, i.e. performance, stability, and resource utilization in real time the way the user perceives them.

    The context is crucial too. It’s important to determine how many users were affected by a slowdown, and what action on a specific operating device prompted the disruption. To truly experience a ‘mobile moment,’ you should follow these tips for monitoring your app performance:

    Measure Frame rendering time

    Only measuring network time doesn’t cut it out for users, who are interested in when the network resumes as well as when it responds.

    Measuring response time by the device or operating system allows the product manager to focus on one platform at a time, respond to performance issues, and deploy resources accordingly.

    Measure battery usage

    It’s crucial to measure the battery consumption of your mobile app, along with data plan impact and memory usage. If the app is responsible for draining your battery, you should bring it in line with best practices.

    Frequent updates can also be minimized so that user satisfaction can be increased.

    Measure cell data consumption

    Recent studies by Dimensional Research have shown that 20% of respondents stopped using apps that were heavy on data usage. This indicates that you should measure the consumption by app version, the total number of users, KBs of data consumed per minute, the amount of data being transferred per hit, and the percentage of data being consumed per hit.

    Measure errors

    HTTP errors from certain URLs need to be captured. This will help track the number of users who received an error, the actions that resulted in it, number of users with errors on app launch, the percentage of actions that led to at least one error, and the app version.

    shivani
    shivani

    She is a Product Marketer with over 9 years of diversified experience across content, branding, and product marketing. Her experience ranges from global brands to nimble startups. She is a custodian of Brand & Content, telling stories about the brand that delights customers and provides compelling business value.

    Related Articles

    April 12, 2019
    Mobile App Testing Strategies

    [xyz-ihs snippet="quickLinks-mobile-app-testing"] In the year 2020, there will be around 5.5 Billion mobile users which accounts for 70% of the world population. More mobile users mean more apps and more competition and to lead the competition we need to make…

    Learn More Arrow

    April 9, 2019
    7 Types Of Mobile App Testing

    [xyz-ihs snippet="quickLinks-mobile-app-testing"] Types of Testing Since there are many players in the mobile app market, the competition is endless. To beat the heat and to keep providing an enhanced mobile app experience to the users, the app needs to be…

    Learn More Arrow

    September 11, 2018
    Moving Beyond Traditional App Testing with AI and DevOps

    App Testing with AI and DevOps   Are we not living in an amazing time? Technologically advanced, digitally sound!   We thrive on all things digital. The world around us is becoming all digital with limitless possibilities. Today as a…

    Learn More Arrow

    Test Your Apps in Different Network Environments

    Avinash
    By Avinash
    March 23, 2017
    1 min Read
    Share This Article
    Test Your Apps in Different Network Environments

    Test your Apps

    Did You Know?

    Test Your Apps function differently in different Network Environments

    There might be a whole set of bugs you probably are marking as “Cannot Reproduce” simply because you are thinking “Hmmm… Funny, it seems to works here”.

    The App you tested in your lab environment will not work the same way when you use it in an elevator, or in a basement, or while you are travelling on a highway. These are probably a whole set of testing scenarios you probably haven’t thought of.

    You Can do now

    Test your Apps

    Mimic different real network environments that impact app functionality and performance

    networks

    Create different Network Profiles and test you apps on multiple Network Environments

    Test your Apps

    Identify and rectify functional and performance issues

    bandwidth

    Modify various parameters like bandwidth, packet loss and latency during testing

    Avinash
    Avinash

    Avinash Tiwari is a thought leader, a recognized keynote speaker, and the co-founder of pCloudy.com. He has 15+ experience in Product development and Testing. He brings with him a passion for emerging technology and quick adoption, both of which have solidified his reputation as a leader in the mobile app testing field.

    Related Articles

    July 30, 2019
    Functional Testing Vs Non-Functional Testing

    According to a report, app crashes cause 71% of uninstalls. Other reasons that compel users to uninstall apps are page response time, confusing UI, and battery consumption. This indicates the importance of functional testing as well as non-functional testing in…

    Learn More Arrow

    April 9, 2019
    7 Types Of Mobile App Testing

    [xyz-ihs snippet="quickLinks-mobile-app-testing"] Types of Testing Since there are many players in the mobile app market, the competition is endless. To beat the heat and to keep providing an enhanced mobile app experience to the users, the app needs to be…

    Learn More Arrow

    August 24, 2018
    Mobile App Performance Monitoring Basics to Advanced

    Monitoring Mobile App Performance   [caption id="attachment_8547" align="aligncenter" width="628"] Source: Pixabay[/caption] With the myriad of apps cluttering our smartphone screens nowadays, it has become all the more important to optimize app performance. While going through your daily routine, you certainly…

    Learn More Arrow

    Monitoring App Performance

    Avinash
    By Avinash
    July 3, 2016
    2 min Read
    Share This Article
    Monitoring App Performance

    Application Performance Monitoring

    Mobile application testing does not stop when all the functional tests pass. Testing the application performance monitoring is a critical step before releasing it.

    pCloudy helps you with performance profiling of your mobile apps. You can view the CPU usage, Memory usage, Data usage and Battery consumption while performing a series of actions on the app. The Real time app performance monitoring feature helps you to keep track of the app’s performance while conducting manual tests. You can even check the performance consistency of the app by running automation test suites in parallel on multiple devices and by also simulating different network environments.

    App Performance while conducting Manual Tests

    The performance of an app can be viewed real-time while conducting functional tests. You can simply select the application installed on the device from the Tools section to view a graph of Memory and CPU usage, with the packets/bytes consumed in the Data Usage.

    application performance monitoring

    All information captured during a testing session is gathered by the platform and stored in a secure location to be used later. When a tester completes his testing session using a device, the performance related information can be found in the Cloud Drive.

    Battery Consumption

    The Battery Consumption graph indicates Battery consumption by Device, Battery consumption by Application and Battery consumption in %.

    Perf_Battery_Consumption

     CPU Consumption

    The CPU consumption of an App is plotted as CPU consumption in percentage against Time Duration in seconds.

    Perf_CPU_Consumption

    Memory Consumption

    The CPU consumption of an App is plotted as Memory consumption in ‘Mb’ against Time Duration in seconds.

    Perf_Mem_Consumption

    Avinash
    Avinash

    Avinash Tiwari is a thought leader, a recognized keynote speaker, and the co-founder of pCloudy.com. He has 15+ experience in Product development and Testing. He brings with him a passion for emerging technology and quick adoption, both of which have solidified his reputation as a leader in the mobile app testing field.

    Related Articles

    April 12, 2019
    Mobile App Testing Strategies

    [xyz-ihs snippet="quickLinks-mobile-app-testing"] In the year 2020, there will be around 5.5 Billion mobile users which accounts for 70% of the world population. More mobile users mean more apps and more competition and to lead the competition we need to make…

    Learn More Arrow

    April 9, 2019
    7 Types Of Mobile App Testing

    [xyz-ihs snippet="quickLinks-mobile-app-testing"] Types of Testing Since there are many players in the mobile app market, the competition is endless. To beat the heat and to keep providing an enhanced mobile app experience to the users, the app needs to be…

    Learn More Arrow

    September 11, 2018
    Moving Beyond Traditional App Testing with AI and DevOps

    App Testing with AI and DevOps   Are we not living in an amazing time? Technologically advanced, digitally sound!   We thrive on all things digital. The world around us is becoming all digital with limitless possibilities. Today as a…

    Learn More Arrow

    Experience pCloudy Today

    Tickmark No Credit Card Required
    Tickmark Exceptional Security

    Copyright All Rights Reserved © 2020