linkedin Functional Testing Archives - pCloudy

Functional Testing Vs Non-Functional Testing

Suyash Dubey
By Suyash Dubey
July 30, 2019
15 min Read
Share This Article
Functional Testing Vs Non-Functional Testing

According to a report, app crashes cause 71% of uninstalls. Other reasons that compel users to uninstall apps are page response time, confusing UI, and battery consumption. This indicates the importance of functional testing as well as non-functional testing in delivering a user-friendly app. So let’s examine functional testing.

What Is Functional Testing?

Functional testing is done to make sure that the functions of an app are working in conformance with the required specification. It is black box testing and does not go into the details of the source code of the app. In functional testing the focus should be on the user-friendliness of the main functions of the app.

To perform functional testing, first we need to identify the test input and compute the expected outcomes with the selected test input values. Then we execute the test cases and compare the actual data to the expected result.

Functional Testing

Functional Testing Types

Let’s have a look at types of functional testing for mobile applications.

  • Unit Testing
  • Integration Testing
  • Sanity Testing
  • Smoke Testing
  • Regression Testing
  • System Testing
  • Beta/User Acceptance Testing
  • Unit Testing

    In Unit Testing, individual components of a software application are tested during the development phase. Unit testing is usually done by developers as opposed to testers. A function, which is in the form of a section of code, is tested to verify accuracy. Drivers, unit testing frameworks, mock objects, and stubs are used to perform unit testing.

    Unit testing is usually automated, but sometimes it can be done manually. A manual approach can be effected with the help of an instructional document. Unit testing can be performed on all types of mobile applications.

    In automated unit testing, a developer writes code in the app to test the function or procedure. Once the app has been deployed, that code can be removed. The function can be isolated to test the app rigorously. It reveals the dependencies between the code being tested and other units. Then the dependencies can be eliminated. Most of the developers use a unit test automated framework to log the failing test cases.

    Integration Testing

    Integration testing is done to check if the individual components of the app function as expected when integrated. Integration testing is usually done after unit testing when all the builds are tested and combined.

    A software module is generally tested by a software developer, whose programming logic might differ from that of other developers. Integration testing is done to check that the logic works in unity. At the time of product development the requirements of the client may change, hence integration testing becomes necessary if the new changes are not unit tested at the time of development. Integration testing is also important because interfaces between the software module and the database could be erroneous.

    Sanity Testing

    Sanity testing is performed after receiving the software build with minor changes in the functionality to make sure that the errors are rectified and there will be no issues due to these changes in the future. The goal is to test the function on a superficial level (not thoroughly) to see if the developer has rationally built the function or not.

    Usually, these following steps are considered while performing sanity testing: Mark the modifications introduced in the code with the newly added features. Evaluate these marked features to ensure that they are fulfilling their motive. Then test the related features, associated parameters, and elements to ensure their proper working. After all this, the build can be subjected to other advanced testing methods. It’s better to use a cloud-based testing platform rather than online Android emulators as they provide hundreds of device browser combinations.

    Smoke Testing

    This is done to check if the build is stable enough to proceed with further testing. Smoke testing is done end to end and not for a particular component as we do in insanity testing.

    Smoke testing, also known as “build verification testing” is a type of software testing that comprises a set of test cases aimed at ensuring that the most important functions work. It is often a set of automated regression and functional tests that are focused on the most important and frequently used feature. The results of this testing are used to decide if a build is stable enough to proceed with further testing. So can we say that smoke testing should also involve performance testing, it should only be executed if the web application has a significant amount of traffic and performance metrics like page loading time are critical?

    Regression Testing

    Regression testing is done to check if the newly added code has any adverse effect on the functionality of the app. This is done when a new feature has been added to the app or if there have been some changes in the code. The changes in the code can impact the existing flow of the product or they may have bugs. This can be evaluated through regression testing, in which the motive is to initiate optimization and enhancement, as well as fixing issues if needed in the existing feature.

    System Testing

    System testing is the testing of a fully integrated software product. The software is interfaced with hardware and other software and a series of tests are conducted on the fully integrated app on the system. It can be a white box or black box or both.

    Here, non-functional testing refers to reliability and performance testing, efficiency testing, portability testing, and maintainability testing. The functional testing at the system level verifies interoperability testing of interfacing with all external components, along with end-to-end business workflows. In system testing, the testers may also need to deal with the undocumented or incomplete requirements of the software.

    System testing needs to fulfill the following requirements:

  • Test for the functional and non-functional requirements of the software.
  • Test the whole software as per the scope of the release.
  • Test must be conducted in a production environment.
  • This is the final phase of testing before the release of the software
  • Beta/User Acceptance Testing

    Beta testing and UAT (user acceptance testing) happens in the final phase of the testing process before the app is ready to be released. It is performed by the client/user to validate end-to-end business flow and user-friendliness.

    This may also involve verification of the help desk functions, backup, recovery, business processes, and user training material evaluation.

    User satisfaction increases as users become more confident that the requirements have been met, without any fear of how the product will behave in a real environment or that critical issues will arise when least expected. The quality criteria of the product are defined in the early phase of development.

    Vendors have improved communication both with the clients and within the team, as the requirement definition is improved through the acceptance tests and signed by the client. The engineering team ends with minimizing the pressure during the implementation and risks of post-implementation. Stakeholders use the information gathered through UAT to better understand the target audience’s need.

    What Is Non-Functional Testing?

    Non-functional testing is designed to evaluate the readiness of an app with various criteria like in load testing, scalability testing, and stress testing. It evaluates how well the app will perform in challenging conditions.

    Non Functional Testing Types

    Let’s look at the five major types of non-functional testing for mobile applications.

    Performance Testing

    Performance testing is done to ensure that the app will work smoothly under the expected workload. The goal is to find performance issues like reliability, resource usage, etc.but not to find bugs. Three main things that we should keep in mind while doing performance testing are quick response, maximum user load, and stability in a varied environment. Even if you are focusing on mobile testing and using online Android emulators, performance testing cannot be avoided.

    Types of Performance Testing:

    Endurance Testing:This is done to check if an app can withstand the load that it is expected to endure for a long period of time.

    Scalability Testing:This is done to check the performance of an app at maximum load and minimum load at software, hardware, and database level.

    Load Testing:In this, the system simulates actual user load on an app to check the threshold for the maximum load the app can bear.

    Stress Testing:This is done to check the reliability, stability, and error handling of an app under extreme load conditions.

    Spike Testing: In this, an app is tested with sudden increment and decrement in the user load. By performing spike testing, we also get to know the recovery time for an app to stabilize.

    Volume Testing:This is done to analyze an app’s behavior and response time when flooded with a large amount of data.

    Compatibility Testing

    Compatibility testing is performed to make sure that the app works as expected on various hardware, operating systems, network environments, and screen sizes.

    Security Testing

    Security testing is the most important part of the mobile app testing process and it ensures that your app is secure and not vulnerable to any external threat like malware and viruses. By doing this we can figure out the loopholes in the app which might lead to loss of data, revenue, or even trust in the organization.

    Let’s have a look at the major security threats that should be eliminated during security testing.

    Privilege elevation

    In this, the hacker might app but he/she can increase the privilege that has been provided by already have an account in your app and using the services provided by your default. For example, if the app has a clause to award credits for referring the app to a friend then the hacker can extend the limit and get more money out of it.

    Unauthorized data access

    The most common type of attack is by gaining unauthorized access to valuable information. This can be done simply by hacking the login credentials or by hacking the server to access the data.

    URL manipulation

    Hackers manipulate the URL query string if the app or website uses the HTTP GET method to transfer data between the client and the server. The QA team can pass a modified parameter value to see if the server accepts it.

    Denial of service

    This type of attack renders the services of the app inaccessible to the end-users. The hackers can also rapture the working mechanism of the app and the server machine to make it unstable.

    Usability Testing

    Usability testing is performed by a small set of users to figure out the usability defects in the application. It is done in the initial phase of software development when the design is proposed. The focus is on how easy it is to use the app and if the system is meeting its expected objective. This type of mobile testing can also be performed on online android emulators.

    There are a few methods by which we can carry out usability testing. One is during the design phase where you can evaluate the design by just drawing on a piece of paper. Random tests can also be done once the app is built to check for usability. This can be done by real users on the site which can then provide results. Conversely, a tool might be very helpful with the statistics based on the inputs on design and wireframes.

    The first step in performing usability testing structurally is to identify the userswho are to operate the usability app. It is necessary to choose the users according to the behavior of the app with reference to geography, age, gender, etc. The next step is to design a task that the user is supposed to perform and then to analyze the results of the testing.

    Usability can be done in an isolated test area in front of observers who inspect the testing and create a report on the assigned task. The next option is remote usability testing in which the observer and the testers are located in separate locations and the task is done remotely. Testers’ details such as their reactions are recorded by automated software.

    Localization Testing

    This is done to assure that the app is customized and behaves as per the culture of the country in which it will be available. The key focus is on the content and the UI of the app. The app goes through a process of testing to check if the default language, date AND time format, currency, etc. are designed as per the targeted region.

    Difference between Functional Testing and Non-Functional Testing

    Functional Testing Non-Functional Testing
    Validates the actions and operations of an application. Verifies the performance of the application.
    Focuses on user requirements. Focus on user expectations.
    Executed before non-functional testing. Executed after functional testing.
    Easy to define functional requirements. Difficult to define requirements for non-functional testing.
    Eg., check login functionality. Eg: the page should load in one second.
    Functional testing is easily executed by manual testing. Non-functional testing should be automated.

    Conclusion

    On average, an app loses 95% of new users after the first three months. This is because there is not enough focus on testing, and as a result the app has bugs. This can be avoided with a good strategy and tools to perform functional testing and non-functional testing on the apps. Cloudy has features which make functional testing and non-functional testing easy and quick. It is important for users as it saves them money and resources.

    Watch this video to learn more about pCloudy :

    Want to test your Mobile App?

    Join pCloudy Platform!


    Related Articles:

  • Webinar – Non Functional Testing – Mobile Apps
  • How to Measure the Success of End to End Testing
  • Importance of Unit Testing
  • 5 Benefits of Parallel Testing
  • Audio Interaction Testing On Remote Devices
  • Suyash Dubey
    Suyash Dubey

    Suyash is a content strategist at pCloudy. He is a frequent contributor to the world's leading mobile technology blogs and tech forums. In his spare time, you will find him reading detective novels, watching a documentary or exploring a new destination.

    Related Articles

    August 7, 2017
    Webinar - Non Functional Testing - Mobile Apps

    Non functional testing pCloudy in partnership with TechGig is bringing in a series of Webinars. Here is a webinar on Non Functional Testing - Mobile Apps by Nilesh Tarale, Vice-President Smart Software Testing Solutions. To view Webinar Video directly Click…

    Learn More Arrow

    June 7, 2017
    App Testing during your Dev Cycle – Developer Tool to Access Real Devices on Cloud

    App Testing If recent past has been any indication, then it is a certainty there are growing expectations from Testers and Developers alike, to take quality head on, as a joint feature. More so in the Mobile App Testing projects…

    Learn More Arrow

    March 23, 2017
    Test Your Apps in Different Network Environments

    Did You Know? Test Your Apps function differently in different Network Environments There might be a whole set of bugs you probably are marking as “Cannot Reproduce” simply because you are thinking “Hmmm... Funny, it seems to works here”. The…

    Learn More Arrow

    App Testing during your Dev Cycle – Developer Tool to Access Real Devices on Cloud

    Avinash
    By Avinash
    June 7, 2017
    3 min Read
    Share This Article
    App Testing during your Dev Cycle – Developer Tool to Access Real Devices on Cloud

    App Testing

    If recent past has been any indication, then it is a certainty there are growing expectations from Testers and Developers alike, to take quality head on, as a joint feature. More so in the Mobile App Testing projects where changes are required faster than ever.

    As part of DevOps practices, it’s has become imperative for developers to run as many tests as possible with every code check-in. These tests could be automated functional, API or Unit Tests. Some of the popular tools to used by Developers to create their tests are Espresso, XCTest and Appium.

    Following are some the Challenges faced by developers:

    • Developers need access of right set of devices across different versions to be able to run their tests.
    • Debugging capability on those devices so that they can fix issues quickly.
    • Access to a specific model of devices to debug production issues.

    Try taking a look at a typical developer’s cubicle and you will see a series of mobile devices connected with several long USB cables running into computers. It does get frustrating to see others furiously plugging USB cables in and out of the mobile devices for App Testing on various devices.

    Many of the organizations are shifting to device cloud to provide their teams access of right Mobile Devices. Device cloud are solving the need of test teams but provide limited debugging capabilities and hence not preferred by Developers.

    To directly address need of Developers, pCloudy recently introduced DeviceTunnel, which fully allows developers to take complete control of the device in cloud. This unique solution provides access of cloud devices through the Android Studio or Eclipse IDE and the command line tool installed in your computer.

    It works as if Device is connected directly to your computer through a USB cable. From the point of view of tools like Android Studio or Eclipse, a cloud-based device appears physically attached. In reality, the Device Tunnel communicates with pCloudy’s servers over Internet.

    App Testing

    Access devices directly from your terminal

    Once a connection is established, the developers can perform the following actions on these devices:

    • Issuing a range of ADB commands for debugging, shell creations, port forwarding, and viewing general information about any cloud-based Android device
    • Copying and pushing files to connected cloud-based devices
    • Installing and uninstalling applications
    • Debugging apps during development or testing by adding breakpoints, inspecting variables, analyzing run-time metrics to optimize your app and more
    • Run their tests on the device directly from their IDE

    To know more on how to connect any device on pCloudy using Device Tunnel Click Here.

    It is undeniable that developers and testers need quicker access to diverse devices for the brisk evaluation of app and debugging. Device Tunnel enables both sets of engineers to instantly connect to any device hosted on cloud and run faster debug sessions and thereby, maximize the quality of their build cycles.

    Avinash
    Avinash

    Avinash Tiwari is a thought leader, a recognized keynote speaker, and the co-founder of pCloudy.com. He has 15+ experience in Product development and Testing. He brings with him a passion for emerging technology and quick adoption, both of which have solidified his reputation as a leader in the mobile app testing field.

    Related Articles

    July 30, 2019
    Functional Testing Vs Non-Functional Testing

    According to a report, app crashes cause 71% of uninstalls. Other reasons that compel users to uninstall apps are page response time, confusing UI, and battery consumption. This indicates the importance of functional testing as well as non-functional testing in…

    Learn More Arrow

    May 6, 2019
    The Role of Artificial Intelligence in Transforming DevOps

    DevOps helps enterprises to build software at a fast pace and with minimal issues. The time to market is accelerated and the bugs are fixed faster in continuous deployment with the help of automated tools. AI is much in line…

    Learn More Arrow

    April 12, 2019
    Mobile App Testing Strategies

    [xyz-ihs snippet="quickLinks-mobile-app-testing"] In the year 2020, there will be around 5.5 Billion mobile users which accounts for 70% of the world population. More mobile users mean more apps and more competition and to lead the competition we need to make…

    Learn More Arrow

    Test Your Apps in Different Network Environments

    Avinash
    By Avinash
    March 23, 2017
    1 min Read
    Share This Article
    Test Your Apps in Different Network Environments

    Test your Apps

    Did You Know?

    Test Your Apps function differently in different Network Environments

    There might be a whole set of bugs you probably are marking as “Cannot Reproduce” simply because you are thinking “Hmmm… Funny, it seems to works here”.

    The App you tested in your lab environment will not work the same way when you use it in an elevator, or in a basement, or while you are travelling on a highway. These are probably a whole set of testing scenarios you probably haven’t thought of.

    You Can do now

    Test your Apps

    Mimic different real network environments that impact app functionality and performance

    networks

    Create different Network Profiles and test you apps on multiple Network Environments

    Test your Apps

    Identify and rectify functional and performance issues

    bandwidth

    Modify various parameters like bandwidth, packet loss and latency during testing

    Avinash
    Avinash

    Avinash Tiwari is a thought leader, a recognized keynote speaker, and the co-founder of pCloudy.com. He has 15+ experience in Product development and Testing. He brings with him a passion for emerging technology and quick adoption, both of which have solidified his reputation as a leader in the mobile app testing field.

    Related Articles

    July 30, 2019
    Functional Testing Vs Non-Functional Testing

    According to a report, app crashes cause 71% of uninstalls. Other reasons that compel users to uninstall apps are page response time, confusing UI, and battery consumption. This indicates the importance of functional testing as well as non-functional testing in…

    Learn More Arrow

    April 9, 2019
    7 Types Of Mobile App Testing

    [xyz-ihs snippet="quickLinks-mobile-app-testing"] Types of Testing Since there are many players in the mobile app market, the competition is endless. To beat the heat and to keep providing an enhanced mobile app experience to the users, the app needs to be…

    Learn More Arrow

    August 24, 2018
    Mobile App Performance Monitoring Basics to Advanced

    Monitoring Mobile App Performance   [caption id="attachment_8547" align="aligncenter" width="628"] Source: Pixabay[/caption] With the myriad of apps cluttering our smartphone screens nowadays, it has become all the more important to optimize app performance. While going through your daily routine, you certainly…

    Learn More Arrow

    pCloudy Plugin for Android Studio

    Avinash
    By Avinash
    August 24, 2016
    4 min Read
    Share This Article
    pCloudy Plugin for Android Studio

    Plugin for Android Studio

    An Overview

    pCloudy has released a plugin for Android Studio which provides you access to the platform for performing manual and automation testing of mobile apps from within Android Studio IDE.

    For Manual Testing

    Now you can build and upload your Android Apps to pCloudy devices directly from your Android Studio project workspace.

    For Automation Testing

    If you are using Appium as automation tool and have created your Automation scripts as a project in Android Studio, you can run the scripts directly on pCloudy devices.

    Prerequisites

    1. An account in pCloudy with enough credits.
    2. An API Access key for authentication with pCloudy.com
    3. Chrome Browser on local machine
    4. Android Studio IDE
    5. Android Studio pCloudy plugin

    Preparation

    Reference the JAR files to Eclipse
    Make a note the API Username and API Access Key

    How to Install the Plugin

    1. Download the Android Studio plugin for pCloudy
    2. Open your Android Studio > go to Setting icon of Android Studio > Select Plugins > All available plugins are shown which can be filtered with the help of Show feature as per requirement.
    3. Click on Install plugin from disk > Browse and select location of your downloaded Android Studio plugin of pCloudy
    4. Click on OK
    5. Now, you can see that pCloudy plugin has been successfully integrated to Android Studio.

    Select pcloudy plugin and click on OK to proceed. Here, we are going to install pCloudy plugin for Android Studio from local disk. Click on Install plugin from disk
    install-plugin-from-disk

    pCloudy plugin for Android Studio has been integrated successfully.

    pcloudy-plugin-over-android-studio

    Click on pCloudy plugin icon shown on Android Studio. pCloudy Login page opens. Enter your Username and Access Key and click on Login. Your login credentials are verified. Click on OK to proceed.
    pcloudy-plugin-over-android-studio

    Note: This is one-time task. Credentials will be valid until you logout.
    Now, you are ready to use the Android Studio Plugin for pCloudy.
    Manual Functional Testing of App

    After successful login, upload your app screen appears. Workspace page helps you to select application path of your local machine.
    successfully-authenticated

    Cloud Drive page lets you to select app from the list and click on Next to proceed.
    upload-your-app

    Local Drive lets you to upload your app from your local machine to perform testing.
    upload-a-project-file

    Mobile Browser enables you to use mobile browser for testing.
    use-mobile-browser-for-testing

    After selecting app, the next step is choosing a device. Select and Connect to devices page provides you list of devices which can be filtered by using filter parameters (Android/iOS, Manufacturer, and OS version). Select duration of your testing. The device will get booked for selected time duration.

    When you click on Next, you get a message “Please wait while we open the pCloudy Platform on your default browser for Manual Testing. Please advised that…”
    select-and-connect-to-devices

    Just after few seconds, you will be redirected to the connected device page over pCloudy to perform manual functional testing of your app on selected device.
    briefing

    Automation Testing of App

    Similarly, the first step towards performing automation testing is selecting an app from workspace/cloud drive/local drive or use mobile browser.
    my-application

    Select your desired device; select duration of automation testing. You can filter devices displaying devices by using filter options (Android/iOS, Manufacturer, and OS version). Click onNext to proceed.
    choose-devices

    Note: Here, you can select multiple devices as well.

    Finally, you get corresponding pseudo code for performing Automation testing of app. You can copy this pseudo code to clipboard and use it within your Automation script…
    paste-the-code-and-proceed

    You will need the pCloudy java connector jar file for the above code to work. The pCloudy-java-conector.jar contains the API wrappers, classes & methods to use the pCloudy RestfulAPIs (without knowing how to call rest webservices from java). The gson.jar is a well known, opensource google library. You should add the pCloudy-java-connector.jar in its build path as a reference within your java project containing appium scripts.

    Avinash
    Avinash

    Avinash Tiwari is a thought leader, a recognized keynote speaker, and the co-founder of pCloudy.com. He has 15+ experience in Product development and Testing. He brings with him a passion for emerging technology and quick adoption, both of which have solidified his reputation as a leader in the mobile app testing field.

    Related Articles

    April 28, 2020
    Test Local And Internal Servers Before Deployment Using Wildnet

    Some testing teams set up their own staging environment to test internal servers but as there is no public access which makes it difficult to perform local testing on remote devices. So how would you test your app which can…

    Learn More Arrow

    April 19, 2020
    What’s New In pCloudy 5.6?

    pCloudy is committed to delivering the best solutions in mobile app testing and therefore we come up with product updates at regular intervals. This time we are thrilled to announce the release of pCloudy 5.6 with exciting new features to…

    Learn More Arrow

    January 31, 2020
    5 Best Python Frameworks For Test Automation In 2020

    Testing framework plays a crucial role in the success of any automated testing process. Choosing the right test automation framework is important as it will maximize the test coverage and improve test efficiency which means a better return on investment.…

    Learn More Arrow

    What Does iPhone 6 / iPhone 6+ Mean for Your Application Testing Needs?

    Avinash
    By Avinash
    June 2, 2015
    8 min Read
    Share This Article
    What Does iPhone 6 / iPhone 6+ Mean for Your Application Testing Needs?
    “iOS 8 is the biggest iOS release ever — for developers and everyone else. But that wasn’t the goal. We simply set out to create the most natural experience. Each enhancement has a purpose. Every new feature deserves to be a new feature. Each function is more considered, each next step is more efficient. It all adds up to an even better experience — one that is pleasantly surprising at first and becomes utterly indispensable before you know it.”

    What is new ?

    Inter-App Communication

    Inter-app communication means different things to different people. For some it’s the ability to push files from one app to another. For others it’s the ability to pull data into any field in any app. The core issue, however, is workflow. People just want an easier way to move their stuff around.

    Touch ID Authentication

    Touch ID is the name of Apple’s personal fingerprint identity sensor. It’s what currently lets you authenticate yourself to unlock your iPhone 5s and authorize iTunes and App Store purchases on your account.

     Custom keyboards

    After years of waiting, Apple has finally brought support for third-party keyboards to iOS. Inside iOS 8, keyboards like Swiftkey and Swype, which have enjoyed huge usage on Android, will have system-wide access to all apps and services on your iPhone and iPad. Swiftkey has confirmed it’s onboard, but if you don’t fancy that, you’ll still able to enjoy Apple’s new QuickType keyboard. The company says the improved keyboard learns from the way you type and text, offering a pick of suggestions for your next word based on the content of your message or the person you’re conversing with. Planning a meal with your friend or loved one? The keyboard will auto-populate words like “dinner” or “eat” as you type. At launch, QuickType will support 14 regions including the US, UK, Canada, Australian English, Brazilian Portuguese, Chinese (that includes Hong Kong and Taiwan), French, German, Italian, Japanese, Spanish and Thai.

    Family Sharing

    New to iOS 8, a user can add six others as family members. Family members can share purchased apps, music, and books using the same credit card. iOS 8 can also automatically set up photo streams for all family members. Calendars may be synced between all members. Kids can also send iTunes download requests for apps, music, movies, and more to their parents provided this service is set up correctly.

    Third-party widgets

    iOS 7 split notifications and widgets apart, putting widgets in their own Today view, moving Weather to the Today Summary, and adding support for Calendar, Reminders, predictive location, and the Tomorrow Summary. Yet they were still limited to built-in apps and services.

    iOS 8 and Extensibility take it even further, allowing App Store apps to offer up their own Today view widgets — helpful information status indicators, simple, interactive utilities, and ways to launch into the full app when and if needed — easily accessible from anywhere on the iPhone or iPad, informational and interactive.

    What a new OS or device release means to the developers and Testers?

    Update Your Device/OS Test Matrix

    Maintaining Device /OS Test matrix is already a big challenge for organizations. It is always a big question that what device and OS combination should be included in the Test Matrix and what should be not. One way to set your Matrix in right direction is you can take the help of Google analytics that what devices your audience/users are coming from.Now each OS Device release we have to refresh this Matrix.

    How should we proceed and what all areas we should focus more:-

    Functional Testing

    According to Apple’s developer site “iOS 8 includes over 4,000 new APIs that let you add amazing new features and capabilities to your apps. Deeper integration with iOS means you can extend the reach of your app content and functionality.”

    Inter App communication

    Test if your application is able to interact seamlessly with other apps and Able to transfer Data from one app to other.

    Notification center

    If your app is using Notification API it is properly updated in Notification center and your application is able to customize its Notifications in Device Notification center.

    Touch ID Authentication

    If Your Application is using Touch ID Authentication the is it working fine as per your implementation

    A/B Testing

    App Login with Touch ID

    In iOS 8 developers can now use Touch ID to let their users log into their app. Typing in a password introduces a lot of friction into the signup flow, but Touch ID is also a new technology that isn’t perfect every time.
    Just because Touch ID is available for app login, doesn’t mean it’s the best option for every user. Set up a simple A/B test to see how leading with one option vs. another impacts sign-ins and sign-ups. Depending on who your audience is and how they use your app, you may be surprised by the results.

    Let’s face it; Touch ID doesn’t always work as promised. It’s important that your app respond before your users start to get frustrated and consider abandoning. Try testing different options for help text and UI elements to guide your users to a successful entry.

    Extended Sharing Options

    iOS 8 now lets developers share content from their apps with a new sharing options API. This release comes a big question: where will developers embed sharing actions? and what type of actions with sharing surface?

    We recommend experimenting with different places in your app you can surface the option to share. Make sure you pick a consistent place that makes sense for users as part of their interaction flow. This can be hard to know for certain, so we recommend brainstorming some options with your team and testing as many as you can.

    The types of actions you surface in your sharing options is also really important. Too many actions will clutter the experience and confuse people, threatening drop off. Think about the high-value actions you want your users to take. Start by highlighting only those. Then, progressively test into adding other actions based on user feedback and team ideas. If you start to see adding actions are negatively impacting your engagement, you know you might be going too far.

    UI Testing

    Remember these points before Testing

    Size Class

    With iOS 8, Apple is introducing “size classes”. Size classes have vertical and horizontal dimensions called “regular” and “compact”. The iPad in both portrait and landscape defaults to the regular size class in both horizontal and vertical directions. The iPhone in portrait defaults to compact size class for horizontal and regular size class for vertical. The iPhone in landscape defaults to compact size class for both horizontal and vertical.
    Apple provides some automatic behaviors based on size classes. For example, if you rotate an iPhone app that uses standard components from portrait to landscape (from compact/regular to compact/compact) the navigation bar gets condensed and the status bar disappears entirely. That’s to maximize the content on a screen that’s suddenly gone from being tall to being very, very short — like a web page on Safari.
    Developers are free to customize the layout for every orientation of every device they support as well. For example, they can have two buttons stacked on top of each other in portrait orientation to take advantage of the height, and those same buttons aligned side-by-side in landscape orientation to take advantage of the width. They’re the same controls, their position and other attributes simply change as the vertical size class changes.

    Split view

    Apple is also bringing split views to the iPhone. That means developers no longer have to maintain two separate interface hierarchies, one for iPad that contains split view, and one for iPhone that does not. Now they can maintain one hierarchy for both and the proper screens will all be rendered based on size class.

    Changed UI API’s

    With iOS 8 Release Apple has removed some of the UI API’s and a list of bug fixes in UI KIT made it very essential for user’s to test the UI thoroughly.

    Orientation

    You need to test the rendering of their application as per the changed classes and views

    UX Testing

    Enhanced Multitasking

    iOS 8 comes with enhanced multitasking functionalities. Testing your app user experience with this enhanced Multitasking feature of iOS 8.

    iOS 8 Touch ID authentications

    Testing the user experience of your application with iOS 8 Touch ID authentications functionality.

    Notification Center

    How Your Application Notification pushed into Device Notification center or What happens when your Device is in Locked state etc.

    Performance Testing

    CPU usage by Your Application on this new OS and device.
    App performance during interaction of one app with others.

    Security Testing

    Security will also be a major focus of this release. New iOS 8 features will make it even easier to keep track of all your passwords and make sure they are as secure as possible

    Avinash
    Avinash

    Avinash Tiwari is a thought leader, a recognized keynote speaker, and the co-founder of pCloudy.com. He has 15+ experience in Product development and Testing. He brings with him a passion for emerging technology and quick adoption, both of which have solidified his reputation as a leader in the mobile app testing field.

    Related Articles

    July 30, 2019
    Functional Testing Vs Non-Functional Testing

    According to a report, app crashes cause 71% of uninstalls. Other reasons that compel users to uninstall apps are page response time, confusing UI, and battery consumption. This indicates the importance of functional testing as well as non-functional testing in…

    Learn More Arrow

    July 10, 2019
    Basics of Appium Mobile Testing

    [xyz-ihs snippet="quickLinks-Appium"]   Appium is an open source mobile app UI testing framework. You can test all types of mobile apps and perform automation test on physical devices as well as on emulators and simulators. Appium mobile testing does not…

    Learn More Arrow

    April 12, 2019
    Mobile App Testing Strategies

    [xyz-ihs snippet="quickLinks-mobile-app-testing"] In the year 2020, there will be around 5.5 Billion mobile users which accounts for 70% of the world population. More mobile users mean more apps and more competition and to lead the competition we need to make…

    Learn More Arrow

    Experience pCloudy Today

    Tickmark No Credit Card Required
    Tickmark Exceptional Security

    Copyright All Rights Reserved © 2020