Home » Blog » Functional Testing Vs Non-Functional Testing

Functional Testing Vs Non-Functional Testing


According to a report, app crashes cause 71% of uninstalls. Other reasons that compel users to uninstall apps are page response time, confusing UI, and battery consumption. This indicates the importance of functional testing as well as non-functional testing in delivering a user-friendly app. So let’s examine functional testing.

What Is Functional Testing?

Functional testing is done to make sure that the functions of an app are working in conformance with the required specification. It is black box testing and does not go into the details of the source code of the app. In functional testing the focus should be on the user-friendliness of the main functions of the app.
 
To perform functional testing, first we need to identify the test input and compute the expected outcomes with the selected test input values. Then we execute the test cases and compare the actual data to the expected result.
 
Functional Testing

Functional Testing Types

Let’s have a look at types of functional testing for mobile applications.

  • Unit Testing
  • Integration Testing
  • Sanity Testing
  • Smoke Testing
  • Regression Testing
  • System Testing
  • Beta/User Acceptance Testing
  • Unit Testing

    In Unit Testing, individual components of a software application are tested during the development phase. Unit testing is usually done by developers as opposed to testers. A function, which is in the form of a section of code, is tested to verify accuracy. Drivers, unit testing frameworks, mock objects, and stubs are used to perform unit testing.
     
    Unit testing is usually automated, but sometimes it can be done manually. A manual approach can be effected with the help of an instructional document. Unit testing can be performed on all types of mobile applications.
     
    In automated unit testing, a developer writes code in the app to test the function or procedure. Once the app has been deployed, that code can be removed. The function can be isolated to test the app rigorously. It reveals the dependencies between the code being tested and other units. Then the dependencies can be eliminated. Most of the developers use a unit test automated framework to log the failing test cases.

    Integration Testing

    Integration testing is done to check if the individual components of the app function as expected when integrated. Integration testing is usually done after unit testing when all the builds are tested and combined.
     
    A software module is generally tested by a software developer, whose programming logic might differ from that of other developers. Integration testing is done to check that the logic works in unity. At the time of product development the requirements of the client may change, hence integration testing becomes necessary if the new changes are not unit tested at the time of development. Integration testing is also important because interfaces between the software module and the database could be erroneous.

    Sanity Testing

    Sanity testing is performed after receiving the software build with minor changes in the functionality to make sure that the errors are rectified and there will be no issues due to these changes in the future. The goal is to test the function on a superficial level (not thoroughly) to see if the developer has rationally built the function or not.
     
    Usually, these following steps are considered while performing sanity testing: Mark the modifications introduced in the code with the newly added features. Evaluate these marked features to ensure that they are fulfilling their motive. Then test the related features, associated parameters, and elements to ensure their proper working. After all this, the build can be subjected to other advanced testing methods. It’s better to use a cloud-based testing platform rather than online Android emulators as they provide hundreds of device browser combinations.

    Smoke Testing

    This is done to check if the build is stable enough to proceed with further testing. Smoke testing is done end to end and not for a particular component as we do in insanity testing.
     
    Smoke testing, also known as “build verification testing” is a type of software testing that comprises a set of test cases aimed at ensuring that the most important functions work. It is often a set of automated regression and functional tests that are focused on the most important and frequently used feature. The results of this testing are used to decide if a build is stable enough to proceed with further testing. So can we say that smoke testing should also involve performance testing, it should only be executed if the web application has a significant amount of traffic and performance metrics like page loading time are critical?

    Regression Testing

    Regression testing is done to check if the newly added code has any adverse effect on the functionality of the app. This is done when a new feature has been added to the app or if there have been some changes in the code. The changes in the code can impact the existing flow of the product or they may have bugs. This can be evaluated through regression testing, in which the motive is to initiate optimization and enhancement, as well as fixing issues if needed in the existing feature.

    System Testing

    System testing is the testing of a fully integrated software product. The software is interfaced with hardware and other software and a series of tests are conducted on the fully integrated app on the system. It can be a white box or black box or both.
     
    Here, non-functional testing refers to reliability and performance testing, efficiency testing, portability testing, and maintainability testing. The functional testing at the system level verifies interoperability testing of interfacing with all external components, along with end-to-end business workflows. In system testing, the testers may also need to deal with the undocumented or incomplete requirements of the software.
     
    System testing needs to fulfill the following requirements:
     

  • Test for the functional and non-functional requirements of the software.
  • Test the whole software as per the scope of the release.
  • Test must be conducted in a production environment.
  • This is the final phase of testing before the release of the software
  • Beta/User Acceptance Testing

    Beta testing and UAT (user acceptance testing) happens in the final phase of the testing process before the app is ready to be released. It is performed by the client/user to validate end-to-end business flow and user-friendliness.
     
    This may also involve verification of the help desk functions, backup, recovery, business processes, and user training material evaluation.
     
    User satisfaction increases as users become more confident that the requirements have been met, without any fear of how the product will behave in a real environment or that critical issues will arise when least expected. The quality criteria of the product are defined in the early phase of development.
     
    Vendors have improved communication both with the clients and within the team, as the requirement definition is improved through the acceptance tests and signed by the client. The engineering team ends with minimizing the pressure during the implementation and risks of post-implementation. Stakeholders use the information gathered through UAT to better understand the target audience’s need.

    What Is Non-Functional Testing?

    Non-functional testing is designed to evaluate the readiness of an app with various criteria like in load testing, scalability testing, and stress testing. It evaluates how well the app will perform in challenging conditions.

    Non Functional Testing Types

    Let’s look at the five major types of non-functional testing for mobile applications.

    Performance Testing

    Performance testing is done to ensure that the app will work smoothly under the expected workload. The goal is to find performance issues like reliability, resource usage, etc.but not to find bugs. Three main things that we should keep in mind while doing performance testing are quick response, maximum user load, and stability in a varied environment. Even if you are focusing on mobile testing and using online Android emulators, performance testing cannot be avoided.

    Types of Performance Testing:

    Endurance Testing:This is done to check if an app can withstand the load that it is expected to endure for a long period of time.

    Scalability Testing:This is done to check the performance of an app at maximum load and minimum load at software, hardware, and database level.

    Load Testing:In this, the system simulates actual user load on an app to check the threshold for the maximum load the app can bear.

    Stress Testing:This is done to check the reliability, stability, and error handling of an app under extreme load conditions.

    Spike Testing: In this, an app is tested with sudden increment and decrement in the user load. By performing spike testing, we also get to know the recovery time for an app to stabilize.

    Volume Testing:This is done to analyze an app’s behavior and response time when flooded with a large amount of data.

    Compatibility Testing

    Compatibility testing is performed to make sure that the app works as expected on various hardware, operating systems, network environments, and screen sizes.

    Security Testing

    Security testing is the most important part of the mobile app testing process and it ensures that your app is secure and not vulnerable to any external threat like malware and viruses. By doing this we can figure out the loopholes in the app which might lead to loss of data, revenue, or even trust in the organization.
     
    Let’s have a look at the major security threats that should be eliminated during security testing.
     
    Privilege elevation

    In this, the hacker might app but he/she can increase the privilege that has been provided by already have an account in your app and using the services provided by your default. For example, if the app has a clause to award credits for referring the app to a friend then the hacker can extend the limit and get more money out of it.
     
    Unauthorized data access

    The most common type of attack is by gaining unauthorized access to valuable information. This can be done simply by hacking the login credentials or by hacking the server to access the data.

    URL manipulation

    Hackers manipulate the URL query string if the app or website uses the HTTP GET method to transfer data between the client and the server. The QA team can pass a modified parameter value to see if the server accepts it.
     
    Denial of service

    This type of attack renders the services of the app inaccessible to the end-users. The hackers can also rapture the working mechanism of the app and the server machine to make it unstable.

    Usability Testing

    Usability testing is performed by a small set of users to figure out the usability defects in the application. It is done in the initial phase of software development when the design is proposed. The focus is on how easy it is to use the app and if the system is meeting its expected objective. This type of mobile testing can also be performed on online android emulators.
     
    There are a few methods by which we can carry out usability testing. One is during the design phase where you can evaluate the design by just drawing on a piece of paper. Random tests can also be done once the app is built to check for usability. This can be done by real users on the site which can then provide results. Conversely, a tool might be very helpful with the statistics based on the inputs on design and wireframes.
     
    The first step in performing usability testing structurally is to identify the userswho are to operate the usability app. It is necessary to choose the users according to the behavior of the app with reference to geography, age, gender, etc. The next step is to design a task that the user is supposed to perform and then to analyze the results of the testing.
     
    Usability can be done in an isolated test area in front of observers who inspect the testing and create a report on the assigned task. The next option is remote usability testing in which the observer and the testers are located in separate locations and the task is done remotely. Testers’ details such as their reactions are recorded by automated software.

    Localization Testing

    This is done to assure that the app is customized and behaves as per the culture of the country in which it will be available. The key focus is on the content and the UI of the app. The app goes through a process of testing to check if the default language, date AND time format, currency, etc. are designed as per the targeted region.
     

    Difference between Functional Testing and Non-Functional Testing

    Functional Testing Non-Functional Testing
    Validates the actions and operations of an application. Verifies the performance of the application.
    Focuses on user requirements. Focus on user expectations.
    Executed before non-functional testing. Executed after functional testing.
    Easy to define functional requirements. Difficult to define requirements for non-functional testing.
    Eg., check login functionality. Eg: the page should load in one second.
    Functional testing is easily executed by manual testing. Non-functional testing should be automated.

    Conclusion

    On average, an app loses 95% of new users after the first three months. This is because there is not enough focus on testing, and as a result the app has bugs. This can be avoided with a good strategy and tools to perform functional testing and non-functional testing on the apps. Cloudy has features which make functional testing and non-functional testing easy and quick. It is important for users as it saves them money and resources.
     
    Watch this video to learn more about pCloudy :

    Want to test your Mobile App?

    Join pCloudy Platform!

     
    Related Articles:

  • Webinar – Non Functional Testing – Mobile Apps
  • How to Measure the Success of End to End Testing
  • Importance of Unit Testing
  • 5 Benefits of Parallel Testing
  • Audio Interaction Testing On Remote Devices
  • Sharing is Caring