System Testing

System Testing

What is System Testing in Software Testing?

System testing means testing the system as a whole. All the modules/components are integrated in order to verify if the system works as expected or not.

System testing is done after integration testing. This plays an important role in delivering a high-quality product. 

List of Tutorials:

  • What is system testing
  • System vs end to end testing

The process of testing of an integrated hardware and software system to verify that the system meets its specified requirements.

Verification: Confirmation by examination and provisions of objective evidence that specified requirements have been fulfilled.

If an application has three modules A, B, and C, then testing done by combining the modules A & B or module B & C or module A& C is known as Integration testing. Integrating all the three modules and testing it as a complete system is termed as System testing.

A car manufacturer does not produce the car as a whole car. Each component of the car is manufactured separately, like seats, steering, mirror, break, cable, engine, car frame, wheels etc.

After manufacturing each item, it is tested independently whether it is working the way it is supposed to work and that is called Unit testing.

Now, when each part is assembled with another part, that assembled combination is checked if assembling has not produced any side effect to the functionality of each component and whether both components are working together as expected and that is called integration testing.

Once all the parts are assembled and the car is ready, it is not ready actually.

The whole car needs to be checked for different aspects as per the requirements defined like if car can be driven smoothly, breaks, gears and other functionality working properly, car does not show any sign of tiredness after being driven for 2500 miles continuously, color of car is generally accepted and liked, car can be driven on any kind of roads like smooth and rough, sloppy and straight etc and this whole effort of testing is called System testing and it has nothing to do with integration testing.

The example worked the way it was expected and the client was convinced about efforts required for system test.

I narrated the example here to encourage the importance of this testing.

It is performed when integration testing is completed.

It is mainly a black box type testing. This testing evaluates working of the system from a user point of view, with the help of specification document. It does not require any internal knowledge of system like design or structure of the code.

It contains functional and non-functional areas of application/product.

Focus criteria:

It mainly focuses on following:

  1. External interfaces
  2. Multiprogram and complex functionalities
  3. Security
  4. Recovery
  5. Performance
  6. Operator and user’s smooth interaction with system
  7. Installability
  8. Documentation
  9. Usability
  10. Load / Stress

#1. It is very important to complete a full test cycle and ST is the stage where it is done.

#2. ST is performed in an environment which is similar to the production environment and hence stakeholders can get a good idea of the user’s reaction.

#3. It helps to minimize after-deployment troubleshooting and support calls.

#4. In this STLC stage Application Architecture and Business requirements, both are tested.

This testing is very important and it plays a significant role in delivering a quality product to the customer.

Let’s see the importance of this testing through the below Examples which include our day to day tasks:

  • What if an online transaction it fails after confirmation?
  • What if an item placed in a cart of an online site does not allow to place an order?
  • What if in a Gmail account creating a new label gives an error on clicking the create tab?
  • What if the system crashes when a load is increased on the system?
  • What if the system crashes and is not able to recover the data as desired?
  • What if installing a software on the system takes much more time than expected and at the end gives an error?
  • What if a website response time increases much more than expected after enhancement?
  • What if a website becomes too slow that user is unable to book his/her travel ticket?

Above are just a few examples to show how System testing would affect if not done in a proper manner.

All the above examples are just the result of either system testing not performed or not done properly. All the integrated modules should be tested in order to ensure that the product works as per the requirements.

System testing can be considered as a black-box test technique.

Black box testing technique does not require internal knowledge of the code whereas white box technique requires internal knowledge of the code.

While performing System testing functional & non-functional, security, Performance and many other testing types are covered and they are tested using black box technique wherein the input is provided to the system and the output is verified. System internal knowledge is not required.

It is basically a part of software testing and test plan should always contain specific space for this testing.

To test the system as a whole, requirements and expectations should be clear and the tester needs to understand the real-time usage of application too.

Also, most used third-party tools, version of OSes, flavours and architecture of OSes can affect system’s functionality, performance, security, recoverability or installability.

Therefore, while testing system a clear picture of how the application is going to be used and what kind of issues it can face in real time can be helpful. In addition to that, a requirements document is as important as understanding the application.

Clear and updated requirements document can save tester from a number of misunderstandings, assumptions and questions.

In short, a pointed and crisp requirement document with latest updates along with an understanding of real-time application usage can make ST more fruitful.

This testing is done in a planned and systematic manner.

Given below are the various steps involved while performing this testing:

  • The very first step is to create a Test Plan.
  • Create System Test Cases and test scripts.
  • Prepare the test data required for this testing.
  • Execute the system test cases and script.
  • Report the bugs. Re-testing the bugs once fixed.
  • Regression testing to verify the impact of the change in the code.
  • Repetition of testing cycle till the system is ready to be deployed.
  • Sign off from the testing team.

Steps to perform System Testing

The points stated below are covered in this testing:

  • End to end testing which includes verifying the interaction between all the components and along with the external peripherals to ensure if the system works fine in any of the scenarios is covered in this testing.
  • It verifies that the input provided to the system provides the expected result.
  • It verifies if all the functional & non–functional requirements are tested and if they work as expected or not.
  • Ad-hoc and exploratory testing can be performed in this testing after scripted testing has been completed. Exploratory testing and ad-hoc testing helps to unfold the bugs which cannot be found in scripted testing as it gives freedom to the testers to test as their desire is based on their experience and intuition.

There are several advantages.

  • This testing includes end to end scenarios to test the system.
  • This testing is done in the same environment as of the Production environment which helps to understand the user perspective and prevents the issues which can occur when the system goes live.
  • If this testing is done in a systematic and proper manner, then it would help in mitigating the post-production issues.
  • This testing tests both the application architecture and the business requirement.

Let’s take a detailed look at the Entry/Exit criteria for System Test.

Entry Criteria:

  • The system should have passed the exit criteria of Integration testing i.e. all the test cases should have been executed and there should be no critical or Priority P1, a P2 bug in an open state.
  • Test Plan for this testing should be approved & signed off.
  • Test cases/scenarios should be ready to be executed.
  • Test scripts should be ready to be executed.
  • All the non–functional requirements should be available and test cases for the same should have been created.
  • Testing environment should be ready.

Exit Criteria:

  • All the test cases should be executed.
  • No critical or Priority or security-related bugs should be in an open state.
  • If any medium or low priority bugs are in an open state, then it should be implemented with the acceptance of the customer.
  • Exit Report should be submitted.

Test Plan is a document which is used to describe the purpose, objective, and scope of a product to be developed. What has to be tested and what should not be tested, testing strategies, tools to be used, environment required and every other detail is documented to proceed further with the testing.

Test plan helps to proceed with testing in a very systematic and strategic manner and that helps to avoid any risks or issues while testing is done.

System Test Plan covers the following points:

  • Purpose & Objective is defined for this test.
  • Scope (Features to be tested, Features not to be tested are listed).
  • Test Acceptance Criteria (Criteria on which the system will be accepted i.e. mentioned points in acceptance criteria should be in the pass state).
  • Entry/Exit criteria (Defines the criteria when system testing should start and when it should be considered as complete).
  • Test Schedule (Estimation of testing to be completed in a specific time).
  • Test Strategy (Includes testing techniques).
  • Resources (Number of resources required for testing, their roles, resource availability etc).
  • Test Environment (Operating System, Browser, Platform).
  • Test Cases (List of test cases to be executed).
  • Assumptions (If any assumptions, they should be included in the test plan).

System test cases cover all the scenarios & use cases and also it covers functional, non- functional, user interface, security-related test cases. The test cases are written in the same way as they are written for functional testing.

Test Plan is a document which is used to describe the purpose, objective, and scope of a product to be developed. What has to be tested and what should not be tested, testing strategies, tools to be used, environment required and every other detail is documented to proceed further with the testing.

Test plan helps to proceed with testing in a very systematic and strategic manner and that helps to avoid any risks or issues while testing is done.

System Test Plan covers the following points:

  • Purpose & Objective is defined for this test.
  • Scope (Features to be tested, Features not to be tested are listed).
  • Test Acceptance Criteria (Criteria on which the system will be accepted i.e. mentioned points in acceptance criteria should be in the pass state).
  • Entry/Exit criteria (Defines the criteria when system testing should start and when it should be considered as complete).
  • Test Schedule (Estimation of testing to be completed in a specific time).
  • Test Strategy (Includes testing techniques).
  • Resources (Number of resources required for testing, their roles, resource availability etc).
  • Test Environment (Operating System, Browser, Platform).
  • Test Cases (List of test cases to be executed).
  • Assumptions (If any assumptions, they should be included in the test plan).

Procedure To Write System Test Cases

System test cases cover all the scenarios & use cases and also it covers functional, non- functional, user interface, security-related test cases. The test cases are written in the same way as they are written for functional testing.

ST is called a superset of all types of testing as all the major types of testing are covered in it. Although a focus on types of testing may vary on the basis of product, organization processes, timeline and requirements.

Functionality Testing: To make sure that functionality of product is working as per the requirements defined, within the capabilities of the system.

Recoverability Testing: To make sure how well the system recovers from various input errors and other failure situations.

Interoperability Testing: To make sure whether the system can operate well with third-party products or not.

Performance Testing: To make sure system’s performance under the various condition, in terms of performance characteristics.

Scalability Testing: To make sure system’s scaling abilities in various terms like user scaling, geographic scaling and resource scaling.

Reliability Testing: To make sure system can be operated for longer duration without developing failures.

Regression Testing: To make sure system’s stability as it passes through an integration of different subsystems and maintenance tasks.

Documentation Testing: To make sure that system’s user guide and other help topics documents are correct and usable.

Security Testing: To make sure that system does not allow unauthorized access to data and resources.

Usability Testing: To make sure that system is easy to use, learn and operate.

It is a type of testing in which system’s ability to maintain data integrity and operation in coordination with other systems in the same environment, is being checked.

Example of System Integration Testing:

Let’s take an example of well-known online ticket booking site – http://irctc.co.in.

This is a ticket booking facility; online shopping facility interacts with PayPal. Overall you can consider it as A*B*C=R.

Now on the system level, online ticket booking facility, online shopping facility and online payment option facility can be system tested independently, followed by check perform Integration tests for each of them. And then the whole system needs to be system tested.

So where does System Integration testing come into the picture?

The web portal http://Irctc.co.in is a combination of systems. You may perform tests at the same level (single system, the system of systems), but at each level, you may want to focus on different risks (integration problems, independent functionality).

  • While testing Online Ticket booking facility, you may verify if you are able to book tickets online. You may also consider integration problems e.g. Ticket booking facility integrates back-end with front-end (UI). For example, how front-end behaves when the database server is slow to respond?
  • Testing of Online ticket booking facility with online shopping facility. You may verify that online shopping facility is available for the users logged into the system to book tickets online. You may also consider verification of integration in online shopping facility.

Difference Between System And Acceptance Testing

Given below are the major differences:

System Testing Acceptance Testing
1 System testing is the testing of a system as a whole. End to end testing is performed to verify that all the scenarios are working as expected. Acceptance testing is done to verify if the product meets customer requirement.
2 System testing includes functional & non- functional testing and is performed by the testers. Acceptance testing is functional testing and is performed by testers as well as a customer.
3 Testing is performed using test data created by the testers. Real/Production data is used while performing acceptance testing.
4 A system as a whole is tested to check the functionality & Performance of the product. Acceptance testing is done to verify that business requirement i.e. it solves the purpose what customer is looking for.
5 Defects found in the testing can be fixed. Any defects found while acceptance testing is considered as a failure of the Product.
6 System and system integration testing are types for System testing. Alpha and Beta testing come under acceptance testing.
Month End Offer - Flat 20% Off + 20% Cashback  
+
WhatsApp us