Software Testing Questions & Answers

Exploratory testing is a hands-on approach in which testers are involved in minimum planning and maximum test execution. The planning involves the creation of a test charter, a short declaration of the scope of a short (1 to 2 hour) time-boxed test effort, the objectives and possible approaches to be used. The test design and test execution activities are performed in parallel typically without formally documenting the test conditions, test cases or test scripts. This does not mean that other, more formal testing techniques will not be used. For example, the tester may decide to use boundary value analysis but will think through and test the most important boundary values without necessarily writing them down. Some notes will be written during the exploratory-testing session so that a report can be produced afterward.

In order to identify and execute the functional requirement of an application from start to finish “use case” is used and the techniques used to do this is known as “Use Case Testing.”

SDLC deals with development/coding of the software while STLC deales with validation and verification of the software

The relationship between test cases and requirements is shown with the help of a document. This document is known as a traceability matrix.

Equivalence partitioning testing is a software testing technique which divides the application input test data into each partition at least once of equivalent data from which test cases can be derived. By this testing method, it reduces the time required for software testing.

White box testing technique involves selection of test cases based on an analysis of the internal structure (Code coverage, branches coverage, paths coverage, condition coverage, etc.) of a component or system. It is also known as Code-Based testing or Structural testing. Different types of white box testing are

  1. Statement Coverage
  2. Decision Coverage

In white box testing following steps are verified.

  1. Verify the security holes in the code
  2. Verify the incomplete or broken paths in the code
  3. Verify the flow of structure according to the document specification
  4. Verify the expected outputs
  5. Verify all conditional loops in the code to check the complete functionality of the application
  6. Verify the line by line coding and cover 100% testing

Black box testing is the software testing method which is used to test the software without knowing the internal structure of code or program. This testing is usually done to check the functionality of an application. The different black box testing techniques are

  1. Equivalence Partitioning
  2. Boundary value analysis
  3. Cause-effect graphing

Static testing: During Static testing method, the code is not executed, and it is performed using the software documentation.

Dynamic testing: To perform this testing the code is required to be in an executable form.

Verification is a process of evaluating software at the development phase. It helps you to decide whether the product of a given application satisfies the specified requirements. Validation is the process of evaluating software at the after the development process and to check whether it meets the customer requirements.

There are four test levels

  1. Unit/component/program/module testing
  2. Integration testing
  3. System testing
  4. Acceptance testing

Integration testing is a level of software testing process, where individual units of an application are combined and tested. It is usually performed after unit and functional testing.

Test design, scope, test strategies, approach are various details that Test plan document consists of.

  1. Test case identifier
  2. Scope
  3. Features to be tested
  4. Features not to be tested
  5. Test strategy & Test approach
  6. Test deliverables
  7. Responsibilities
  8. Staffing and training
  9. Risk and Contingencies

System Testing: System testing is finding defects when the system undergoes testing as a whole; it is also known as end-to-end testing. In such type of testing, the application suffers from beginning till the end.

UAT: User Acceptance Testing (UAT) involves running a product through a series of specific tests which determines whether the product will meet the needs of its users.

Retesting: It is a process of checking bugs that are actioned by the development team to verify that they are fixed.

Data Driven Testing (DDT): In data driven testing process, the application is tested with multiple test data. The application is tested with a different set of values.

  • Record: Log and handle any problems which have happened
  • Report: Report the issues to higher level manager
  • Control: Define the issue management process

Difference between test scenarios and test cases is that

Test Scenarios: A Test Scenario is any functionality that can be tested. It is also called Test Condition or Test Possibility.

Test Cases: It is a document that contains the steps that have to be executed; it has been planned earlier.

Test Script: It is written in a programming language and it’s a short program used to test part of the functionality of the software system. In other words a written set of steps that should be performed manually.

Latent defect: This defect is an existing defect in the system which does not cause any failure as the exact set of conditions has never been met

To know the quality of test execution, we can use two parameters

  • Defect reject ratio
  • Defect leakage ratio

Phantom is a freeware and is used for windows GUI automation scripting language. It allows us to take control of windows and functions automatically. It can simulate any combination of keystrokes and mouse clicks as well as menus, lists and more.

Test Deliverables are a set of documents, tools and other components that have to be developed and maintained in support of testing.

There are different test deliverables at every phase of the software development lifecycle

  • Before Testing
  • During Testing
  • After the Testing

Mutation testing is a technique to identify if a set of test data or test case is useful by intentionally introducing various code changes (bugs) and retesting with original test data/ cases to determine if the bugs are detected.

  • Technical Feasibility
  • Complexity level
  • Application stability
  • Test data
  • Application size
  • Re-usability of automated scripts
  • Execution across environment

For the risk analysis following steps need to be implemented

  1. Finding the score of the risk
  2. Making a profile for the risk
  3. Changing the risk properties
  4. Deploy the resources of that test risk
  5. Making a database of risk

Categories for debugging

  1. Brute force debugging
  2. Backtracking
  3. Cause elimination
  4. Program Slicing
  5. Fault tree analysis

When the presence of one defect hides. At that time the presence of another defect in the system is known as fault masking.

Example: If the “Negative Value” cause a firing of unhandled system exception, the developer will prevent the negative values input. This will resolve the issue and hide the defect of unhandled exception firing.

A test plan can be defined as a document describing the scope, approach, resources, and schedule of testing activities and a test plan should cover the following details.

  • Test Strategy
  • Test Objective
  • Exit/Suspension Criteria
  • Resource Planning
  • Test Deliverables

It helps you to eliminate product risk in your project, and there is a simple yet crucial step that can reduce the product risk in your project.

  • Investigate the specification documents
  • Have discussions about the project with all stakeholders including the developer
  • As a real user walk around the website

The common risk that leads to a project failure are

  • Not having enough human resource
  • Testing Environment may not be set up properly
  • Limited Budget
  • Time Limitations

To estimate your project, you have to consider the following points

  • Divide the whole project into the smallest tasks
  • Allocate each task to team members
  • Estimate the effort required to complete each task
  • Validate the estimation
Task Member
  • Analyze software requirement specification
  • All the members
  • Create the test specification
  • Tester/Test Analyst
  • Build up the test environment
  • Test administrator
  • Execute the test cases
  • Tester, a Test administrator
  • Report defects
  • Tester

To get an expected test outcome, a standard procedure is followed which is referred to as Testing Type.

Commonly used testing types are

  • Unit Testing: Test the smallest code of an application
  • API Testing: Testing API created for the application
  • Integration Testing: Individual software modules are combined and tested
  • System Testing: Complete testing of the system
  • Install/UnInstall Testing: Testing done from the point of client/customer view
  • Agile Testing: Testing through Agile technique

The things that have to be taken in considerations are

  • Is your project on schedule
  • Are you over budget
  • Are you working towards the same career goal
  • Have you got enough resources
  • Are there any warning signs of impending problems
  • Is there any pressure from management to complete the project sooner
  • Matching resources to wrong projects
  • Test manager lack of skills
  • Not listening to others
  • Poor Scheduling
  • Underestimating
  • Ignoring the small problems
  • Not following the process

A test report contains the following things:

  • Project Information
  • Test Objective
  • Test Summary
  • Defect

The benefits of test reports are:

  • Current status of project and quality of product are informed
  • If required, stakeholder and customer can take corrective action
  • A final document helps to decide whether the product is ready for release

Management review is also referred to as Software Quality Assurance or SQA. SQA focusses more on the software process rather than the software work products. It is a set of activities designed to make sure that the project manager follows the standard process. SQA helps test manager to benchmark the project against the set standards.

The best practices for an effective SQA implementation is

  • Continuous Improvement
  • Documentation
  • Tool Usage
  • Metrics
  • Responsibility by team members
  • Experienced SQA auditors

RTM is prepared before test case designing. Requirements should be traceable from review activities.

Test Matrix: Test matrix is used to capture actual quality, effort, the plan, resources and time required to capture all phases of software testing

Traceability Matrix: Mapping between test cases and customer requirements is known as Traceability Matrix

Both stubs and drivers are part of incremental testing. In incremental testing, there are two approaches namely bottom-up and top-down approach. Drivers are used in bottom-up testing and stub is used for a top-down approach. In order to test the main module, the stub is used, which is a dummy code or program.

Once a defect is found you would follow the step

a) Recreate the defect

b) Attach the screenshot

c) Log the defect

This technique uses the actual test case document developed by testers using a spreadsheet containing special “key Words”. The key words control the processing.

When a “flow of data” through an information system is graphically represented, then it is known as Data Flow Diagram. It is also used for the visualization of data processing.

LCSAJ stands for ‘linear code sequence and jump.’ It consists of the following three items

a) Start of the linear sequence of executable statements

b) End of the linear sequence

c) The target line to which control flow is transferred at the end of the linear sequence

The variation of regression testing is represented as N+1. In this technique, the testing is performed in multiple cycles in which errors found in test cycle ‘N’ are resolved and re-tested in test cycle N+1. The cycle is repeated unless there are no errors found.

Fuzz testing is used to detect security loopholes and coding errors in software. In this technique, random data is added to the system in an attempt to crash the system. If vulnerability persists, a tool called fuzz tester is used to determine potential causes. This technique is more useful for bigger projects but only detects a major fault.

The benefit of statement coverage metric is that

a) It does not require processing source code and can be applied directly to object code

b) Bugs are distributed evenly through the code, due to which percentage of executable statements covered reflects the percentage of faults discovered

a) If characters in new string > characters in the previous string. None of the characters should get truncated

b) If characters in new string< characters in the previous string. Junk characters should not be added

c) Spaces after and before the string should not be deleted

d) String should be replaced only for the first occurrence of the string

  • I will talk individually to each person and note their concerns
  • I will find a solution to the common problems raised by team members
  • I will hold a team meeting, reveal the solution and ask people to co-operate

Mainly there are three defect categories

  • Wrong: When a requirement is implemented incorrectly
  • Missing: It is a variance from the specification, an indication that a specification was not implemented or a requirement of the customer is not met
  • Extra: A requirement incorporated into the product that was not given by the end customer. It is considered as a defect because it is a variance from the existing requirements

The code coverage testing tool runs parallel while performing testing on the actual product. The code coverage tool monitors the executed statements of the source code. When the final testing is done, we get a complete report of the pending statements and also get the coverage percentage.

In simple terms when a defect reaches the end customer, it is called a failure while the defect is identified internally and resolved; then it is referred to as a defect.

The project span across the software development lifecycle in the following manner

  • Central/Project test plan: It is the main test plan that outlines the complete test strategy of the project. This plan is used till the end of the software development lifecycle
  • Acceptance test plan: This document begins during the requirement phase and is completed at the final delivery
  • System test plan: This plan starts during the design plan and proceeds until the end of the project
  • Integration and Unit test plan: Both these test plans start during the execution phase and last until the final delivery

Black box test cases are written first as to write black box test cases; it requires project plan and requirement document all these documents are easily available at the beginning of the project. While writing white box test cases requires more architectural understanding and is not available at the start of the project.

  • Latent defect: A latent defect is an existing defect that has not caused a failure because the sets of conditions were never met
  • Masked defect: It is an existing defect that has not caused a failure because another defect has prevented that part of the code from being executed

Bottom-up testing is an approach to integration testing, where the lowest level components are tested first, then used to facilitate the testing of higher level components. The process is repeated until the component at the top of the hierarchy is tested.

Different types of test coverage techniques include

  • Statement Coverage: It verifies that each line of source code has been executed and tested
  • Decision Coverage: It ensures that every decision in the source code is executed and tested
  • Path Coverage: It ensures that every possible route through a given part of the code is executed and tested

Breath testing is a test suite that exercises the full functionality of a product but does not test features in detail

Code Walk Through is the informal analysis of the program source code to find defects and verify coding techniques

The essential components of defect report format include

  • Project Name
  • Module Name
  • Defect detected on
  • Defect detected by
  • Defect ID and Name
  • Snapshot of the defect
  • Priority and Severity status
  • Defect resolved by
  • Defect resolved on

End-to-end testing is done after functional testing. The purpose behind doing end-to-end testing is that

  • To validate the software requirements and integration with external interfaces
  • Testing application in real-world environment scenario
  • Testing of interaction between application and database

A test harness is configuring a set of tools and test data to test an application in various conditions, and it involves monitoring the output with expected output for correctness.

In testing project testing activities, you would automate are

  • Tests that need to be run for every build of the application
  • Tests that use multiple data for the same set of actions
  • Identical tests that need to be executed using different browsers
  • Mission critical pages
  • A transaction with pages that do not change in a short time

It helps prevent defects from being introduced into the code.

Risk-based Testing is the term used for an approach to creating a Test Strategy that is based on prioritizing tests by risk. The basis of the approach is a detailed risk analysis and prioritizing of risks by risk level. Tests to address each risk are then specified, starting with the highest risk first.

Preventative tests are designed early; reactive tests are designed after the software has been produced.

The purpose of exit criteria is to define when a test level is completed.

The likelihood of an adverse event and the impact of the event determine the level of risk.

Decision table testing is used for testing systems for which the specification takes the form of rules or cause-effect combinations. In a decision table, the inputs are listed in a column, with the outputs in the same column but below the inputs. The remainder of the table explores combinations of inputs to define the outputs produced.

The techniques of equivalence partitioning and boundary value analysis are often applied to specific situations or inputs. However, if different combinations of inputs result in different actions being taken, this can be more difficult to show using equivalence partitioning and boundary value analysis, which tend to be more focused on the user interface. The other two specification-based techniques, decision tables, and state transition testing are more focused on business logic or business rules. A decision table is a good way to deal with combinations of things (e.g., inputs). This technique is sometimes also referred to as a ’cause-effect’ table. The reason for this is that there is an associated logic diagramming technique called ’cause-effect graphing’ which was sometimes used to help derive the decision table

To identify defects in any software work product.

Metrics from previous similar projects and discussions with the development team

In order to measure test effectiveness, a powerful metric is used to measure test effectiveness known as DRE (Defect Removal Efficiency) From this metric we would know how many bugs we have found from the set of test cases. The formula for calculating DRE is

DRE=Number of bugs while a testing/number of bugs while testing + number of bugs found by a user

Failure is a departure from specified behavior.

Is it really a test if you put some inputs into some software, but never look to see whether the software produces the correct result? The essence of testing is to check whether the software produces the correct result and to do that, and we must compare what the software produces to what it should produce. A test comparator helps to automate aspects of that comparison.

An inexpensive way to get some benefit

Identifying test conditions and Identifying test cases

It avoids author bias in defining effective tests.

The exit criteria are determined on the bases of ‘Test Planning’.

Pre-release testing by end user representatives at the developer’s site.

Testing performed by potential customers at their own locations.

The difference between a pilot and beta testing is that pilot testing is actually done using the product by the group of users before the final deployment, and in beta testing, we do not input real data, but it is installed at the end customer to validate if the product can be used in production.

if width > length 
   thenbiggest_dimension = width
     if height > width 
             thenbiggest_dimension = height 
     end_if
elsebiggest_dimension = length  
            if height > length 
                thenbiggest_dimension = height 
          end_if
end_if

Testing technique: – Is a process for ensuring that some aspects of the application system or unit functions properly there may be few techniques but many tools.

Testing Tools: – Is a vehicle for performing a test process. The tool is a resource to the tester, but itself is insufficient to conduct testing

Component testing, also known as unit, module, and program testing, searches for defects in and verifies the functioning of software (e.g., modules, programs, objects, classes, etc.) that are separately testable. Component testing may be done in isolation from the rest of the system depending on the context of the development life cycle and the system. Most often stubs and drivers are used to replace the missing software and simulate the interface between the software components simply. A stub is called from the software component to be tested; a driver calls a component to be tested.

Testing the end to end functionality of the system as a whole is defined as a functional system testing.

Independent testers are unbiased and identify different defects at the same time.

The bulk of the test design work begun after the software or system has been produced.

There are currently seven different agile methodologies that I am aware of:

  1. Extreme Programming (XP)
  2. Scrum
  3. Lean Software Development
  4. Feature-Driven Development
  5. Agile Unified Process
  6. Crystal
  7. Dynamic Systems Development Model (DSDM)

A ‘Test Analysis’ and ‘Design’ includes evaluation of the testability of the requirements and system.

Random testing is often known as monkey testing. In such type of testing data is generated randomly often using a tool or automated mechanism. With this randomly generated input, the system is tested, and results are analyzed accordingly. These testing are less reliable; hence it is normally used by the beginners and to see whether the system will hold up under adverse effects.

  1. Provide developers and other parties with feedback about the problem to enable identification, isolation, and correction as necessary.
  2. Provide ideas for test process improvement.
  3. Provide a vehicle for assessing tester competence.
  4. Provide testers with a means of tracking the quality of the system under test.
  1. Equivalence Partitioning.
  2. Use Case Testing.
  3. Data Flow Analysis.
  4. Exploratory Testing.
  5. Decision Testing.
  6. Inspections.

Because they share the aim of identifying defects but differ in the types of defect they find.

In contrast to informal reviews, formal reviews follow a formal process. A typical formal review process consists of six main steps:

  1. Planning
  2. Kick-off
  3. Preparation
  4. Review meeting
  5. Rework
  6. Follow-up.

The moderator (or review leader) leads the review process. He or she determines, in co-operation with the author, the type of review, approach and the composition of the review team. The moderator performs the entry check and the follow-up on the rework, in order to control the quality of the input and output of the review process. The moderator also schedules the meeting, disseminates documents before the meeting, coaches other team members, paces the meeting, leads possible discussions and stores the data that is collected.

An input or output ranges of values such that only one value in the range becomes a test case.

Structure-based testing techniques (which are also dynamic rather than static) use the internal structure of the software to derive test cases. They are commonly called ‘white-box’ or ‘glass-box’ techniques (implying you can see into the system) since they require knowledge of how the software is implemented, that is, how it works. For example, a structural technique may be concerned with exercising loops in the software. Different test cases may be derived to exercise the loop once, twice, and many times. This may be done regardless of the functionality of the software.

After the software has changed or when the environment has changed Regression testing should be performed.

A negative test is when you put in an invalid input and receives errors. While positive testing is when you put in a valid input and expect some action to be completed in accordance with the specification.

The purpose of test completion criterion is to determine when to stop testing

For example memory leaks.

Re-testing ensures the original fault has been removed; regression testing looks for unexpected side effects.

In experience-based techniques, people’s knowledge, skills, and background are a prime contributor to the test conditions and test cases. The experience of both technical and business people is important, as they bring different perspectives to the test analysis and design process. Due to previous experience with similar systems, they may have insights into what could go wrong, which is very useful for testing.

Yes, because both help detects faults and improves quality.

The answer depends on the risk for your industry, contract and special requirements.

It depends on the risks for the system being tested. There are some criteria based on which you can stop testing.

  1. Deadlines (Testing, Release)
  2. Test budget has been depleted
  3. Bug rate fall below a certain level
  4. Test cases completed with certain percentage passed
  5. Alpha or beta periods for testing ends
  6. Coverage of code, functionality or requirements are met to a specified point

The primary purpose of the integration strategy is to specify which modules to combine when and how many at once.

Semi-random test cases are nothing, but when we perform random test cases and do equivalence partitioning to those test cases, it removes redundant test cases, thus giving us semi-random test cases.

Read p

Read q

IF p+q> 100

THEN Print “Large”

ENDIF

IF p > 50

THEN Print “p Large”

ENDIF

1 test for statement coverage, 2 for branch coverage

Because errors are frequently made during programming of the different cases near the ‘edges’ of the range of values.

It is led by a trained leader, uses formal entry and exit criteria and checklists.

Because configuration management assures that we know the exact version of the testware and the test object.

A software development model that illustrates how testing activities integrate with software development phases

Triggered by modifications, migration or retirement of existing software

Test coverage measures in some specific way the amount of testing performed by a set of tests (derived in some other way, e.g., using specification-based techniques). Wherever we can count things and can tell whether or not each of those things has been tested by some test, then we can measure coverage.

Because incremental integration has better early defects screening and isolation ability

To freeze requirements, to understand user needs, to define the scope of testing

We split testing into distinct stages because of the following reasons,

  1. Each test stage has a different purpose
  2. It is easier to manage to test in stages
  3. We can run different test into different environments
  4. Performance and quality of the testing is improved using phased testing

I am text block. Click edit button to change this text. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Ut elit tellus, luctus nec ullamcorper mattis, pulvinar dapibus leo.

The most important one must be tested first

The fault has been built into more documentation, code, tests, etc

It is a partial measure of test thoroughness.

Test boundary conditions on, below and above the edges of input and output equivalence classes. For instance, let say a bank application where you can withdraw maximum Rs.20,000 and a minimum of Rs.100, so in boundary value testing we test only the exact boundaries, rather than hitting in the middle. That means we test above the maximum limit and below the minimum limit.

Commercial Off The Shelf.

Rapid Application Development (RAD) is formally a parallel development of functions and subsequent integration. Components/functions are developed in parallel as if they were mini projects, the developments are time-boxed, delivered, and then assembled into a working prototype. This can very quickly give the customer something to see and use and to provide feedback regarding the delivery and their requirements. Rapid change and development of the product are possible using this methodology. However the product specification will need to be developed for the product at some point, and the project will need to be placed under more formal controls before going into production.