SQT_Unit-2
SQT_Unit-2
SQT_Unit-2
➢ The test plan serves as the blueprint that changes according to the progressions in
the project and stays current at all times.
➢ It serves as a base for conducting testing activities and coordinating activities among
a QA team.
➢ It is shared with Business Analysts, Project Managers, and anyone associated with
the project.
Why are Test Plans Important:
The following are some of the key benefits of making a test plan:
➢ Quick guide for the testing process: The test plan serves as a quick guide for the
testing process as it offers a clear guide for QA engineers to conduct testing
activities.
➢ Helps to avoid out-of-scope functionalities: The test plan offers detailed aspects
such as test scope, test estimation, strategy, etc.
➢ Helps to determine the time, cost, and effort: The Test serves as the blueprint to
conduct testing activities thus it helps to deduce an estimate of time, cost, and effort
for the testing activities.
➢ Provide a schedule for testing activities: A test plan is like a rule book that needs
to be followed, it thus helps to schedule activities that can be followed by all the
team members.
➢ Test plan can be reused: The test plan documents important aspects like test
estimation, test scope, and test strategy which are reviewed by the Management
Team and thus can be reused for other projects.
Objectives of the Test Plan:
1. Overview of testing activities: The test plan provides an overview of the testing
activities and where to start and stop the work.
2. Provides timeline: The test plan helps to create the timeline for the testing
activities based on the number of hours and the workers needed.
3. Helps to estimate resources: The test plan helps to create an estimate of the
number of resources needed to finish the work.
4. Serves as a blueprint: The test plan serves as a blueprint for all the testing
activities, it has every detail from beginning to end.
5. Helps to identify solutions: A test plan helps the team members They consider
the project’s challenges and identify the solutions.
6. Serves as a rulebook: The test plan serves as a rulebook for following rules
when the project is completed phase by phase.
Types of Test Plans:
The following are the three types of test plans:
➢ Master Test Plan: In this type of test plan, includes multiple test strategies and has
multiple levels of testing. It goes into great depth on the planning and management
of testing at the various test levels and thus provides a bird’s eye view of the
important decisions made, tactics used, etc. It includes a list of tests that must be
executed, test coverage, the connection between various test levels, etc.
➢ Phase Test Plan: In this type of test plan, emphasis is on any one phase of testing. It
includes further information on the levels listed in the master testing plan.
Information like testing schedules, benchmarks, activities, templates, and other
information that is not included in the master test plan is included in the phase test
plan.
➢ Specific Test Plan: This type of test plan, is designed for specific types of testing
especially non-functional testing for example plans for conducting performance tests
or security tests.
Test Coverage:-
Test coverage is the degree to which a test or set of tests exercises a particular
program or system. The more code that is covered by a test, the more confidence
developers have that the code is free of bugs. Measuring test coverage can be
difficult because it is often hard to determine what percentage of the code is actually
being executed by a test. In general, however, the more code that is covered by a
test, the better.
There are many different types of test coverage, but in general, test coverage
is a measure of how much of the code or functionality of a system is being tested by
a particular set of tests. For example, if a set of tests only covers 50% of the code,
then it has 50% coverage. There are many different ways to measure test coverage,
and the level of coverage that is considered acceptable varies from organization to
organization. In some cases, 100% test coverage may be required, while in others,
80% may be considered adequate.
One way to think of test coverage is as a spectrum, with lower levels of
coverage being less effective and higher levels being more effective. However, it is
important to remember that no single level of coverage is right for all situations, and
the level of coverage that is appropriate will depend on the specific system under test
and the risks involved.
Purpose of Test Coverage
The purpose of test coverage is to ensure that all code paths in a program are
executed at least once during testing. This helps to ensure that all bugs and potential
problems are found and fixed before the program is released. Test coverage can be
measured in a number of ways, but the most common metric is line coverage, which
simply measures the percentage of lines of code that are executed during testing.
➢ Cover all possible code paths: All code paths should be covered in a test in
order to ensure that all parts of the code are executed at least once. This will
help to find potential bugs that may be present in the code.
➢ Important functionality should be covered: All important functionality
should be covered in a test in order to ensure that it is working correctly. This
includes ensuring that all inputs are handled correctly and that all outputs are
as expected.
➢ Potential bugs should be fixed before release: All potential bugs should be
covered in a test in order to ensure that they are found and fixed before the
code is released.
➢ All potential edge cases should be covered: All potential edge cases should
be covered in a test in order to ensure that the code behaves as expected in all
situations.
➢ All potential error conditions should be considered: All potential error
conditions should be covered in a test in order to ensure that the code behaves
as expected in all situations.
➢ Executed all tests at least once: All tests should be executed at least once in
order to ensure that they are actually being run and that they are working as
expected.
➢ Covered all possible input values: All possible input values should be
covered in a test in order to ensure that the software can handle them
correctly.
➢ Considered all possible combinations of input values: All possible
combinations of input values should be covered in a test in order to ensure
that the software can handle all potential combinations correctly.
Test Coverage Metrics
There are many different ways to measure test coverage, but some common metrics
include:
➢ Lines of code covered: This metric simply measures the number of lines of
code that are covered by tests. This is a good starting point, but it doesn’t give
the whole picture, since some lines of code are more important than others.
➢ Blocks covered: This metric measures the number of blocks of code (e.g., if-
statements, for-loops, etc.) that are covered by tests. This is a more granular
metric than lines of code, but it still doesn’t give the whole picture.
➢ Functions/methods covered: This metric measures the number of functions
or methods that are covered by tests. This is a more granular metric than
blocks, but it still doesn’t give the whole picture.
➢ Statements covered: This metric measures the number of statements that are
covered by tests. This is the most granular metric, but it still doesn’t give the
whole picture.
➢ Conditions covered: This metric measures the number of conditions (e.g., if-
statements, boolean expressions, etc.) that are covered by tests.
Types of Test Coverage
1. Statement Coverage: A test suite is said to achieve statement coverage if
every statement in the software has been executed at least once during
testing. This type of test design coverage ensures that every statement in
the software has been executed at least once. This is the most basic form
of test design coverage and is often used as a starting point for more
comprehensive testing.
2. Decision Coverage: A test suite is said to achieve decision coverage if
every decision (e.g. if-then-else) in the software has been taken both true
and false during testing. This type of test design coverage ensures that
every decision (i.e. every ‘if’ or ‘case’) in the software has been executed
with all possible outcomes. This is a more comprehensive form of testing
than statement coverage and can help to uncover errors that may not be
obvious from simply executing every statement.
3. Condition Coverage: A test suite is said to achieve condition coverage if
every condition in the software has been evaluated as both true and false
during testing. This type of test design coverage ensures that all the
conditions within a decision have been executed with all possible
outcomes. This is a more comprehensive form of testing than decision
coverage and can help to uncover errors that may not be obvious from
simply executing every decision.
4. Multi-Condition Coverage: A test suite is said to achieve multi-condition
coverage if all combinations of conditions in the software have been
evaluated during testing. This type of test design coverage extends
condition coverage by testing all the conditions within a decision with all
possible combinations of values. This is a more comprehensive form of
testing than condition coverage and can help to uncover errors that may
not be obvious from simply executing every condition.
5. Modified Condition: A test suite is said to achieve modified conditions if
all modified conditions and decisions in the software have been evaluated
during testing. This type of test design coverage extends decision coverage
by testing all the conditions within a decision with all possible outcomes,
taking into account the effect of modifying the conditions. This is a more
comprehensive form of testing than decision coverage and can help to
uncover errors that may not be obvious from simply executing every
decision.
6. Data Flow Coverage: A test suite is said to achieve data flow coverage if
all executable paths through the software have been executed during
testing. This type of test design coverage ensures that all the possible paths
through the software have been executed. This is a more comprehensive
form of testing than all the other types of coverage and can help to
uncover errors that may not be obvious from simply executing the
software.
Benefits of Test Coverage
➢ Reduced bugs: Test coverage can help reduce the number of bugs in your
code. By testing your code, you can find and fix bugs before your users do.
➢ More reliable code: Code that has been tested is more reliable than code that
has not been tested. By testing your code, you can find and fix bugs before
your users do.
➢ Easier to find bugs: Test coverage can help you find bugs in your code. By
testing your code, you can find and fix bugs before your users do.
➢ Faster debugging: By testing your code, you can find and fix bugs before
your users do. This can save you time when debugging your code.
➢ Better code quality: Code that has been tested is typical of higher quality
than code that has not been tested. By testing your code, you can find and fix
bugs before your users do.
➢ Reduced time to market: By testing your code, you can find and fix bugs
before your users do. This can save you time when releasing your code to the
public.
➢ Ensure that all parts of the code are executed at least once: This is
important because it helps to find bugs that might otherwise go unnoticed.
➢ Prevent regressions: This is important because it helps to ensure that new
code does not break old code.
➢ Ensure that new code doesn’t break old code: This is important because it
helps to maintain the stability of the codebase.
Limitations of Test Coverage
➢ Time-consuming and expensive to set up and maintain: In order to get
reliable test coverage, a lot of time and effort must be put into setting up the
tests and making sure they are effective. This can be costly, both in terms of
money and time.
➢ Difficult to interpret: The results of test coverage can be difficult to
understand and interpret. This can make it difficult to know how effective the
tests are and what needs to be improved.
➢ False sense of security: Having high test coverage can give a false sense of
security, as it can make it seem like the system is more robust than it actually
is. This can lead to complacency and a lack of vigilance, which can ultimately
lead to problems.
➢ Can miss edge cases: No matter how good the test coverage is, there is
always the possibility that it will miss some edge cases. This can lead to
problems that only occur in rare circumstances, which can be difficult to find
and fix.
➢ Expensive to calculate and maintain: Test coverage can be expensive to
calculate and maintain because it requires running tests and then analyzing
the results. This can be time-consuming and may require special tools.
Additionally, if the test coverage changes frequently, it can be difficult to
keep up with the changes.
➢ Hard to interpret: Test coverage can be hard to interpret because it is a
measure of how many tests were run and how many lines of code were
covered. It does not necessarily indicate how effective the tests were at
finding bugs.
➢ Hide problems: Test coverage can be used to hide problems because it can
make it seem like the code is better tested than it actually is. This can lead to
problems if the tests are not effective at finding bugs.
➢ Make things seem better than they are: Test coverage can be used to make
things seem better than they are because it can make it seem like the code is
better tested than it actually is. This can lead to problems if the tests are not
effective at finding bugs.
Test Evaluation:-
Test Evaluation Report (TER) is a document that contains a summary of all the testing
activities, methods used for testing, and a summary of the final test results of a Software
project. TER is prepared after the completion of testing and the Test Summary Report and
provides all the necessary information regarding software testing to the developers and the
key stakeholders. These stakeholders can then evaluate the quality of the tested product and
make a decision on the software release.
➢ Conducts analysis and assessment of Test Summary Report (TSR), source codes, test
results, and the measures used for product testing.
➢ Enables objective evaluation and assessment of product quality.
➢ Consists of corresponding recommendations that may be required for the next testing
efforts.
➢ Validates that no bug or error was missed by the tester.
➢ Reviews important change requests and issue details.
➢ Assess the product quality before it moves ahead in the development process.
➢ Allows the team members to effectively monitor each and every testing phase of the
development process.
➢ Test Evaluation Reports are essential for making sure that the product under
development is achieving an acceptable level of quality before it is released to the
market.
➢ Stakeholders and customers can take corrective actions if needed for future
development processes.
➢ Done right; this can add true value to your development lifecycle by providing the
right feedback at the right time.
1. Project Information
All the information regarding the project and the customer, such as Project Name, Customer
Name, and Project No. etc., is mentioned under this section. For a Change Request (CR), the
CR number can be mentioned as well.
2. Introduction
The introduction section can consists of the following:
➢ Purpose: This describes the purpose of the Test Evaluation in terms of test coverage
and defect analysis.
➢ Scope: This describes the scope of this document; associated project(s), and other
items of interest to the test team.
➢ Definitions: This can contain definitions, abbreviations, and acronyms required to
interpret the Test Evaluation document.
➢ References: This identifies the documents referenced in the Test Evaluation
Summary by title, report number, date, and author.
➢ Overview: This describes the rest of the Test Evaluation document and how the
document is organized.
3. Test Results
Test Results are summarized in this section. Test Results are generally the outcome of the
whole process of the Software Testing Life Cycle. The produced results offer an insight into
the deliverables of a software project which represents the project status to its stakeholders.
4. Test Coverage
Test Coverage is covered in this section which includes both the Requirements-based Test
Coverage and the Code-based Test Coverage. Test Results of both the coverages are
mentioned here and are compared with the previous test results.
5. Recommendations
This section identifies any suggested actions that need to be made based on the evaluation of
the test results. These recommendations help the developers/stakeholders to understand and
work accordingly for the next phase in the development life cycle.
6. Diagrams and Graphs
Under this section, diagrams, charts, graphs, or other data visualization of the test results can
be added. This helps in better debugging and root cause analysis
Example:
➢ Concise and Clear: The information captured in the test report should be short, clear,
and easy to understand.
➢ Detailed: The report should provide detailed information about the testing activities
whenever and wherever necessary. The information provided should not be abstract as
it won’t help the stakeholders in drawing a clear picture of it.
➢ Standard: The report should follow a standard template as it is easy for stakeholders
to review and understand.
➢ Specific: The report should describe and summarize the test result specification and
focus on the main point only.
Acceptance Test:-
Acceptance Testing is a method of software testing where a system is tested for
acceptability. The major aim of this test is to evaluate the compliance of the system with the
business requirements and assess whether it is acceptable for delivery or not. Standard
Definition of Acceptance Testing:
Acceptance Testing is the last phase of software testing performed after System Testing and
before making the system available for actual use.
Reporting test execution results is an essential part of Testing; whenever the test
execution cycle is complete, the tester should make a full test results report, including the test
cycle’s Test Pass/Fail status.
If manual Testing is done, the test pass/fail result should be captured in an Excel sheet. If
automation testing uses an automation tool, the HTML or XML reports should be provided to
stakeholders as test deliverables.
Test results are the outcome of the whole software testing life cycle process. The results thus
produced offer an insight into the deliverables of a software project, significant in
representing the project’s status to the stakeholders.
Few concepts:
Testing: Identifying whether a bug/error hides within a project and assessing the
observation’s impact. It forms an essential activity of Quality Assurance.
Debugging: This method involves identification and then correction of bugs/errors. When
developers encounter an error in the code, they debug. Debugging is thus a part of unit
testing.
Test Case: A test case is a document that consists of test data, preconditions, postconditions,
and expected results that are developed for a specific kind of test scenario intended for
serving a particular purpose.
Test Suite: It is a collection of test cases aimed at testing a software application to detect that
the application adheres to the requirement specifications. It consists of a detailed set of
instructions to attain a common goal.
Two broad categories/types of Testing help to obtain the test results most appropriately. They
are as follows :
Manual Testing is the process of Testing comprising a group of testers who examine the
code for the presence of a bug. The Testing is performed without the use of any tool. The
tester tests the application just as an end-user would do to find defects, if any.
Automation Testing- Automating the test process is done with the help of a script or tool. A
piece of code is used to detect a bug/error.