Software Testing

UNIT IV – Test Management and Tools

TEST PLANNING
What is a Test Plan?
A test plan is a document that consists of all future testing-related
activities. It is prepared at the project level and in general, it defines work
products to be tested, how they will be tested, and test type distribution among
the testers. Before starting testing there will be a test manager who will be
preparing a test plan. The test plan serves as the blueprint.

Why is Test Plan creation important?

The following are some of the key benefits of making a test plan:
Defines Objectives
A test plan clearly outlines the testing objectives and the scope of testing
activities, ensuring that all team members understand what needs to be achieved.
Structured Approach
It provides a systematic approach to testing, detailing the steps and
processes involved, which helps in organizing the testing effort.
Avoids Scope Creep
By defining what will and will not be tested, the test plan helps manage the
scope of testing activities, preventing unnecessary work and focusing on
irrelevant areas.
Resource Allocation
Helps in identifying the necessary resources, including personnel, tools,
and environments, ensuring they are available when needed.

1
Identifies Risks
A test plan identifies potential risks and outlines mitigation strategies,
helping to address issues proactively rather than reactively.
Contingency Plans
These include contingency plans for dealing with unexpected events or
issues that may arise during testing.
Stakeholder Alignment
Facilitates communication among stakeholders, including developers,
testers, project managers, and clients, ensuring everyone is aligned on the testing
objectives, approach, and schedule.
Documentation
Serves as a comprehensive document that can be referred to by all team
members, aiding in knowledge sharing and transparency.
Resource Optimization
Helps in efficiently utilizing available resources, including time and
personnel, by providing a clear plan of action.
Focus on Priorities
Ensures that testing efforts are focused on high-priority areas that are
critical to the success of the project.

2
Difference between Test Strategy and Test Plan

COST BENEFIT ANALYSIS OF TESTING

Cost Benefit analysis is thing that everyone must do so as to think of a
powerful or an efficient system. But while thinking out on cost and benefit
analysis, we also need to find out factors that really affect benefits and costs of
system. In developing cost estimates for a system, we need to consider some of
cost elements. Some elements among them are hardware, personnel, facility,
operating and supply cost.

3
The following are the cost factors:
Hardware Cost
Hardware cost includes actual purchase and peripherals (external devices)
that are connected to computer. For example, printer, disk drive etc. Actually,
finding actual cost of hardware is generally more difficult especially, when
system is shared by various users so as to compared to a system which dedicated
stand alone. In some case, best way is to treat it as operating cost.

Personnel Cost
Personnel costs includes EDP staff salaries and benefits as well as pay for
those who are involved in process of development of system. Cost occurred
during development of system which are one time costs and are also called
development cost. Once system is installed, cost of operating and maintaining
system becomes recurring cost that one has to pay very frequently based on
requirement.

Facility Cost
Facility cost is amount of money that is spent in preparation of a site that is
physical where application or computer will be in operation. This includes
wiring, flooring, lighting and air conditioning. These costs are treated as one-
time costs and are included into overall cost estimate of candidate system.

Operating Cost
These includes all costs associated with day-to-day (everyday) operation
of system and amount depends on number of shifts, nature of applications. There
are various ways of covering operating costs.

4
Supply Cost
Supply cost are variable costs that increase with increased use of paper,
disks and like. They should be estimated and included in overall cost of system.

MONITORING AND CONTROL

TEST CONTROL
What is Test Monitoring?
Test Monitoring is a software testing stage, that involves continuous
supervision and assessment of testing activities to verify their efficiency and
alignment with project goals.
This phase involves:
Vigilant observation of Test Execution Progress,
Test Coverage
Defect identification, and
Defect resolution.

5
 Test monitoring yields valuable insights into the software’s quality under
examination, aids in the detection of potential risks, and enables timely
adjustments to testing strategies or resource allocations.
Test Monitoring is a continuous procedure that ensures:
Testing endeavors (activities) stay on course,
Issues are promptly dealt with, and
Project stakeholders are well-informed about the testing status.

What is Test Control?

Test control involves managing and regulating the software testing process
to ensure its effectiveness and alignment with project objectives.
It involves:
Establishing guidelines and processes that provide a structured framework
for testing.
Setting standards that dictate the testing process and help ensure
consistency and adherence to best practices.
Deciding resource distribution, encompassing personnel, tools, and
environments, to facilitate and bolster testing activities.
Observing testing activities progress, which encompasses test execution,
coverage, identification, and resolution of defects.
Enhancing testing process to guarantee efficiency and effectiveness in the
delivery of a top-notch product.

Why is Test Monitoring and Test Control is important?

By using Test monitoring, organizations can:
Significantly improve software quality
Accelerate time to market, and
6
 Gain a competitive edge.

Test Monitoring & Test Control Metrics

Test monitoring & Test Control provides a wealth of data through various
metrics, offering valuable insights into the health and performance of your
software under test.
Some metrics are:
Test Execution metrics
Resource Utilization metrics
Performance metrics
Advanced metrics

Test Execution Metrics

Test execution metrics are quantitative measures used to assess and analyze
various aspects of the testing process during the actual execution of test cases,
providing insights into the progress, efficiency, and quality of the testing efforts.
Test Execution Metrics include
Pass/Fail Ratio
Test Execution Time
Defect Detection Rate
Test Case Coverage

Resource Utilization Metrics

Resource utilization metrics refer to quantitative measures that assess the
efficiency and effectiveness of resources, such as personnel, tools, and
infrastructure, in the context of a project or process, helping to optimize resource
allocation and utilization.
7
Resource Utilization Metrics include
CPU/Memory usage
Network traffic
Database load

Performance Metrics
Performance metrics in the context of software or system performance
typically include measures such as response time, throughput, resource utilization,
and error rates, providing insights into the efficiency and effectiveness of the
system under various conditions.
Performance Metrics include
Response time
Throughout
Error rate

Advanced Metrics
Apart from key metrics, there are some metrics that you can track for a
deeper understanding of how your testing processes are working.
Advanced Test Metrics include
Code coverage by lines or branches
Mutation testing
Machine learning-based anomaly detection

TEST REPORTING
Test Reporting
Like any project, the testing also should be driven by a plan. The test plan
generates the report for the execution and tracking of the entire testing project.
8
Preparing the test plan
What needs to be the tested - the scope of testing, including clear
identification of what will be the tested & what will not be tested.
How the testing is going to be performed – breaking down the testing into
small and manageable tasks and identifying the strategies to be used for
carrying out the tasks.
Resource needed for testing
The timelines by which the testing activities will be performed.
Risks that may be faced in all of the above, with appropriate mitigation and
contingency plans.
Test Case Specification
Using the test plan as the basis, the testing team design test case
specification which then becomes the basis for preparing for individual test cases.
A test case is nothing but a series of step executed on a product, using a
predefined set of input data, expected to produce a pre-defined set of outputs, in a
given environment.
Test case specifications are useful as it enlists the specification details of
the items.
Test Case Specification Description
Test Case ID(TC_ID)
Unique ID to identify/report the bug if present in
the functionality of software

Test Case Objective The purpose of the test. The lists can be generated
to perform intended task, for which software is
developed. Results should always follow the test
case objective.

9
Pre-requisite This can include environment setup, supporting
software environment setup, for the project, or any
fields in which user will give the input. So that test
cases can be planned accordingly.
Steps This includes steps to be performed to give the
input to the system, so that system can perform its
specified task and display the result accordingly.
Input Data
The choice of input data will be depended on the
test case itself. and the technique followed in the
test case.
For e.g. Equivalence Partitioning, Boundary
Value Analysis etc.

Expected Result It will be the user required output to be shown

Actual Result
This step should do a comparison of the
expected and actual results to highlight any
differences.

Status
Whether expected results and actual result
match, if it matches then PASS or else FAIL

SPECIALIZED TESTING
What is Specialized testing?
Specialized testing refers to specific types of software testing focused on
particular aspects or requirements of an application, such as performance, security,
usability, or compatibility.

10
Refer notes of System and Performance Testing, Usability Testing of Unit II
for Specialized testing

OBJECT ORIENTED TESTING

Software typically undergoes many levels of testing, from unit testing to
system or acceptance testing. Typically, in-unit testing, small “units”, or modules
of the software, are tested separately with a focus on testing the code of that
module. In higher-order testing (e.g, integration testing), the entire system is
tested with a focus on testing the functionality or external behavior of the system.
As information systems are becoming more complex, the object-
oriented paradigm is gaining popularity because of its benefits in analysis,
design, and coding. Conventional testing methods cannot be applied for testing
classes because of problems involved in testing classes, abstract classes,
inheritance, dynamic binding, message passing, polymorphism, concurrency, etc.

Techniques of object-oriented testing are as follows:

Fault Based Testing
This type of testing involves creating test cases based on user
specifications, code, or both, to identify potential faults that could cause errors.
Each fault gets a corresponding test case to reveal these errors, ensuring all code
paths are executed. However, this approach doesn't catch all error types,
particularly specification and interface errors. Scenario-based testing in object-
oriented models can identify interaction errors.
Class Testing Based on Method Testing
This approach is the simplest approach to test classes. Each method of the
class performs a well defined cohesive function and can, therefore, be related to

11
unit testing of the traditional testing techniques. Therefore all the methods of a
class can be involved at least once to test the class.
Random Testing
It is supported by developing a random test sequence that tries the
minimum variety of operations typical to the behavior of the categories.
Partition Testing
This methodology categorizes the inputs and outputs of a category so as to
check them severely. This minimizes the number of cases that have to be
designed.
Scenario-based Testing
It primarily involves capturing the user actions then stimulating them to
similar actions throughout the test.

Purpose of Object Oriented Testing

Object Interaction Validation
Check to make sure objects interact with one another appropriately in
various situations. Testing makes ensuring that the interactions between objects
in object-oriented systems result in the desired results.
Determining Design Errors
Find the object-oriented design’s limitations and design faults. Testing
ensures that the design complies with the desired architecture by assisting in the
identification of problems with inheritance, polymorphism, encapsulation and
other OOP concepts.
Finding Integration Problems
Evaluate an object’s ability to integrate and communicate with other
objects when it is component or subsystem. This helps in locating integration
difficulties, such improper method calls or issues with data exchange.
12
Assessment of Reusable Code
Evaluate object-oriented code’s reusability. Code reuse is promoted by
object-oriented programming via features like inheritance. Testing ensures that
reusable parts perform as intended in various scenarios.
Verification of Handling Exceptions
Confirm that objects respond correctly to error circumstances and
exceptions. The purpose of object-oriented testing is to make sure that the
software responds carefully and is durable in the face of unforeseen occurrences
or faults.

AUTOMATED TOOLS FOR TESTING

Automated testing is the use of software tools to run tests on a program
automatically, without manual intervention. It helps check if the software behaves
as expected, quickly identifies bugs, and saves time compared to manual testing.
We have various types of automation testing tools available.
Some of the most commonly used automation testing tools are as follows:
Selenium
Watir
QTP
Telerik Studio
Testim
Selenium
It is an open-source and most commonly used tool in automation testing.
This tool is used to test web-based applications with the help of test scripts, and
these scripts can be written in any programming language such
as java, python, C#, Php, and so on.

13
Features of Selenium
Selenium supports only web-based application, which means that the
application can be opened by the browser or the URL like Gmail, Amazon,
etc.
Selenium does not support a stand-alone application, which means that the
application is not opened in the browser or URL like Notepad, MS-Word,
Calculator, etc.
Selenium web driver is the latest tool in the selenium community that
removes all the drawbacks of the previous tool (selenium-IDE).
Selenium web-driver is powerful because it supports multiple programming
languages, various browsers, and different operating systems and also
supports mobile applications like iPhone, Android.

Watir
Watir stands for web application testing in ruby which is written in
the Ruby programming language. It is a web application testing tool, which is
open-source and supports cross-browser.

Features of Watir
It supports various browsers on different platforms like Google Chrome,
Opera, Firefox, Internet Explorer, and Safari.

14
 Watir is a lightweight and powerful tool.
We can easily download the test file for the UI.
We can take the screenshots once we are done with the testing, which helps
us to keep track of the intermediate testing.
This tool has some inbuilt libraries, which helps to check the alerts, browser
windows, page-performance, etc.

QTP
QTP tool is used to test functional test cases of the web-based application.
QTP stands for Quick Test Professional, and now it is known as Micro Focus
UFT [Unified Functional Testing]. This is very helpful for the new test engineer
because they can understand this tool in a few minutes. QTP is designed on the
scripting language like VB script to automate the application.

Features of QTP
This tool support record and playback feature.
Both technical and non-technical tester can use QTP.
QTP supports multiple software development environments like Oracle,
SAP, JAVA, etc.
With the help of QTP, we can test both desktop and web-based applications.

15
Telerik Studio
It is modern web application which support functional test automation. With
the help of this tool, we can test the load, performance, and functionality of the
web and mobile applications and also identify the cross-browser issues.

Features of Telerik test studio

Telerik test studio allows us to deliver quality products on time.
This tool supports all types of applications, such as desktop, web, and
mobile.
This tool supports Asp.Net, AJAX, HTML, JavaScript and Silverlight
application testing.
It supports multiple browsers like Firefox, Safari, Google Chrome, and
Internet Explorer for the test execution process.

Testim
It is another automation testing tool, which can execute the test case in very
little time and run them in various web and mobile applications. This tool will help
us to enhance the extensibility and stability of our test suites. It provides the
flexibility to cover the functionalities of the platform with the help
of JavaScript and HTML.

16
Features of Testim
This tool will support parallel execution.
In this tool, we can capture the screenshots.
This tool will automatically create the tests.
With the help of this tool, we can perform requirements-based and
parameterized testing.

TOOL SELECTION AND IMPLEMENTATION

When selecting a software testing tool, several essential factors should guide
your decision:
Cost
Evaluate budget constraints and align them with end-user needs to choose a
suitable solution.
Compatibility
Ensure the tool can effectively execute test cases across various platforms,
saving time and effort.
Ease of Use
Opt for a user-friendly tool to streamline testing processes and minimize
learning curves.

17
Technical Support
Look for comprehensive support options such as FAQs, live chat, and
documentation to address any issues promptly.
Community Support
Engage with professional communities to stay updated on industry trends as
well as best practices in software testing.

Essential Factors in Choosing a Software Test Automation Tool

Selecting the right software test automation tool hinges on several critical
factors tailored to your project's needs:
Project Requirements
Define specific project needs, such as application type (desktop, API,
mobile, web), supported platforms, and required testing types like cross-browser
compatibility.
Team's Skills and Learning Curve
Consider your team's proficiency level—whether they need codeless or
coding-based tools—and evaluate available training resources to bridge skill gaps.
Budget Considerations
Align tool costs with your project's budget, factoring in training time,
automation efficiency, and potential savings in human resources.
Ease of Maintenance and Test Case Creation
Opt for tools with user-friendly interfaces and minimal maintenance
requirements. Look for customization options to tailor test cases to specific needs.
Reusability and Data-Driven Testing
Prioritize tools that support the reusability of scripts across environments
and enable efficient data-driven testing for comprehensive application validation.

18
Reporting Mechanism
Ensure robust reporting capabilities with features like error stack traces,
screenshots, and customizable templates for clear insights into test results.
24/7 Support and Training
Select tools offering extensive training resources, round-the-clock customer
support, and advanced learning opportunities to maximize team efficiency and
expertise.

By prioritizing these factors, you can efficiently optimize your testing

workflows and ensure the successful implementation of automated testing in your
development cycles.

Choosing the right testing tools in software development is crucial for

ensuring high-quality software delivery. Here's a curated list of leading software
testing and test automation tools currently in demand:
1. QA Wolf: QA Wolf combines automated testing tools with professional QA
services, achieving rapid end-to-end test coverage for web applications.
Key Features
Accelerates test coverage to 80% within weeks through parallel execution.
Eliminates flaky tests with proactive maintenance.
Charges based on achieved test coverage, not hourly rates.
Tests web and native mobile applications, including performance and
accessibility.

2. aqua cloud: aqua cloud is an AI-powered test management solution that

enhances QA efficiency with 100% traceability and seamless integration with
leading automation frameworks.
19
Key Features
AI-driven test management for rapid test case generation.
QA-tuned chatbot for faster test creation and validation.
Enhanced test coverage control and bug reporting capabilities.
Productivity-boosting dashboards and customizable reports.

3. Ranorex: Ranorex Studio is a GUI test automation framework for web-based,

desktop, and mobile applications.
Key Features
Automates tests on Windows desktop and executes on real or virtual
machines.
Supports parallel testing for cross-browser compatibility.
Tests on real iOS or Android devices, simulators, and emulators.

4. TestProject: TestProject is a free platform for end-to-end automation of web,

mobile, and API testing, built on Selenium and Appium.
Key Features
Saves tests locally for offline access without a cloud footprint.
Enables codeless tests with self-healing capabilities and community add-ons.
Provides detailed reports with screenshots and logs for comprehensive
insights.

CHALLENGES IN TEST AUTOMATION

Today, test automation has become a new trend in the tech industry when
it comes to efficiency and quickness. Undoubtedly it helps in achieving
milestones more quickly and on the other hand, also boosts productivity. In fact,

20
most IT firms are now shifting towards automation. But, as we all know,
anything can have pros and cons and in the field of software development.

Today, from small to large, every firm is adopting automation testing and it
directly allows them to let their resources work on other parts. So, it becomes
very crucial to understand the pain points of having automation testing in your
organization.

7 challenges in test automation:

1. Team Collab Issues
It is necessary to keep your team in a loop and to ensure that they all are in
the same alignment. With the growing age of technology, firms are now moving
towards automation and that has become a challenge for QAs. The reason being
they do lack a skill set and are not much well aware of the backend codes. Thus,
this becomes very challenging for them to work with the script.
Therefore, this issue can be solved by keeping your whole team in a loop.
Besides, some other practices can be also be taken into considerations:
By providing proper training so that they can be well aware of the outlook.
By creating a common communication channel.

2. Network Issues
During the automation testing, it is being observed that network stability
& disconnection has been one of the most challenging issues for QAs. The
dedicated team needs to have a good and stable connection, and if the connection
is poor, it becomes hard to access databases, third-party services, etc.

21
Some other issue that occurs with the network is:
Disruption in the network also blocks QA to access virtual
environments that are used for testing.
Any delay in testing, specifically due to network issues can cause delays in
the whole development cycle.
In the development cycle, the whole system fails if testing goes wrong.
This happens due to its architecture.

3. Cost Factor
You might not be surprised that test automation has become a very crucial
part of software testing. Besides automation comes at a costly price especially
during its initial phase. So, if you’re running on a tight budget, you might need
to reconsider it again because the implementation is going to cost you on a higher
side which also includes license cost for the software. Likewise, going for
an open-source solution will lead you to train and maintenance costs.

4. Right Expectations
It is a must to understand that expectations should always be right either
it’s for selecting testing tools or the methods you’re going to implement or
maybe the outcome you’re expecting. No matter how much automation we’re
going to make but human interaction and involvement are necessary and will play
a very crucial role in fixing bugs and running tests.
Therefore, for non-automation testing, humans will be required, and not to
forget the most important thing is ‘A testing phase gets success not because of the
number of executions, but its success defines by the number of successful
compilations done with a meaningful outcome.’

22
5. Testing Accuracy
If you try to run processes on outdated data, then it will show you the false
outputs which are the most common mistakes in automation. That’s why it is
very crucial for a team to be aligned in a singular channel which can help them in
having clear communication to avoid any such chaos. In
Automation, DevOps needs a quick response from testers in order to get more
accuracy over the data. Thus, data relevancy and accuracy are the only factors to
achieve in testing accuracy.

6. Selection of Tool
The selection of the right tool is the most crucial part of the testing phase.
Lack of expertise can hamper the whole development cycle. Generally, typical
testing consists of functional, unit, integration, etc. Besides the fact, the
challenging part is to decide whether your project needs Automation or Manual
Testing. Make sure to dig into these considerations before selecting any tool for
your project:
Gather information about your project and its requirement.
Technical support & assistance.
Check for cross-browsing testing compatibility.
Ease in maintenance
Cost

7. Fixing Code smell

It is very important to address and fix up a bad code smell from any frame.
Why is it so crucial to fix code smell? A Code smell is a part of a code that is not
“technically wrong” but its implementation is chaotic (messy) and becomes a
pain during the execution. So, it’s a responsibility for both programmers and
23
testers to take care of the quality of code so that it doesn’t affect the development
cycle.

GUI TESTING
What is GUI Testing?
Graphical User Interface Testing (GUI) Testing is the process for ensuring
proper functionality of the graphical user interface (GUI) for a specific
application. GUI testing generally evaluates a design of elements such as layout,
colors and also fonts, font sizes, labels, text boxes, text formatting, captions,
buttons, lists, icons, links, and content. GUI testing processes may be either
manual or automatic.

Feature of Graphical User Interface Testing (GUI)

It provides a customizable test report.
It is run tests in parallel or distributed on a Selenium Grid with built-in
Selenium Webdriver.
It allows you to test the functionality from a user’s perspective.
Sometimes the internal functions of the system work correctly but the user
interface doesn’t then GUI testing is good to have in addition to the other
types.

Types of Graphical User Interface Testing (GUI) Testing

There are two types of GUI testing which are given below:
Analog Recording, and Object based recording.

24
Analog Recording
This is always what people associate with GUI testing tools by analog
recording, the testing tool basically captures specific mouse clicks, keyboard
presses, and other user actions and then simply stores them in a file for playback.
For example, it might record that a user left-clicked at position X = 500 pixels, Y
= 400 pixels or typed the word “Search” in a box and pressed the [ENTER] key
on their keyboard.

Object based Recording

In object based recording, the testing tool is able to connect
programmatically to the application being tested and “see” each of the individual
user interface components (a button, a text box, a hyperlink) as separate entities
and is able to perform operations (click, enter text) and read the state (is it
enabled, what is the label text, what is the current value) reliably regardless of
where that object is on the screen.

Objectives of GUI Testing

Functionality: Ensure all interactive elements work as expected.
Usability: Assess how user-friendly and intuitive the interface is.
Consistency: Check for uniformity across different screens and platforms.
Performance: Measure response time and resource usage under load.
Compatibility: Validate the interface across different devices, browsers, and
operating systems.

Common GUI Testing Tools

Selenium: Open-source tool for automating web applications.

25
Test Complete: Comprehensive GUI testing tool for desktop, web, and mobile
applications.
QTP/UFT (Quick Test Professional/Unified Functional Testing): Automated
functional testing tool.
Ranorex: Supports GUI testing for various applications, including desktop and
mobile.
Robot Framework: Open-source automation framework that supports GUI testing
through various libraries.

26