CCS366

SOFTWARE TESTING & AUTOMATION

UNIT 2

TEST PLANNING
 TEST PLANNING

• Test planning is the process of outlining the strategy, scope, objectives, resources, and schedule
for testing activities.
• It involves determining the test objectives, assigning roles and responsibilities, and identifying
the test deliverables.
• This ensures that testing is systematically organized and aligned with project goals to ensure
software quality.
• The test plan serves as the blueprint that changes according to the progressions in the project and
stays current at all times.
 GOALS OF TEST PLANNING

• Quick guide for the testing process: The test plan serves as a quick guide for the testing process as it
offers a clear guide for QA engineers to conduct testing activities.

• Helps to avoid out-of-scope functionalities: The test plan offers detailed aspects such as test scope,
test estimation, strategy, etc.

• Helps to determine the time, cost, and effort: The Test serves as the blueprint to conduct testing
activities thus it helps to deduce an estimate of time, cost, and effort for the testing activities.
• Provide a schedule for testing activities: A test plan is like a rule book that needs to be
followed, it thus helps to schedule activities that can be followed by all the team members.

• Test plan can be reused: The test plan documents important aspects like test estimation, test
scope, and test strategy which are reviewed by the Management Team and thus can be reused
for other projects.

• Environment Setup: Plan for the setup of test environments that mimic the production
environment and it will ensure all necessary tools and configurations are in place for testing.
OBJECTIVES OF THE TEST PLAN

• Overview of testing activities

• Provides timeline
• Helps to estimate resources
• Serves as a blueprint
• Helps to identify solutions
• Serves as a rulebook
 TYPES OF TEST PLANS

• Master Test Plan: Comprehensive plan covering all levels and types of testing for the
entire project.
• Unit Test Plan: Focuses on testing individual components or modules of the software.
• Integration Test Plan: Ensures that different modules or components work together as
expected
• System Test Plan: Validates the complete and integrated software system against the
specified requirements.
• Acceptance Test Plan: Confirms the software meets business requirements and is ready
for delivery to end users.
COMPONENTS AND ATTRIBUTES OF TEST PLAN
• Introduction: Overview of the test plan's purpose and scope.

• Objectives: Goals and objectives for testing activities.

• Scope: Defines what will and won't be tested.

• Test Approach: Strategy and methods to be used for testing.

• Resources and Roles: Team members and their responsibilities.

• Test Environment: Hardware and software needed for testing.

• Test Deliverables: Documents and reports to be produced.

• Schedule: Timeline for test activities.

• Risks and Mitigation: Potential issues and how to address them.

• Approval: Sign-off by stakeholders for plan validation.

 HIGH LEVEL EXPECTATIONS

• Comprehensive Coverage: Ensure all functionalities and requirements are tested.

• Resource Allocation: Proper assignment of roles, tools, and environments.
• Risk Management: Identify potential risks and establish mitigation strategies.
• Clear Objectives: Define specific, measurable goals for testing activities.
• Efficient Execution: Implement a structured approach to optimize test efforts.
• Detailed Documentation: Maintain thorough records of test plans, cases, and results.
• Stakeholder Communication: Regular updates and clear communication with parties.
• Quality Assurance: Ensure the software meets predefined quality standards and criteria.
 TEST PLANNING EXPECTATIONS

• Clear Test Objectives: Define what to test and why.

• Comprehensive Test Coverage: Test all aspects of the software.

• Realistic Test Schedule: Create a realistic testing timeline.

• Effective Resource Allocation: Allocate necessary resources efficiently.

• Risk-Based Testing: Focus on high-risk areas of the software.

• Test Environment Readiness: Ensure the test environment is ready.

• Test Data Management: Prepare relevant, accurate, and secure test data.

• Defect Management: Establish a process for defect reporting and resolution.

 DELIVERABLES

• Test Plan Document: A detailed document outlining the test plan, approach, and strategy

• Test Schedule: A project schedule outlining testing activities and timelines

• Test Cases and Scripts: A set of test cases and scripts to be executed during testing

• Test Environment Setup: A configured test environment ready for testing

• Test Data: A set of test data to support testing activities

• Defect Management Process: A process for reporting, tracking, and resolving defects
 INTERGROUP RESPONSIBLITIES
• Test Plan: A document that outlines the test strategy, objectives, scope, and resources required for
testing.

• Test Schedule: A timeline that outlines the start and end dates for each testing phase, including test
design, execution, and reporting.

• Test Cases: A document that outlines the steps required to test a specific feature or functionality.

• Test Data: Data required to execute test cases, including input and expected output data.

• Test Scripts: Automated test scripts used to execute test cases.

• Test Reports: Reports that summarize the results of testing, including the number of test cases
executed, the number of defects found, and the status of each defect.
 RESPONSIBLITIES OF TEST MANAGER

• Ensure all functionalities and requirements are tested.

• Proper assignment of roles, tools, and environments.
• Identify potential risks and establish mitigation strategies.
• Define specific, measurable goals for testing activities.
• Implement a structured approach to optimize test efforts.
• Maintain thorough records of test plans, cases, and results.
• Regular updates and clear communication with all involved parties.
• Ensure the software meets predefined quality standards and criteria.
 RESPONSIBILITIES OF A TEST LEAD

• Create detailed test cases and scenarios.

• Provide direction and support to the test team.
• Monitor and manage test execution activities.
• Track, report, and ensure resolution of defects.
• Monitor testing progress and report to the test manager.
• Ensure tests are performed to meet quality standards.
• Prepare and maintain test environments.
• Liaise between the test team and other project stakeholders.
 RESPONSIBILITIES OF A TESTER

• Create detailed test cases based on requirements.

• Run tests and compare results with expected outcomes.

• Identify and document software bugs and issues.

• Develop and maintain automated test scripts (if applicable).

• Re-test to ensure fixes don't introduce new issues.

• Maintain accurate records of test cases, results, and defects.

• Work with developers and other stakeholders to understand requirements and resolve issues.

• Ensure the software meets the specified quality standards.

 TEST PHASES

• Unit Testing: Test individual components or modules for functionality.

• Integration Testing: Ensure different modules or components work together correctly.
• System Testing: Validate the complete and integrated system against the specified requirements.
• Acceptance Testing: Confirm the software meets business requirements and is ready for delivery to
end users.
• Static Testing: Review and analyze code, documents, and requirements without executing the
software.
Test Reporting Phase:

• Document and report the test results.

• Provide metrics and analysis on test coverage, defect density, test pass/fail rates, etc.
• Share reports with stakeholders and conduct test review meetings.

Acceptance Testing Phase:

• Conduct user acceptance testing (UAT) to validate the system against business requirements.
• Ensure the system is ready for production deployment.
• Obtain formal sign-off from stakeholders.

By defining and planning these phases, the testing process becomes more organized, predictable,
and manageable, leading to higher quality software and a more efficient testing cycle.
EXECUTION OF TEST PHASES
 BENEFITS OF TEST PHASES
The benefits of dividing the testing process into distinct test phases are
numerous, contributing to the overall effectiveness, efficiency, and
quality of the software development lifecycle. Here are the key
benefits:

• Improved Test Coverage

• Early Detection of Defects
• Better Resource Allocation
• Enhanced Quality Assurance
• Improved Communication and Collaboration
 TEST STRATEGY
A test strategy is a high-level document outlining the overall approach and principles for testing,
including objectives, methods, resources, and timelines to ensure software quality and compliance
with requirements.
Test Strategy document:
A test strategy document is a well-described document that is derived from actual business
requirements that guide the whole team about the software testing approach and objectives for each
activity in the software testing process.

• The test strategy document is approved and reviewed by the test team lead, development
manager, quality analyst manager, and product manager.
• The test strategy document specifies the resources, scope, plan, and methodology for different
testing activities.
• It answers all the questions like what needs to get done, how to accomplish it, etc.
TESTING STRATEGIES
COMPONENTS OF A TEST STRATEGY
• Scope: Defines what will be tested and what won't.

• Objectives: Specifies the goals and purpose of testing.

• Test Approach: Describes the testing methods and levels to be used.

• Resources: Identifies the tools, environments, and personnel needed.

• Schedule: Provides a timeline for testing activities.

• Risk Management: Lists potential risks and mitigation strategies.

• Deliverables: Details the documents and reports to be produced.

• Communication Plan: Outlines how and when updates will be shared with stakeholders.
 TEST STRATEGY VS TEST PLAN
S No. Test Plan Test Strategy

1. It’s developed from a set of software requirements It comes from a Business Requirement paper (BRS).
(SRS).

2. The test lead or manager is in charge of preparing it. The project manager or the business analyst creates it.

3. The test plan’s components include the test plan’s id, The components of a test strategy include objectives and scope,
features to be tested, test techniques, testing tasks, documentation formats, test processes, team reporting structure,
features pass or fail criteria, test deliverables, client communication strategy, and so on.
responsibilities, and timetable, among others.

4. After the requirements have been approved, the test plan The test strategy comes first, followed by the test plan.
is written.

5. The test plan should be simple and straightforward. The test approach serves as a general guide for the project at
hand.
 TYPES OF TEST STRATEGIES

• Analytical Strategy: Based on risk analysis and requirements.

• Model-Based Strategy: Uses models to design and execute tests.

• Methodical Strategy: Follows predefined procedures and checklists.

• Process-Oriented Strategy: Focuses on improving testing processes.

• Standard-Compliant Strategy: Adheres to industry standards and regulations.

• Reactive Strategy: Responds to testing needs as they arise during development.

• Consultative Strategy: Relies on input from experts and stakeholders.

• Regression-Averse Strategy: Emphasizes minimizing regression risks with extensive retesting.

 RESOURCE REQUIREMENTS

Resource requirements for test planning in software testing involve identifying the necessary resources to
complete the testing project successfully. These resources include:

• Human Resources: Testers, developers, and other team members required for testing.
• Hardware and Software Requirements: The necessary infrastructure, such as computers, devices, and
software, to support testing.
• Test Environment: A realistic environment that mimics the production environment, including
hardware, software, and network configurations.
• Tools and Equipment: Any additional tools or equipment required to support testing, such as test
management tools, defect tracking tools, and automation frameworks.

It's essential to plan and allocate these resources effectively to ensure the testing project is completed on
time, within budget, and meets the required quality standards.
 RESOURCE PLANNING ACTIVITIES

• Resource Estimation: estimating the resources required for testing

• Resource Allocation: assigning resources to specific testing activities

• Resource Scheduling: scheduling resources to ensure timely completion of testing

activities

• Resource Monitoring: tracking resource utilization and adjusting plans as needed

• Onboard New Members: Integrate new team members into the project, providing
necessary information and resources.
 BENEFITS OF RESOURCE PLANNING
• Efficient Resource Utilization: ensures that resources are used effectively and
efficiently

• Improved Testing Quality: ensures that testing is thorough and comprehensive

• Reduced Testing Time and Costs: optimizes testing activities to reduce time
and costs

• Enhanced Collaboration: ensures that all stakeholders are aware of resource

requirements and plans

• Efficient Allocation: Ensures that resources (human, hardware, software) are

allocated efficiently, preventing overuse or underuse.
EXAMPLE OF RESOURCE PLANNING
 TEST ASSIGNMENT

• Tester assignments refer to the process of allocating test cases or test

scripts to individual testers or test teams.

• The goal of tester assignments is to ensure that each test case is executed
by a qualified tester who has the necessary skills, knowledge, and
resources to complete the testing task efficiently and effectively.

• Test assignment involves designating specific testing tasks and

responsibilities to team members to ensure a structured and efficient testing
process.

• By providing a clear framework for task distribution, test assignment

contributes significantly to the overall success of the testing process and
the quality of the final product.
• Test Case Development: Create test cases based on requirements and specifications.
• Test Data Management: Prepare and manage test data for testing.
• Test Environment Setup: Set up and configure the test environment.
• Test Scripting: Create automated test scripts using tools like Selenium or Appium.
• Manual Testing: Execute manual tests to identify defects and report results.
• Defect Reporting: Identify, report, and track defects found during testing.
• Monitor: Continuously monitor the progress of assigned tasks.
• Review: Adjust assignments as needed based on progress and feedback.
• Test Automation: Develop and execute automated tests to increase testing efficiency.
• Exploratory Testing: Perform exploratory testing to identify defects not caught by scripted
tests.
• Regression Testing: Execute regression tests to ensure changes haven't introduced new
defects.
• Test Result Analysis: Analyze test results to identify trends and areas for improvement.

By following these detailed steps, the test assignment process can be well-organized,
ensuring that all aspects of the testing lifecycle are covered efficiently and effectively. This
structured approach helps in managing the workload, enhancing team collaboration, and
delivering high-quality software products.
 KEY ASPECTS OF TEST ASSIGNMENT
• Allocating test cases based on tester skills and expertise

• Balancing workload among testers

• Considering test environment availability

• Assigning high-risk test cases to experienced testers

• Managing test case dependencies

• Considering tester availability and test script complexity

• Evaluate the skills and experience of each team member.

• Establish realistic deadlines and milestones to track progress and ensure timely
completion.
 BENEFITS OF TEST ASSIGNMENT

• Improved testing efficiency and productivity

• Better utilization of tester skills and expertise

• Reduced testing time and costs

• Increased test coverage and quality

• Improved communication and collaboration among testers

• Clear assignments promote better coordination among team members.

• Standardized assignments ensure consistent testing practices and

adherence to best practices.
 TEST SCHEDULE
• A Test Schedule is a detailed plan outlining the timeline and milestones for testing activities in
a software development project.
• It defines when and how testing will be performed, including the allocation of resources,
tasks, and deadlines. It includes
• Start and End Dates
• Testing Phases
• Task Assignments
• Duration and Effort
• Dependencies
• Milestones
• Resource Allocation
EXAMPLE
 PURPOSE OF TEST SCHEDULER

• Ensure testing is completed on time and within budget.

• Coordinate testing activities with other project activities, such as development and deployment.

• Allocate resources effectively and efficiently.

• Identify and mitigate risks that could impact testing.

• Provide a clear understanding of testing progress and milestones.

 TYPES OF TEST SCHEDULERS

• Manual Test Schedulers: Uses spreadsheets, calendars, or other manual tools to

schedule testing activities.

• Automated Test Schedulers: Uses software tools, such as test management tools or
project management tools, to automate testing activities.

• Hybrid Test Schedulers: Combines manual and automated scheduling techniques to

optimize testing activities.
 BENIFITS AND CHALLENGES
BENEFITS:
• Improved Test Coverage
• Better Resource Utilization
• Enhanced Collaboration
• Reduced Testing Time

CHALLENGES:

• Dynamic Test
Environments
• Test Case
Interdependencies
• Resource Constraints
• Complexity of Test Cases
 TEST CASE
A Test Case is a detailed, step-by-step description of a specific testing scenario, including the
expected results, used to validate that a software application or system meets its requirements
and works as expected.

Parameters
• Test Case ID
• Test Case Description
• Preconditions
• Steps
• Expected Results
• Test Data
• Postconditions
 WHEN DO WE WRITE TEST CASES?

• Before development: Test cases could be written before the actual coding as that would help to
identify the requirement of the product/software and carry out the test later when the
product/software gets developed.
• After development: Test cases are also written directly after coming up with a product/software
or after developing the feature but before the launching of a product/software as needed to test
the working of that particular feature.
• During development: Test cases are sometimes written during the development time, parallelly.
so whenever a part of the module/software gets developed it gets tested as well.
 WHY WRITE TEST CASES?

• To check whether the software meets customer expectations

• To check software consistency with conditions
• Narrow down software updates
• Better test coverage
• For consistency in test execution
• Helpful during maintenance
 TYPES OF TEST CASE

• Functional Test Cases: Test the functionality of the system, ensuring it performs as expected.
• Non-Functional Test Cases: Test the system's performance, security, usability, and other non-
functional aspects.
• Positive Test Cases: Test the system with valid inputs and expected results.
• Negative Test Cases: Test the system with invalid inputs or unexpected scenarios to ensure it
handles errors correctly.
• Edge Case Test Cases: Test the system with extreme or boundary values to ensure it handles
unusual scenarios.
TEST CASE EXAMPLE
 BUG REPORTING

What is bug?

• A malfunction in the software/system is an error that may cause components or the system
to fail to perform its required functions.
• For example, incorrect data description, statements, input data, design, etc.

Why Bugs Occur?

• Lack of Communication
• Repeated Definitions Required
• Policy Framework Does Not Exist
• Performance Errors
• Lots of Recycling
 BUG REPORTING IN SOFTWARE TESTING

• Bug reporting is an essential part of software testing, where testers identify and document defects
or bugs found in the software application or system.
• The goal of bug reporting is to provide detailed information about the bug to the development
team, enabling them to reproduce, diagnose, and fix the issue.

Why is Bug Reporting Important?

• Improves Software Quality

• Reduces Defect Density
• Enhances Customer Satisfaction
• Saves Time and Resources
FACTORS TO BE CONSIDERED WHILE REPORTING A BUG

• The whole team should clearly understand the different conditions of the trauma before starting
research on the life cycle of the disability.
• To prevent future confusion, a flawed life cycle should be well documented.
• Make sure everyone who has any work related to the Default Life Cycle understands his or her
best results work very clearly.
• Everyone who changes the status quo should be aware of the situation which should provide
sufficient information about the nature of the feature and the reason for it so that everyone
working on that feature can easily see the reason for that feature.
• A feature tracking tool should be carefully handled in the course of a defective life cycle work to
ensure consistency between errors.
 TOOLS FOR BUG REPORTING

• JIRA: A popular project management and bug tracking tool.

• Trello: A visual project management tool that can be used for bug tracking.
• Bugzilla: A free and open-source bug tracking system.
• TestRail: A test management tool that includes bug tracking features.
• Microsoft Azure DevOps: A comprehensive development platform that includes bug
tracking and project management features.
 METRICS AND STATISTICS
• Metrics and statistics play a crucial role in test planning, as they help testers measure the
effectiveness and efficiency of their testing efforts.
• By collecting and analyzing data, testers can identify areas for improvement, optimize their testing
processes, and make informed decisions about resource allocation.

Cost Scope
Quality

Schedule
 KEY FACTORS OF METRICES

• Quantifiable: Software testing metrics should be measurable and quantifiable, allowing for
objective analysis and comparison.

• Relevant: Metrics should be relevant to the testing process and provide valuable insights
into the testing activities.

• Actionable: Metrics should be actionable, enabling teams to make informed decisions and
take corrective actions to improve the testing process.

• Reliable: Metrics should be reliable and consistent, ensuring that the data collected is
accurate and trustworthy.

• Valid: Metrics should be valid, measuring what they are intended to measure, and providing
a true representation of the testing process.
• Simple and Easy to Understand: Metrics should be simple and easy to
understand, avoiding complexity and ensuring that stakeholders can comprehend
the data.

• Comparable: Metrics should be comparable, allowing teams to compare data

across different testing cycles, projects, or teams.

• Timely: Metrics should be timely, providing real-time or near-real-time data to

support agile testing and rapid decision-making.

• Cost-Effective: Metrics should be cost-effective, minimizing the overhead of data

collection and analysis.

• Flexible: Metrics should be flexible, adapting to changing testing requirements and

environments.
 TYPES OF TEST METRICS

Product Metrics: Measure product quality e.g. defect density defect leakage.

Process Metrics: Measure testing process efficiency e.g. test coverage test effectiveness.

Project Metrics: Measure project progress and e.g. test schedule variance test cost variance.
resource allocation

Quality Metrics: Measure defect rates and resolution e.g. defect rate mean time to detect.
times

Test Automation Metrics: Measure automation e.g. automation coverage automation ROI.
effectiveness and efficiency

User Experience Metrics: Measure user satisfaction e.g. user satisfaction net promoter score.
and experience

Test Environment Metrics: Measure test environment e.g. test environment test data quality.
availability and utilization availability