Unit Testing

It’s useful to think about software testing as a set of concentric rings, from small
to large. These rings represent the total scope of testing. Unit software testing is
the innermost ring with the smallest scope. Think of a software
developer building a piece of code, and then putting it through the paces to
make sure it works properly. The developer can test the unit in isolation to make
sure that if you provide the proper inputs (which the developer can simulate),
that you get the expected output.

For example, if the developer is coding the login feature for a website, a unit test
might be what happens when you feed some mock data into the module to
see whether the expected output happens (perhaps setting an authentication
token for the user).

Role of the Business Analyst in unit testing: The developer and maybe his
teammates are the ones driving the unit testing, so the role of other people is
minimal at this stage. However, the developer may have a question about the
output she is seeing. She may want to know how that matches against
business requirements. The Business Analyst would be the person to answer those
questions.

Integration Testing

The next, larger concentric ring is integration testing and is often done by a
separate testing team. At this point, the developer is reasonably confident the
software unit works as she intends. So now it’s time to connect it to real
interfaces that provide data from databases or other applications, and see
what happens then. Integration testing also ensures the software runs properly
on whatever hardware platform will support it.

To continue the website example, perhaps there is an interface that returns a

user’s name and address when they successfully log in; integration testing
makes sure the name and address passes correctly. It would also ensure the
software works properly on its platform (a web server, the cloud, etc.)

Role of the Business Analyst in integration testing: Just as with unit testing, the
developer is still in the driver’s seat here. And as with unit testing, the developer
may see unexpected results from the integration test. She may consult with the
Business Analyst to see whether the results meet business requirements; and if
not, then figure out how to change the code. If there are problems with the
interfaces themselves, the Business Analyst may have to clarify or add interface
requirements.

System Testing

Next up in scope is system software testing. At this point, the testing team wants
to know how the code module performs in the context of the entire system, on
the platform it will be running on. A system can include other applications
running concurrent with, prior to, or after execution of the tested code module.
A system is like a set of interlocking gears, where everything must run smoothly
together. System testing, then, makes sure that the new “gear” you are inserting
isn’t going to break something else along the way.

To continue the website development example, a system test might involve the
entire process of logging in to the website, pulling your name and address,
ordering a product, and paying for it. Everything must work together perfectly.

There are two important sub-sets of system testing to know about: Usability
testing and Regression testing.

What is usability testing and regression testing?

Usability testing is, obviously enough, how usable the whole system is. This often
involves business users interacting with a system interface (like a website) to see
whether it behaves the way the business wants it to. From a user’s perspective,
you don’t want any unexpected behavior and usability testing will ensure this to
be the case.

Regression testing compares the result of what the system would have done
BEFORE the new software changes with what it does AFTER the change. The
system’s gears may still work well together, but the new code could result in
unexpected output that is totally unrelated to the changes you actually wanted
to make. Software systems are complex things, and when you add more
complexity on top you will sometimes get unexpected results. Continuing the
website example, regression testing might detect that the website works, but
quotes a different price for a product compared to before. Since software code
for logging in should not affect the price of a product, this is a problem the
developer must fix.
Role of the Business Analyst in system testing. The Business Analyst must play a
flexible role in system software testing. Again, anytime there is unexpected
behavior, the Business Analyst can expect to provide answers based on
requirements. Since we’re getting closer to releasing the software, questions
about whether the software meets requirements are likely to increase. The
Business Analyst may also serve as the user for usability testing, to help ensure the
system does what a user would expect. For regression testing, the Business
Analyst may provide sample output (such as an older customer invoice) to
compare against the new output.

User Acceptance Testing: This last phase of testing is the one most likely to
heavily involve the Business Analyst. In user acceptance testing (or UAT), the
developer submits the software for final sign off and approval by the business. By
now, the developer and testing team have thoroughly vetted the code and
believe they have resolved all outstanding bugs. Once the business agrees the
code meets all requirements, the development team puts the software into its
final permanent location.

Role of the Business Analyst in user acceptance testing: The Business Analyst
plays two key roles in UAT. First, the Business Analyst may write high level test
cases. These are hypothetical end-to-end scenarios that lead to an expected
result that the Business Analyst documents. The testing team must put the system
through each hypothetical scenario, using real or mock data. The Business
Analyst then validates that the final result matches the expected one. Second,
the Business Analyst either provides the final sign off, or works to get it
from stakeholders with the authority to do so.

Regression Testing is considered a part of or support for UAT testing. However,

unlike the test cases for other types of testing, the test bed and outcome
benchmarks for Regression Testing can often be reused and remain applicable
from one project to another. The exception to this general “test data
recycles/reuse” rule is the need for frequent revision in order to stay current with
the latest system changes.

Other widely used testing types that are considered optional by many business
analysts are:

- Parallel Testing: Where testing results in an older system or a dedicated Test

Environment are to be similar to, if not exactly mirroring, related system
processes in a newer system or the Production Environment under the same
circumstances.
- This type of testing sometimes involves much larger test data volumes to
parallel real-time volume processing in a production environment.

- Stress Testing: This is conducted when observing abnormal circumstances

within a system or impacting its environment. Common examples include, but
are not limited to, overloaded network bandwidth and insufficient memory.
While we can establish many test scenarios to test such abnormal conditions, it is
wise to always consider such abnormal conditions within the realm of
reasonable hardware/software constraints.

User Acceptance Testing (UAT) is a phase in the software development life cycle
where the intended users of a system participate in validating that the solution
meets their needs. There are varying perspectives to the definition of UAT. While
some see the UAT as a “test of requirements”, others argue that the same
deficiencies missed in the framing of requirements will be missed during testing if
this philosophy is adopted.

Randall Rice provides a comprehensive objective of UAT: User Acceptance

Testing is conducted to assess if the system can support day-to-day business and
user scenarios and to ensure the system is sufficient and correct for business
usage.

Why Should Business Analysts Be Involved in UAT?

The Business Analyst Role is central to achieving success in UAT sessions.

Here are 6 reasons why:

1. BAs understand the functionality the system is supposed to deliver and as

such, have the knowledge needed to validate the system (confirm whether the
solution meets business needs or not). The fact that a system has been built to
specification does not make it automatically acceptable. UAT helps
stakeholders to determine whether the system can be put to use in real-life
business scenarios or not.

2. The UAT session is an opportunity for users to see the solution in action and
confirm that it meets their needs. Users need to test their own systems to ensure
that it works the way they expect it to, to prevent drastic changes after the
system has gone live and to increase the chances of project success.

Getting involved in UAT presents an opportunity for the BA to confirm the

correctness of previously elicited requirements and improve on future projects.

3. Even though it's recommended that users see prototypes of the proposed
system and are kept informed throughout the software development lifecycle,
change requests still come up during UAT sessions.

Being part of the UAT session will help the Business Analyst understand the
rationale for any proposed change, suggest the most appropriate way to meet
the business need and help in selling the idea to the Change Control Board.

4. Though the objective of UAT is not to train users, they would need to
understand how the system works to test it effectively. When BAs who have a
firsthand knowledge of system functionalities are involved in UAT, they are able
to support training efforts prior to UAT and answer users' questions during test
sessions.

5. As part of solution validation, the BA may be brought in to support the team in

assessing the severity of defects, their impact on the business, which defects
must be resolved before go-live and what can be done to mitigate the risks of
the defects that cannot be resolved.
6. Business Analysts are involved in determining the set of requirements the
solution must meet to be considered acceptable. In an ideal world, users are
expected to write their own test scripts but in reality, BAs usually have to support
this task or write the test scripts themselves.

Depending on the type of organization, BAs are involved in UAT sessions to

varying degrees. On most of my projects, I organize and manage test sessions,
write test cases, execute system tests and prepare the defect reports (I wear the
hat of a Tester). Once I'm done with the system tests which are usually more
exhaustive and technical than user acceptance tests, I invite the users for UAT
sessions. While BAs attached to organizational units may be less involved in
organizing and managing test sessions, they may be nominated to participate in
UAT along with users or to represent a user group. Such BAs are also expected to
be analytical, independent thinkers and knowledgeable enough about the
business to support system users and test the system in place of
stakeholders who are unavailable to do so themselves

Main Concepts of Testing: Testing Methods

After successfully defining the testing scope, we need to pinpoint each method
required to complete testing. Testing strategies come in various forms; however,
there are some that are widely used in most industries.

- Top-down: Approaching testing efforts with this method would allow the tester
to focus more on the top of a given system hierarchy. This is done by perceiving
the broad interface that is integrating various system modules or by observing
the surface of each of the main control modules.

- Bottom-up: Testing with this method would lead the tester to the bottom of a
system hierarchy of modules until reaching the top module or by reviewing each
unit within a single module. This testing approach is applicable whether a unit is
operating independently or requires functional integration with other units to
operate properly.

- Middle-out: This is simply a combination of the top-down and bottom-up

methods. This involves approaching the system from a mid-point of the module
hierarchy, while progressing towards both the top and bottom system modules
in a hierarchy.

- White-box: This method observes the flow of data, logic or the sequence of
such logic of a proposed system change. This is meant for systems developed in-
house, rather than commercial products or outsourced systems, where all the
internal components of a system are thoroughly understood and manageable.

- Black-box: All internal components and units of a module are disregarded as

the focus is only on the input and output into a given module. For example, if we
were to test a calculator using this testing approach, then we would enter
“1+1=” as the input and we anticipate receiving “2” as the output. Notice that
we are not testing the programming structure that allows digits such as “1” and
“2” to appear on the screen and that calculator functions such as “+” and “=”
don’t display on the screen. Neither are we testing the logic behind the “+” or
“=” calculator functions. With this testing method, we are purely testing input
and corresponding output.

- Gray-box: This is a hybrid approach that combines both the white-box and
black-box testing. In this case, both the function of a module and its internal
coding are being tested. This is ideal for large-scale system enhancements
developed in-house, or when major upgrades and system changes are
anticipated in the future.

Positive/negative testing: This approach focuses on the impacted system

changes by focusing only on how a system processes data. This means positive
testing would observe that the system processes a certain type of data as
expected and negative testing would observe the system to process a certain
type of data in a near opposite or unexpected manner. This is done to ensure
that the other types of data that are not meant to be impacted by a system
change, remain unaffected by how the system processes a specific type of
data.

Often it is the type of software release, system implementation or system

changes that dictate what testing strategy to consider. In fact, the vast majority
of successful test plans incorporate several testing strategies.