Introduction To Software Quality Assurance & Testing
Introduction To Software Quality Assurance & Testing
Introduction To Software Quality Assurance & Testing
• Focus
– Completeness, Consistency, Feasibility, Testability
Software Quality Assurance
• The two primary SQA activities:
– Technical Reviews
– Software Testing
Technical Reviews
• A “review” is a meeting where a work product is reviewed by a small group of people
who are qualified to give feedback, find problems, suggest improvements, etc.
• Anything can be reviewed: requirements spec, functional spec, design, code, test plan,
user documentation
• Reviews are the most effective QA technique, both in terms of cost and number of defects
discovered
Review Meetings
• Review the product – not the producer
• Set an agenda and keep it
• Limit debate and rebuttal
• Enunciate problem areas but don’t try to fix anything
• Take written notes
• Limit the number of participants
• Insist upon advance preparation
• Develop a checklist
• Allocate resources and time schedules
• Conduct meaningful training
Software Testing
• Testing is the process of detecting errors by running the actual software and
verifying that it works as it should
– Test cases, Expected results, Actual results
• Testing is by far the most popular QA activity (but not the most effective)
• Formal technical reviews are cheaper and more effective than testing, but are
often ignored
• Research has shown that all forms of testing combined usually find less than
60% of the errors present
• We must do partial testing because we only have enough resources (time and money) to
run relatively few test cases
– If the system passes all your test cases, there could still be defects, you just need
more or better test cases to find them
Software Testing
• Effective testing lies in intelligently choosing the relatively few test cases that will
actually be executed
– Test all requirements and features defined in the requirements spec. and functional
spec.
– Test cases should not be redundant (i.e., each one should follow a different path
through the code)
– Analyze the program’s design and code to find potential weak areas
– Analyze all points at which data enters the system and look for ways to attack it
– Code coverage
Software Testing
• Approaches for test case design are generally divided into two broad
categories: Black Box Testing and White Box Testing
• Effective testing requires the assumption that you will find defects
• If you think you won't find defects, or you don't want to, you will have set up a
self-fulfilling prophecy
• Needs to be automated
Regression Testing
Build results
Test data for version x
Build tests
for version x
Run tests
for version x Compare Verdict
Automating Regression Testing
• Challenging
• What parts of program output should be checked?
• Simple but annoying issues
– Use of dates in output
– Changes in whitespace
– Format changes
– Lead text changes
• Answer
– Don’t use complete output
– Just extract the relevant information
Regression Testing
• Keep it updated!
– Bug fixes – tests to ensure they stay fixed
– Functionality additions
– Platform changes
– Etc.
• If you branch the code, you must branch the regression tests
Formal Verification
• In addition to Technical Reviews and Software Testing, Formal Verification is another
approach to QA
• “Check” the model by formally proving that it implements the desired behavior
– Automated theorem proving systems are often applied
– Or, prove that the model does not behave correctly, thus revealing a defect
• Historically, formal verification has been expensive and limited to relatively small
programs, but techniques are improving all the time. Challenges include:
– Complex systems are hard to formalize
– State space explosion: real systems have so many possible states that proving
things about them is hard
– Ensuring that the “model” accurately captures the system’s behavior
– Making it accessible to people who aren’t formal verification experts
Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.
Alternative Proxies: