Inspection As Verification Adversarial Testing Testbench Design Timing Verification

Verification Techniques for Macro Blocks (IP)
Overview Inspection as Verification Adversarial Testing Testbench Design Timing Verification
Overview of Macro Verification

Overall verification plan Strategies Sub-block simulation Macro simulation Prototyping Limited production
Overall Verification Plan

Functional verification based on

Requirements Specifications Behavioral models Extended functions (for generalization and re-use)
Engineering review of verification plan Documentation of verification steps
Strategy: Bottom Up Testing

Verify individual sublocks
Focus on coverage of data values and internal states

Focus on interfaces and timing among sub-blocks Test I/O timing and corner cases Proto-board PC board FPGA Platform based design*
Verify complete macro (block)

Perform prototyping
Verification vs. Validation

In the hardware world:

Verification means testing Validation means formal proof Verification means formal proof Validation means testing
In the software world:

Dont be confused
Verification vs. Manufacturing Testing

Verification is to confirm that the design is correct

It meets the requirements, specifications It meets timing, power, and area goals It meets the re-use goals
It assumes a debugging cycle

Manufacturing Test

Fixed set of tests (vectors from above) Go/NoGo tests
Fundamental Ideas
Controllability
Can you wiggle every wire? Can you set every bit? Can you see the state of every bit? Can you see the value on every wire?
Observability

During validation, we can do this in the simulator especially in a bottom-up mode (In manufacturing test this is not so easy)
Types of Verification Tests

Compliance Testing
Against specifications and/or standards

Try to break the design with complex scenarios Answer Yes or No [Y]? > fred Use statistical models to generate data patterns Work with the software team, use real data Keep all tests in a suite and add to them
Corner Case Testing

Random Testing
Real Code Testing
Regression Testing
Testing in the Product Development Cycle

More complete tests find more subtle bugs Fixes often make design more complex Test time goes up New bugs are introduced by fixes Repeat
bugs found
time
Verification Tools
Simulation
functional, cycle, RTL, gate level, circuit level

Verisity (and others) testing language Measurements of what got wiggled Comparison against / with real hardware High speed hardware boxes interfaced to simulator
Testbench automation tools
Code coverage tools
Hardware Modeling
Emulation
Prototyping
Static Tools
Static timing analysis tools Lint-like tools Formal verification tools
Model checking
Build abstract model of design and environment Uses smart exhaustive methods for safety and liveness
tests

Liveness: something good will eventually happen Safety: nothing bad will ever happen
Theorem proving
Uses automatic theorem provers to verify assertions
about the design
Inspection as Verification
The fastest, cheapest, and most effective way to detect and remove bugs is by careful inspection of the design and the code Design reviews
Large group review requirements, specifications, assumptions and overall design Smaller groups perform line-by-line review of implementation
Code reviews
Reviews must be done in a collegial atmosphere with the goal of enhanced quality, not to asses designer performance
Prototyping
Use Prototyping when:
You can afford it

$cost $time
You can not wait for the time it takes to find bugs using available simulation techniques
The prototype is just a fast simulator for the
device itself
Adversarial Testing
The designers tests have the goal to show that the system will work
Adversarial Testing uses a verification team of experts in testing all kinds of designs Goal: develop tests to show that the system will fail Report failures, and test cases to design team
Sub-Block Simulation
1. Use automated response checking
Not appropriate for designer to check waveforms by eye How many times will it be done? Use of automated test to build macro block testbench Use assertions in code to test assumptions
2. Test sub-blocks for 100% statement and

path coverage
Testbench
The testbench For each component

Is the model of its external world Must generate (all) legal input sequences Must check output sequences Must alert the designer and log discrepancies between expected and actual results
The testbench is often more complex than component itself
Sub-block Testbench Design
Input Transaction Generator
Device Under Test (DUT)
Output Transaction Checker
Assume simple I/O relationships Input transaction data types, sequences Output transaction data types, sequences
Input Stimulus
Generate all legal sequences of inputs Look at functionality of DUT Check corner cases Achieve 100% coverage
Output Checking
Check for legal sequences of outputs Look at functionality of DUT Check corner cases Check input/output relationships
Ideally each sub-block has

Simple Functionality Simple I/O
Getting ready for integration

Timing checks

Timing plan for each sub-block Clocking, handshaking Timing budgets, per-cycle latency budget Are the sub-blocks within their bugets
Area and power checks
Concurrent Design, Synthesis & Testbench Development

Test Team Design Team Synthesis Team
Macro Specifications / Behavioral Model
Functional Specifications, Documentation, Behavioral Model
Develop MacroTest Bench, Test Cases
Specify Blocks Code and Test
Coordinate Timing Synthesize
Macro Partitioning and Block Specifications Block Design
Block Integration
Test Full Block
Synthesize Full Block
Validate against Specification and Test Plan / Productization
Integration Steps
Integrating the sub-blocks Top level netlist Final synthesis Final testing Manufacturing testability Scan insertion ATPG Test coverage
Timing Verification
Post synthesis timing
more real delays

Delay models based on statistical models No layout Not real delays
Check for clock/data skew Signal loading, critical path analysis
Post Synthesis Test Bench Development

Refine the Behavioral Model for testing and verification
Wrappers for each sub-block as needed (for signal compatibility before/after synthesis)
After synthesis every wire is a bit
Document test plan and test vectors
Macro Testbench
Design is more complex

Functional complexity I/O Complexity
More people involved Testbench will be delivered with product
Testbench will be used to develop scan path and manufacturing tests
Generating Testbenches
Not simple do scripts Complex state machines:
Loop
Generate stimulus
Get output Check results
Log results
Change stimulus parameters
Simple Example
Simulate Busses

Check bus timing Check data ranges

Clock/Timing Generator
DUT
Simulate ROM
Address Bus

Register File

Data Bus
I/O Bus
Also, supply data values Check address ranges Also, r/w timing Overwriting data Also, response to asynchronous events Response to out of range data
ROM Memory Register File
I/O

I/O Testbench
Help!
Use existing models for bench components
Have a test team, with their own test library Take library components and enhance them with checking/logging functions
Bus functional models
Need to have statistical bounds on input/output timing Use random generators for stimulus sequences
Use behavioral models for bench
Use C/C++ models and co-simulation for bench Need to have a master bench controller to coordinate I/O behaviors on all components
Use the Behavioral Model

Synthesized Block
Input Transaction Generator

Behavioral Model
Output Transaction Checker
Use Commercial Tools

Vera - Synopsis Specman Elite Verisity
Both tools use abstract data types to allow user to define data-ranges, states, and scenarios. Tools support statistical models, for stimulus, checking, and logging (beyond the scope of this class)
Test, Test, Test

Back to Requirements and Specifications Corner Cases Random Cases Regression Testing Code Coverage
Types of Coverage
Statement Coverage
# times each statement executed

Each side of each if/then else executed Checks each boolean clause in branch condition Paths between blocks (if if, if else, else if) Each signal on sensitivity list 1/0 transition on every signal
Branch Coverage
Condition Coverage
Path Coverage
Trigger Coverage
Toggle Coverage
Still Not Enough

Does not verify that code works right, just that it was run Does not verify that design meets specifications or requirements Does not check results of optimization and synthesis tools

Inspection As Verification Adversarial Testing Testbench Design Timing Verification

Uploaded by

Copyright:

Available Formats

Inspection As Verification Adversarial Testing Testbench Design Timing Verification

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Inspection As Verification Adversarial Testing Testbench Design Timing Verification

Uploaded by

Copyright:

Available Formats

Verification Techniques for Macro Blocks (IP)

Overview Inspection as Verification Adversarial Testing Testbench Design Timing Verification

Overview of Macro Verification

Overall Verification Plan

Engineering review of verification plan Documentation of verification steps

Strategy: Bottom Up Testing

Focus on coverage of data values and internal states

Verify complete macro (block)

Verification vs. Validation

In the software world:

Verification vs. Manufacturing Testing

It assumes a debugging cycle

Fixed set of tests (vectors from above) Go/NoGo tests

Types of Verification Tests

Against specifications and/or standards

Corner Case Testing

Real Code Testing

Testing in the Product Development Cycle

functional, cycle, RTL, gate level, circuit level

Testbench automation tools

Code coverage tools

about the design

You can afford it

2. Test sub-blocks for 100% statement and

The testbench is often more complex than component itself

Sub-block Testbench Design

Input Transaction Generator

Device Under Test (DUT)

Output Transaction Checker

Ideally each sub-block has

Getting ready for integration

Area and power checks

Concurrent Design, Synthesis & Testbench Development

Functional Specifications, Documentation, Behavioral Model

Develop MacroTest Bench, Test Cases

Specify Blocks Code and Test

Coordinate Timing Synthesize

Macro Partitioning and Block Specifications Block Design

Test Full Block

Synthesize Full Block

Validate against Specification and Test Plan / Productization

more real delays

Check for clock/data skew Signal loading, critical path analysis

Post Synthesis Test Bench Development

Document test plan and test vectors

Functional complexity I/O Complexity

More people involved Testbench will be delivered with product

Testbench will be used to develop scan path and manufacturing tests

Check bus timing Check data ranges

ROM Memory Register File

Use behavioral models for bench

Use the Behavioral Model

Input Transaction Generator

Output Transaction Checker

Use Commercial Tools

Test, Test, Test

# times each statement executed

Still Not Enough