SOC Verification Methodology

http://www.cse.psu.edu/~vijay/verify/protected/Lec tures.html

Agenda
Verification Overview Verification Strategies Tools for Verification SoC Verification Flow

Terms in verification
Verification Validation Testing Test bench Test vector Scenario DUT Equivalence checking Checker Comparator Monitor Bug Debug Bug fix Close Reference

Verification Terminology
EDA: Engineering Design Automation Tool vendors. Synopsys, ModelTech, Cadence, etc. Behavioral: Code written to perform the function of logic on the interface of the DUV. Macro: 1) A behavioral. 2) A piece of logic. Driver/Agitator/Stimulator/Generator/Bus Functional Model (BFM): Code written to manipulate the inputs of the DUV. Typically this is behavioral code. It understands the interface protocols.

Verification Terms (continued)

Checker: Code written to verify the outputs of the DUV. A checker may have some knowledge of what the driver has done. A check must also verify interface protocol compliance. Snooper/Monitor: Code that watches interfaces or internal signals to help the checkers perform correctly. Also can be used by drivers to be more stressful and adaptive.

Verification Terms (continued)

Architecture: Design criteria as seen by the customer. Designs architecture is specified in documents usually a specification in which the design must be compliant with (verified against) Micro Architecture: The designs implementation. It refers to the constructs that are used in the design (pipelines, caches, etc).

Hardware Functional Verification

By: John Goss Verification Engineer IBM gossman@us.ibm.com

Other References
Text References:
Writing Testbenches: Functional Verification of HDL Models by Janick Bergeron Verilog HDL A Guide to Digital Design and Synthesis.pdf by Samir Palnitkar

Additional information can be on the web found at:

http://janick.bergeron.com/wtb

Introduction

What this course is about?

60% 80% of effort in design is dedicated to verification Unlike synthesizable code, no strict coding styles for verification (free-for-all) Absence of constraints and lack of available expertise and references in verification has resulted in ad hoc approaches Most HDL books (VHDL or Verilog) deal with design, not verification

What this course is about? (cont)

Over the years, these HDL books have been refined as synthesis tools have been refined To teach necessary concepts for tools of verification Describe process for carrying out effective functional verification Present techniques for applying stimulus and monitoring the response of a design utilizing bus functional models Present the importance of behavioral modeling

Prior Knowledge
This class focuses on functional verification of hardware design using either VHDL or Verilog Expect students to have a basic knowledge of one of these languages Expect students to have basic understanding of digital hardware design Class will focus more on Verilog

VHDL vs.. Verilog

What language should I use? This is usually dictated by ones experience and personal preference Typically, when working with a language, you do not notice the things that are simple to do, instead you notice the frustrations and how easy it would be if you were using the other language Both languages are inadequate for verification (by themselves) Both languages are equal in terms of the area under the learning curve.

Why HDL Verification?

I mentioned 60% - 80% time spent in verification WHY??
Product time-to-market
hardware turn-around time volume of "bugs"

Development costs
"Early User Hardware" (EUH)

Why HDL Verification? (cont)

Cost of bugs over time
Longer a bug goes undetected, the more expensive it is
Bug found early (designer sim) has little cost Finding a bug at chip/system has moderate cost
Requires more debug time and isolation time Could require new algorithm, which could effect schedule and cause board rework

Finding a bug in System Test (test floor) requires new spin of a chip Finding bug in customers environment can cost hundreds of millions and worst of all - Reputation

What is Verification?
Not a testbench Not a series of testbenches

Verification is a process used to demonstrate the functional correctness of a design. Also called logic verification or simulation.

What is a testbench?
A testbench usually refers to the code used to create a predetermined input sequence to a design, then optionally observe the response.
Generic term used differently across industry Always refers to a testcase Most commonly (and appropriately), a testbench refers to code written (VHDL, Verilog, etc) at the top level of the hierarchy. The testbench is often simple, but may have some elements of randomness No inputs or outputs effectively a model of the universe as far as the design is concerned. What input patterns to supply to the Design Under Verification and what is expected for the output for a properly working design

Completely closed system

Verification challenge

Importance of Verification
Most books focus on syntax, semantics and RTL subset
Given the amount of literature on writing synthesizable code vs.. writing verification test benches, one would think that the former is a more daunting task. Experience proves otherwise.

70% of design effort goes to verification

Properly staffed design teams have dedicated verification engineers. Verification Engineers usually outweigh designers 2-1

80% of all written code is in the verification environment

Verification is on critical path

Want to minimize Verification Time!

Ways to reduce verification time

Verification can be reduced through:
Parallelism: Add more resources Abstraction: Higher level of abstraction (i.e. C vs.. Assembly)
Beware though this means a reduction in control

Automation: Tools to automate standard processes

Requires standard processes Not all processes can be automated

Reconvergence Model
Conceptual representation of the verification process Most important question
What are you verifying?

Transformation

Verification

Human Factor in Verification Process

An individual (or group of and individuals) transform must into
Specification Interpretation
RTL Coding

interpret

specification

Verification

correct function.

Ways to reduce human-introduced errors

Automation
Take human intervention out of the process

Poka-Yoka
Make human intervention fool-proof

Redundancy
Have two individuals (or groups) check each others work

Automation
Obvious way to eliminate humanintroduced errors take the human out.
Good in concept Reality dictates that this is not feasible
Processes are not defined well enough Processes require human ingenuity and creativity

Poka-Yoka
Term coined in Total Quality Management circles Means to mistake-proof the human intervention Typically the last step in complete automation Same pitfalls as automation verification remains an art, it does not yield itself to well-defined steps.

Redundancy
Duplicate every transformation
Every transformation made by a human is either:
Verified by another individual Two complete and separate transformations are performed with each outcome compared to verify that both produced the same or equivalent result

Simplest Most costly, but still cheaper than redesign and replacement of a defective product Designer should NOT be in charge of verification!

What is being verified?

Choosing a common origin and reconvergence points determines what is being verified and what type of method to use. Following types of verification all have different origin and reconvergence points:
Formal Verification Model Checking Functional Verification Testbench Generators

Formal Verification
Once the end points of formal verification reconvergence paths are understood, then you know exactly what is being verified. 2 Types of Formal:
Equivalence Model Checking

Equivalence Checking
Compares two models to see if equivalence Proves mathematically that the origin and output are logically equivalent Examples:
RTL to Gates (Post Synthesis) Post Synthesis Gates to Post PD Gates

Equivalence Reconvergence Model

Synthesis

RTL

Gates

Check

Model Checking
Form of formal verification Characteristics of a design are formally proven or disproved Looks for generic problems or violations of user defined rules about the behavior of the design

Model Checking Reconvergence Model

RTL
Specification RTL Interpretation Model Assertions Checking

Functional Verification
Verifies design intent
One must trust that the transformation of a specification to RTL was performed correctly

Prove presence of bugs, but cannot prove their absence

Functional Reconvergence Model

Specification

RTL

Functional Verification

Testbench Generators
Tool to generate stimulus to exercise code or expose bugs Designer input is still required RTL code is the origin and there is no reconvergence point Verification engineer is left to determine if the testbench applies valid stimulus If used with parameters, can control the generator in order to focus the testbenches on more specific scenarios

Testbench Generation Reconvergence Model

Code Coverage/Proof RTL Metrics Testbench Generation

Testbench

Functional Verification Approaches

Black-Box Approach
White-Box Approach

Grey-Box Approach

Black-Box
Inputs
Some piece of logic design written in VHDL
Outputs

The black box has inputs, outputs, and performs some function. The function may be well documented...or not. To verify a black box, you need to understand the function and be able to predict the outputs based on the inputs. The black box can be a full system, a chip, a unit of a chip, or a single macro.

White-Box
White box verification means that the internal design is visible and utilized by the testbench stimulus. Examples: Unit/Module level verification

Grey-Box
Grey box verification means that a limited number of facilities are utilized in a mostly black-box environment. Example: Most environments! Prediction of correct results on the interface is occasionally impossible without viewing an internal signal.

Perfect Verification
To fully verify a black-box, you must show that the logic works correctly for all combinations of inputs. This entails:
Driving all permutations on the input lines Checking for proper results in all cases

Full verification is not practical on large pieces of designs, but the principles are valid across all verification.

Verification VS. Test

Two often confused Purpose of test is to verify that the design was manufactured properly Verification is to ensure that the design meets the functionality intent

Verification and Test Reconvergence Model

HW Design
Specification

Fabrication

Net list

Silicon

Verification

Test

Scan Based Testing

All internal registers are connected as chains in scan mode Tested through input pattern in scan mode and result compared with expected. Extra hardware and so extra cost

Design for Verification

Design to consider Testability requirements Verification has to be considered early in the design cycle with 2 Qs.
What design should do? How it will be verified?

Eg: programmable registers

Programmable Muxes

Verification And Design Reuse

Wont use what you dont trust. How to trust it?
Verify It.

For reuse, designs must be verified with more strict requirements

All claims, possible combinations and uses must be verified. Not just how it is used in a specific environment.

Cost of Verification
Necessary Evil
Always takes too long and costs too much Verification does not generate revenue

Yet indispensable
To create revenue, design must be functionally correct and provide benefits to customer Proper functional verification demonstrates trustworthiness of the design

Cost of Verification

So when is verification complete??

Thank You