February 22-24, 2006

Guidelines for creating a formal verification testplan

There is no substitute for thinking!
Harry Foster - Mentor Graphics Corp Lawrence Loh - Jasper Design Automation Bahman Rabii - Google Vigyan Singhal - Oski Technology 1

Agenda
Observation Verification Testplanning Process Seven Steps of Formal Testplanning AHB-I2C Example

Harry Foster, 2006

Observation
Successful verification is not ad hoc in nature Success depends on methodical verification planning combined with systematic verification processes. The key to success is the verification testplan.

Harry Foster, 2006

Verification Testplan
A projects functional verification testplan is the specification for the verification process. Developing this testplan usually involves the entire engineering team
architects, designers, and verification engineers

In general, the verification testplan defines exactly what functionality will be verified, how it will be verified
the verification strategy and resource allocation

when the verification process is complete

metrics for measuring progress or completion criteria

Harry Foster, 2006

FV for bug hunting or assurance?

Bug hunting Many RTL assertions Pushbutton tools Success: # bugs found Assurance One comprehensive spec Semi-automated tools Success: ckt meets spec

Bugs Rate

Testplan critical!

Assurance Bug hunting Time

Rev 0 RTL Tapeout

Harry Foster, 2006

Agenda
Observation Verification Testplanning Process Seven Steps of Formal Testplanning AHB-I2C Example

Harry Foster, 2006

Where to apply formal?

Formal verification of properties on RTL designs is a known hard problem Formal verification algorithm complexity
Exponential in the size of the designs

Nave application of formal verification

State-space explosion Impractical computer run- times

Our first step is to identify appropriate blocks for formal Prior to creating a formal testplan

Harry Foster, 2006

Where to apply formal

One coarse measure of prediction of the tractability
number of state-holding elements in the cone of influence of the property

Design Block
Cone of Influence Irrelevant Logic Property

Harry Foster, 2006

Where to apply formal

Real examples include:
Arbiters of many different kinds On-chip bus bridge Power management unit DMA controller Host bus interface unit Scheduler, implementing multiple virtual channels for QoS Interrupt controller Memory controller Token generator Credit manager block Standard interface (AMBA, PCI Express, ) Proprietary interfaces Clock disable unit

Common characteristic: concurrency, multiple data streams

Multiple, concurrent streams Hard to completely verify using simulation 10 bugs per 1000 gates -Ted Scardamalia, IBM

Harry Foster, 2006

Where not to apply formal

Examples of designs not appropriate for formal include:

(Im not talking about theorem proving!)

Floating point unit Graphics shading unit Convolution unit in a DSP chip MPEG decoder
Common characteristic:
often sequential in nature, potentially involves some type of data transformation (math)

Single, sequential/functional streams 2 bugs per 1000 gates -Ted Scardamalia, IBM

Harry Foster, 2006

10

Control versus datapath

Design Verification

Control

Datapath

Data Transport

Data Transform

Harry Foster, 2006

11

Simulation versus formal testplans

What vs. how Two key differences between a formal and simulation testplan:
A strict separation of checks (observability) and input scenarios (stimulus) A preference of a more general specification style

In simulation checkers and stimulus can be tightly coupled In formal verification, properties are defined in terms of:
Generic behavior, independent of particular input scenarios. Minimal correctness criteria Avoid cycle accurate specification when possible
Harry Foster, 2006 12

Compositional Reasoning
The process of reducing an analysis of a larger concurrent system to reasoning about its individual functional pieces
Effective for managing proof complexity and state explosion Transfers the burden of proof from the global level to the local functional component level
assume always !(A & B);

A B

Block A

Block B

assert . . .

assert always !(A & B);

When possible, prove assumptions on neighboring blocks Assume - guarantee

13

Harry Foster, 2006

Compositional Reasoning

Free Variables Abstracted Inputs

Cone of Influence

Analysis Region

Design Cone of Block Influence a b

Irrelevant Logic

Property

Formal Abstraction

Harry Foster, 2006

14

Agenda
Observation Verification Testplanning Process Seven Steps of Formal Testplanning AHB-I2C Example

Harry Foster, 2006

15

Seven steps of a formal testplan

identify

1 2 3

Identify good candidates

describe

English list

Define interface

Requirements checklist

inputs

Requirements
outputs

Block

Formal description

Verification strategy

6 7

assert_never ();
Formal description

Coverage goals

Harry Foster, 2006

16

Agenda
Observation Verification Testplanning Process Seven Steps of Formal Testplanning AHB-I2C Example

Harry Foster, 2006

17

AHB Lite to I2C Example

Identify good candidates

identify

1 2 3

-arch

Great candidate for formal!

inputs

English list

Define interface

Requirements checklist

Requirements
outputs

Block

Formal description

Verification strategy

6 7

assert_never ();
Formal description

Coverage goals

d[31:0] a[31:0] size

AMBA AMBA AHB Lite AHB Lite

d_ready

Bridge Bridge

I2C I2C

clk_en i2c_data_vld i2c_data_out i2c_data_in

I/O I/O Merge Merge

SCL SDA

bridge_ready i2c_data_start

High speed, high bandwidth bus Bridge (with 64 byte write FIFO)
Harry Foster, 2006

One bit serial connection I2C Clocking

18

Step 2. Describe Intent

inputs

Identify good candidates

identify

1 2 3

-arch

English list

Define interface

Requirements checklist

Requirements
outputs

Block

Formal description

NOTE: This can be a video taped white-board discussion!

The commands flow one direction from AHB to I2C, but the data flows both ways. For the write direction, data are written into a FIFO. When the FIFO is full, the AHB signal HREADYout is deasserted until there is room in the FIFO again. Upon receiving the write data, as long as there is room in the FIFO, the AHB bus is free for other devices sharing the AHB to proceed to their transactions. A read-cycle, however, will hold up the bus until the data is ready (because a SPLIT transaction is not supported by the bridge). Therefore, the read transaction has priority over the write transaction except when there is a coherency issue. For example, if a read address matches the write address of one of the entries in the. . . . .

Verification strategy

6 7

assert_never ();
Formal description

Coverage goals

Harry Foster, 2006

19

Step 3. Define Interface

Signal Name HCLK HRESETn HADDR HBURST HTRANS HSIZE HWRITE HSEL HREADYin HWDATAHWDATA HRDATA HRESP HREADYout SDA SCL i2c_clk_ratio Description AHB Clock Master Reset (active low) AHB Address AHB Burst length AHB Transaction Type AHB Transfer Size AHB Write AHB Select AHB HReady AHB Write Data AHB Read Data AHB Response AHB HREADYOUT I2C Data I2C Clock HCLK to I2C Clock ratio Size 1-bit 1-bit 7-bit 3-bit 2-bit 3-bit 1-bit 1-bit 1-bit 32-bit 32-bit 2-bit 1-bit 1-bit 1-bit 2-bit Direction In In In In In In In In In In Out Out Out In/Out Out In
inputs

Identify good candidates

identify

1 2 3

-arch

English list

Define interface

Requirements checklist

Requirements
outputs

Block

Formal description

Verification strategy

6 7

assert_never ();
Formal description

Coverage goals

Requirement Model inputs DUT outputs

Harry Foster, 2006

20

Step 4. Requirements Checklist

inputs

Identify good candidates

identify

1 2 3

-arch

English list

Define interface

Requirements checklist

Requirements
outputs

Block

Formal description

Verification strategy

6 7

assert_never ();
Formal description

AMBA AHB-Lite interface requirements. In general, we can partition AMBA AHB-Lite requirements into two categories: master requirements and slave requirements. 1. 2. 3. 4. 1. 2. 3. 4. 1. 2. Slave must assert HREADYOUT after reset

Coverage goals

Slave must provide zero wait-state HREADYOUT=1 response to IDLE transaction Slave must provide zero wait-state HREADYOUT=1 response to BUSY transaction ... SDA should remain stable when SCL is high There should not be another start after a start until an end occurs in the I2C bus. The data between a start and an end should be divisible by 9 (8 bit/transfer + 1-bit ack) ... Data is not dropped, duplicated, or corrupted for a read-cycle Data is not dropped, duplicated, or corrupted for a write-cycle 21

I2C interface requirements.

End-to-end requirements.

3. Harry Foster, 2006

Step 5. Formalize Requirements

Identify good candidates

identify

1 2 3

-arch

English list

To demonstrate the formal specification process, we convert the following I2C requirement into both PSL and SVA: There should not be another start after a start until an end occurs in the I2C bus.
NOTE: In this example, i2c_s tand i2c_end represent modeling code tar associated with the assertion, composed of SCL and SDA.

Define interface

Requirements checklist

inputs

Requirements
outputs

Block

Formal description

Verification strategy

6 7

assert_never ();
Formal description

Coverage goals

default clock = HCLK; PSL A_no_start: assert (always i2c_start -> next(~i2c_start until i2c_end)) abort (~RESETn); property P_no_start; @(posedge HCLK) disable iff (~HRESETn) i2c_start |=> ~i2c_start[*0:$] ##1 i2c_end; endproperty A_no_start: assert property (P_no_start);
Harry Foster, 2006 22

SVA

Step 6. Strategy
inputs

Identify good candidates

identify

1 2 3

-arch

English list

Define interface

Requirements checklist

Requirements
outputs

Block

Formal description

Verification strategy

6 7

assert_never ();
Formal description

Coverage goals

In this step, you will define the verification plan and proof strategy. For example, you might choose to prove certain requirements by initially applying a set of restrictions
Add constraints to restrict the explored behavior to only read-mode transaction, or only write-mode transactions. This approach lets you focus the verification on specific modes of operation, and simplifies the verification process. Once the various modes function correctly, the restrictions are removed and the IP is verified under all modes of operation.

Harry Foster, 2006

23

Step 6. Strategy
inputs

Identify good candidates

identify

1 2 3

-arch

English list

Define interface

Requirements checklist

Requirements
outputs

Block

Formal description

Verification strategy

6 7

assert_never ();
Formal description

Coverage goals

Compositional Reasoning Partitioning Strict Mode Applying restrictions initially, and then loosening them over a sequence of steps Abstraction

Harry Foster, 2006

24

Step 7. Coverage
inputs

Identify good candidates

identify

1 2 3

-arch

English list

Define interface

Requirements checklist

Requirements
outputs

Block

Formal description

Verification strategy

6 7

assert_never ();
Formal description

Coverage goals

Define a set of coverage goals that must be met during the verification process.
To ensure that you are verifying what you think you are verifying, check that constraints on specific states in your requirements model have not over-constrained the states. For formal verification, you can check to see if specific states in your requirement model are reachable. For simulation-based approaches, you can apply functional coverage to various states or sequences of states in the requirements model.

The checking procedure varies depending on your verification process.

Harry Foster, 2006

25

Step 7. Coverage
inputs

Identify good candidates

identify

1 2 3

-arch

English list

Define interface

Requirements checklist

Requirements
outputs

Block

Formal description

Verification strategy

6 7

assert_never ();
Formal description

Coverage goals

Set 1: Input coverage Read/write access with different burst types, sizes, and lengths, and with HREADYOUT asserted and deasserted. Set 2: Output coverage Read/write with acknowledgment, no acknowledgement. Set 3: Internal main state-machines I2C state-machines, AHB state-machines where they can enter and exit each state.

Harry Foster, 2006

26

Seven steps of a formal testplan

identify

1 2 3

Identify good candidates

describe

English list

Define interface

Requirements checklist

inputs

Requirements
outputs

Block

Formal description

Verification strategy

6 7

assert_never ();
Formal description

Coverage goals

Harry Foster, 2006

27

February 22-24, 2006

Guidelines for creating a formal verification testplan

There is no substitute for thinking!
Harry Foster - Mentor Graphics Lawrence Loh - Jasper Design Automation Bahman Rabii - Google Vigyan Singhal - Oski Technology 28