Testing and Veriﬁcation

(1)

Testing and Verification

Martin Schoeberl

Technical University of Denmark Embedded Systems Engineering

March 2, 2022

1 / 34

(2)

Overview

I Thomas Aakjer presents digital design at Microchip I Review components

I Debugging and testing

I Digital designers (sometimes) call testing verification I To distinguish from final chip testing

2 / 34

(3)

DTU Chip Day

I Note the date: Tu 19 April afternoon I Start with sandwiches and finish with beer I Presentation of chip design and verification

work/companies in Denmark

I The first open-source chip from DTU

I Several chip companies will present and are prticipating I Opportunity to network for: theses with companies,

internship, student jobs

3 / 34

(4)

Last Chisel Lab (week 3)

I On components and small sequential circuits I Registers plus combinational circuits I Did you finish the exercises?

I Do the poll

I They are not mandatory, but helpful for preparation for the final project

I Let’s look at solutions

4 / 34

(5)

Components are Modules

I Components are building blocks I Like concrete, physical ICs

I Components have input and output ports (= pins) I Organized as aBundle

I Assigned to the fieldio

I We build circuits as a hierarchy of components I You did a 4:1 multiplexer out of three 2:1 mulitplexers I In Chisel a component is calledModule

I Components/Modules are used to organize the circuit I Similar to using methods in Java

I But they are connected withwires

5 / 34

(6)

A Binary Watch

I Built out of discrete, digital components

Source: Diogo Sousa,public domain

6 / 34

(7)

Let Us Build a Counter

I Counting from 0 up to 9 I Restart from 0

I Build it out of components I We need:

I Adder I Register I Multiplexer I

I But these are very tiny components

7 / 34

(8)

An Adder (Component/Module)

+ Adder

a

y b

class Adder extends Module { val io = IO (new Bundle {

val a = Input ( UInt (8. W)) val b = Input ( UInt (8. W)) val y = Output ( UInt (8. W)) })

io .y := io .a + io .b }

8 / 34

(9)

A Register

d q

Register

class Register extends Module {

val io = IO (new Bundle { val d = Input ( UInt (8. W)) val q = Output ( UInt (8. W)) })

val reg = RegInit (0. U) reg := io .d

io .q := reg }

9 / 34

(10)

The Counter Schematics

Register

next 1

Adder

0 result Count10

count

dout a

b y

d q

10 / 34

(11)

The Counter in Chisel

class Count10 extends Module { val io = IO (new Bundle {

val dout = Output ( UInt (8. W)) })

val add = Module (new Adder () ) val reg = Module (new Register () ) val count = reg . io .q

// connect the adder add . io .a := 1. U add . io .b := count val result = add . io .y

val next = Mux ( count === 9.U , 0.U , result ) reg . io .d := next

io . dout := count

} _{11 / 34}

(12)

Summarize Components

I Think like concrete components (ICs) I They have named pins (io.name)

I In hardware language these pins are often called ports I Ports have a direction (input or output)

I They need to be created:

I val mc = Module(new MyComponent()) I and pins need to be connected with:=

I One module is special, as it is the top model

12 / 34

(13)

Chisel Main

I Create one top-level Module

I Invoke theemitVerilog()from the App I Pass the top module (e.g.,new Hello()) I Optional: pass some parameters (in anArray)

I Following code generates Verilog code for theHello World object Hello extends App {

emitVerilog (new Hello () ) }

13 / 34

(14)

Testing and Debugging

I Nobody writes perfect code ;-)

I We need a method to improve the code I In Java we can simply print values:

I println("42");

I What can we do in hardware?

I Describe the whole circuit and hope it works?

I We can switch an LED on and off

I Test it with switches and LEDs in an FPGA I We need some tools fordebugging

I Writing testers in Chisel

I We test by running a simulation of the circuit

14 / 34

(15)

ScalaTest

I Testing framework for Scala and Java I Tests are placed undersrc/test/scala I sbtunderstands ScalaTest

I Run all tests with:

sbt test

I When all (unit) tests are ok, the test suit passes I A little bit funny syntax

I ChiselTest is based on ScalaTest

15 / 34

(16)

Testing with Chisel

I A test contains

I a device under test (DUT) and I the testing logic

I Set input values withpoke

I Advance the simulation withstep I Read the output values withpeek I Compare the values withexpect I Import following packages

import chisel3 ._

import chiseltest ._

import org . scalatest . flatspec . AnyFlatSpec

16 / 34

(17)

An Example DUT

I A device-under test (DUT) I Just 2-bit AND logic

class DeviceUnderTest extends Module { val io = IO (new Bundle {

val a = Input ( UInt (2. W)) val b = Input ( UInt (2. W)) val out = Output ( UInt (2. W)) })

io . out := io .a & io .b }

17 / 34

(18)

A ChiselTest

I Extends classAnyFlatSpecwithChiselScalatestTester I Has the device-under test (DUT) as parameter of the

test()function

I Test function contains the test code I Testing code can use all features of Scala I Is placed insrc/test/scala

I Is run withsbt test

18 / 34

(19)

A Simple Tester

I Just usingprintlnfor manual inspection

class SimpleTest extends AnyFlatSpec with ChiselScalatestTester {

" DUT " should " pass " in {

test (new DeviceUnderTest ) { dut =>

dut . io .a. poke (0. U) dut . io .b. poke (1. U) dut . clock . step ()

println (" Result is : " +

dut . io . out . peek () . toString ) dut . io .a. poke (3. U)

dut . io .b. poke (2. U) dut . clock . step ()

println (" Result is : " +

dut . io . out . peek () . toString ) }

} }

19 / 34

(20)

A Real Tester

I Poke values andexpectsome output

class SimpleTestExpect extends AnyFlatSpec with ChiselScalatestTester {

" DUT " should " pass " in {

test (new DeviceUnderTest ) { dut =>

dut . io .a. poke (0. U) dut . io .b. poke (1. U) dut . clock . step ()

dut . io . out . expect (0. U) dut . io .a. poke (3. U) dut . io .b. poke (2. U) dut . clock . step ()

dut . io . out . expect (2. U) }

} }

20 / 34

(21)

Generating Waveforms

I Waveforms are timing diagrams

I Good to see many parallel signals and registers sbt "testOnly SimpleTest -- -DwriteVcd=1"

I Or setting an attribute for thetest()function test (new DeviceUnderTest )

. withAnnotations ( Seq ( WriteVcdAnnotation )) I IO signals and registers are dumped

I Option--debugputs all wires into the dump I Generates a .vcd file

I Viewing with GTKWave or ModelSim

21 / 34

(22)

Waveform Testing Demo

I Counter with a limit from last Chisel lab (Count6) I Show Count6 tester: the original and the waveform I Run it and look at waveform

I Add the solution

I Run again and reload the waveform

22 / 34

(23)

A Self-Running Circuit

I Count6is a self-running circuit I Needs no stimuli (poke) I Just run for a few cycles

test (new Count6 ) { dut =>

dut . clock . step (20) }

23 / 34

(24)

The WaveForm

I The complete test

I Note the.withAnnotations(Seq(WriteVcdAnnotation)

class Count6WaveSpec extends AnyFlatSpec with ChiselScalatestTester {

" CountWave6 " should " pass " in { test (new

Count6 ). withAnnotations ( Seq ( WriteVcdAnnotation )) { dut =>

dut . clock . step (20) }

} }

24 / 34

(25)

Vending Machine Testing

I I provide a minimal tester to generate a waveform I Adding some coins and buying

I You can and shall extend this tester I Better having more than one tester I Show the waveform of the test

25 / 34

(26)

Printf Debugging

I We canprintin the hardware during simulation I Printing happens on the rising edge of the clock I Good to see many parallel signals and registers I printfanywhere in the module definition

class DeviceUnderTestPrintf extends Module { val io = IO (new Bundle {

val a = Input ( UInt (2. W)) val b = Input ( UInt (2. W)) val out = Output ( UInt (2. W)) })

io . out := io .a & io .b

printf (" dut : %d %d %d\n", io .a , io .b , io . out ) }

26 / 34

(27)

Test Driven Development (TDD)

I Software development process

I Can we learn from SW development for HW design?

I Writing the test first, then the implementation I Started with extreme programming

I Frequent releases

I Accept change as part of the development I A path toAgile Hardware Development!

I Not used in its pour form

I Writing all those tests is simply considerer too much work I But, write at least one test for each component

27 / 34

(28)

Regression Tests

I Tests are collected over time

I When a bug is found, a test is written to reproduce this bug I Collection of tests increases

I Runs every night to test forregression

I Did a code change introduce a bug in the current code base?

28 / 34

(29)

Continuous Integration (CI)

I Next logical step from regression tests I Run all tests whenever code is changed

I Automate this with a repository, e.g., on GitHub I Run CI on GitHub

I Show about this on the Chisel book I Showsbt test

I Live demo on GitHub (I moved from Travis yesterday) I Mail from GitHub when it fails

I https://github.com/schoeberl/chisel-book/actions I Maybe show how to set this up (it is easy ;)

I Start with thechisel-emptytemplate I Open it with IntelliJ

I Add action in GitHub

29 / 34

(30)

Testing versus Debugging

I Debugging is during code development

I Waveform andprintlnare easy tools for debugging I Debugging does not help for regression tests

I Write small test cases for regression tests

I Keeps your code baseintactwhen doing changes I Better confidence in changes not introducing new bugs

30 / 34

(31)

Scala Build Tool (sbt)

I Downloads Scala compiler if needed

I Downloads dependent libraries (e.g., Chisel) I Compiles Scala programs

I Executes Scala programs

I Does a lot of magic, maybe too much I Compile and run with:

sbt " runMain simple . Example "

sbt run sbt test

sbt " testOnly MySpec "

sbt compile

31 / 34

(32)

Build Configuration

I File name: build.sbt

I Defines needed Scala version I Library dependencies

scalaVersion := " 2.12.13 "

scalacOptions ++= Seq (" - feature ",

" - language : reflectiveCalls ") resolvers ++=

Seq ( Resolver . sonatypeRepo (" releases ")) addCompilerPlugin (" edu . berkeley . cs " %

" chisel3 - plugin " % " 3.5.0 " cross CrossVersion . full )

libraryDependencies += " edu . berkeley . cs " %%

" chisel3 " % " 3.5.0 "

libraryDependencies += " edu . berkeley . cs " %%

" chiseltest " % " 0.5.0 "

32 / 34

(33)

Today’s Lab

I Testing a faulty multiplexer

I Do not look into the multiplexer code, find out with testing I Use ChiselTest (the description has been updated) I You have to start from scratch with the tester

I Show and discuss your testing code with a TA (or me) I Lab 5

33 / 34

(34)

Summary

I Small sequential circuits are our building blocks I We build larger circuits by combining components

(modules)

I There is noprintlnin (real) hardware I We need to write tests for the development I Debugging versus regression tests

34 / 34