Welcome to Design Verification

Hello, welcome to the Design Verification module (COMS30026) website. Here you will find all the course content and related information. Lecture recordings and information on coursework (where applicable) are hosted privately on Blackboard at the University of Bristol. For further information about this course, please contact Kerstin Eder.

Overview

This unit familiarises students with the state of the art in Design Verification, and gives them the technical background plus some of the practical skills expected from a professional design verification engineer.

What you need to know prior to taking this unit and what you can expect to learn

Students intending to take this unit should know that the unit requires a significant amount of initiative and motivation, independent study, e.g. teaching yourself a new language and familiarising yourself with EDA tools in the labs, and diligent time management to meet the learning objectives. Essential pre-requisites to take this unit include good programming and software engineering skills and a basic understanding of computer architecture.

In return, on successful completion of this unit, you will be able to add to your CV knowledge and skills in the following areas:

• the Verilog Hardware Description Language,
• EDA tools including QuestaSim (ModelSim) and the Cadence verification tools,
• the e Verification Language,
• Testbench Automation Techniques,
• Constraint-Based Pseudo Random Test Generation,
• Assertion-Based Verification,
• Coverage-Driven Verification,
• Functional Formal Verification.

Lecture and lab times

Autumn Term 2024

• Lectures are scheduled in weeks 1-5, 7-8, and in week 12 we will have a 1h revision session. Please refer to the timetable for lecture times and locations. Any changes to scheduled lecture times and lecture theatres will be announced during lectures and via Blackboard.

• Lecture recordings will be made available via Blackboard. There are roughly three hours of recordings per week, though you should attend the live lectures as timetabled, and use the recordings only for revision.

• Labs (called Practicals in your timetable) are scheduled in weeks 1-5 and 7-8 on Wednesdays at 12:00 and Fridays at 12:00 in MVB. Labs will be supervised by a Teaching Assistant (TA). Labs can cover setting up remote access to the EDA tools, an introduction to using the EDA tools, help with using the various languages, methodologies, tools, etc. New material and demos will normally be covered in the Wednesday lab session. In general, the lab session on Fridays offers an opportunity for you to catch up.

• You are expected to engage with the supplementary material and the opportunities highlighted in the lectures and on this unit page.

Making contact

Autumn Term 2024

• Your main ways of getting help with this unit are:
  1. Ask directly after the lectures.
  2. Ask the unit TA, ideally during the labs/practicals on Wednesdays.
  3. Post your question on the online Blackboard forum for the unit after checking the forum content.
  4. Ask a fellow student via Teams.
• Please avoid sending Kerstin an email to get help as this may not result in the type of help you may need. This is because Kerstin gets far too many emails per day to process them all in a timely manner.

Unit Content

Lectures

The slides provided in the table below may be updated with revised versions as necessary. Please note that the lectures and recordings may not cover all slides; some of the slides not covered may provide further information for exercises and self study.

Week	Topic	Slides	Supplementary material
Week 1	Introduction to the Design Verification unit COMS30026	View	The limits of Correctness from the ACM Digital Library
Week 1	Introduction to Design Verification	View	ITRS Reports Read the paper by Harry Foster on Why the Design Productivity Gap never happened
Week 1	Verification Hierarchy	View	Foster: Provably Correct Design Read the paper on System Deadlocks
Week 2	Verification Tools	View	Verification challenges Intelligent Testing (Dot Graham) EWD 340: The humble programmer
Week 2	Driving & Checking	View	Explore the foretellix blog Review the 2020 Wilson Research Group Functional Verification Study Slides, FPGA Trend Report, IC ASIC Trend Report 2022 Wilson Research Group Functional Verification Study
Week (2)	Hardware Design Languages (self-study)	View	BOOK: Verilog HDL by Samir Palnitkar [in QB Library]
Week 3	Verification Cycle, Methodology & Plan	plain with ink	Verification plan template I’m done simulating: Now what?
Week 3	Stimuli Generation Part I: Foundations	plain with ink	IEE Stress Tests by Darren Galpin
Week 3	Stimuli Generation Part II: Test Automation	plain with ink	GenesysPro: Innovations in Test Program Generation for Functional Processor Verification
Week 4	Checking	plain with ink	Design Verification Glossary Verification Academy
Week 4	Assertion Based Verification Part I: Introduction	View with ink	OVL quick reference card
Week 4	Assertion Based Verification Part II: Property Formalisation	View with ink	ModelSim PSL quick reference card SVA quick reference
Week 5	Coverage Part I: Introduction and Code Coverage	plain with ink	Why statement coverage is not enough An introduction to FSM coverage A practical tutorial on MC/DC coverage
Week 5	Coverage Part II: Functional Coverage	plain with ink
Week 5	Coverage Part III: Coverage Analysis	plain with ink
Week 7	High-level Verification: sn and e - Part 1	View with ink	The e Language: A fresh separation of concerns IEEE Standard for E (2016)
Week 7	High-level Verification: sn and e - Part 2	View with ink	Coverage-Driven Verification Methodology Summary for UVM by Doulos SN and e quick reference (2018) SN and e quick reference (2014)
Week 7	Functional Formal Verification	View with ink	PSL quick reference card for Verilog PSL quick reference card for VHDL
Week 8	Closing the Cycle Notes on Analysis and Adaptation with a focus on Failure Analysis	View with ink	DV Club on Signing Off: What are your Verification Tape-Out Criteria? Presentations from Arm, Infineon and Graphcore
Week 8	SoC Verification (Guest Lecture by Mike Benjamin)	View	Functional verification of IBM’s POWER7 processor core

EDA Software Access

You will require access to two EDA tools for this module: Siemens QuestaSim (ModelSim) and Cadence SpecMan and JasperGold.

The EDA tools are accessible in the Linux labs in MVB, 2.11 and 1.15, and also remotely. To set up remote access you need to follow the instructions on how to set up X2GO.

The following are the command line instructions you can use from a terminal once you are logged in, either directly in a lab or remotely:

> module use /eda/cadence/modules
> module load course/COMS30026

You should then be able to call QuestaSim (ModelSim) from the command line:

> vsim &

You may also want to check whether you can call Cadence SpecMan and JasperGold:

> specview &
> jaspergold &

The simulator for Specman only works within a virtual machine. The guide to access EDA tools in a virtual machine is in Access EDA tools in a virtual machine.

In both cases the tools should come up with GUI interfaces. If this works for you, then you are ready to start the exercises and, where applicable, your coursework.

You can also download this worksheet on how to use X2GO for remote access.

Exercises

The exercises are designed to further your understanding of Verilog, the simulator and the e language, and to give you an opportunity to practice the material covered during the lectures. The QuestaSim (ModelSim) simulator and SpecMan Elite are installed under Linux, for access instructions please see above.

Exercise 1

Your first exercise is a self-study exercise for you to teach yourself the basics of the Hardware Description Language (HDL) Verilog.

In particular, you will need the behavioural coding style for the first practical. Please remember that we “only” want to verify HDL designs. This means that we need to be able to understand the basics of HDLs (in this case Verilog); we do NOT intend to design hardware in this unit.

There are several good tutorials on Verilog online. ALDEC Inc. offers the Evita Verilog interacitve tutorial. To download it you need to sign up on the ALDEC web page. Please use your University of Bristol email address to do so; ALDEC do not accept generic emails like gmail, yahoo or hotmail. Installation is straightforward; the tutorial is self-extracting and runs under Windows.

You may also want to study the set of slides that come with the book “The Verilog Hardware Description Language” by Donald Thomas.

For future use you might want to download and print the Verilog Reference Card: verilog_reference_card.pdf

Exercise 2

This exercise introduces you to the ModelSim/QuestaSim simulator by Siemens.

Introduction to the ModelSim/QuestaSim Simulator

For future use you might want to download and print the ModelSim/QuestaSim quick reference guide: modelsim_se_6_5_quick_ref.pdf

Files needed for the ModelSim/QuestaSim introduction:

ZIP archive of all the below files

• mux421_structural.v
• mux421_dataflow.v
• mux421_behavioural.v
• mux421_gen.v
• mux421_check.v

• mux421_example_testbench.v

• mux421_structural_faulty.v
• mux421_dataflow_faulty.v
• mux421_behavioural_faulty.v

• mux421_faulty_testbench.v - Only download this one if you do not want to (learn how to) write your own.

• mux_int.v
• mux_int_test.v

Exercise 3

How to collect Code Coverage with ModelSim

Exercise 4

Introduction to SpecMan Elite

SpecMan Elite is part of the Incisive Functional Verification platform provided by Cadence.

Files needed for the SpecMan tutorial:

• sn_cpu_tutorial.tar.gz
• specman_tutorial.pdf

FAQs

For some answers to frequently asked questions about Verilog and Specman/E, please see the following pages.

• Writing testbenches with Specman/E
• Writing testbenches with Verilog
• Gluing Verilog modules together
• Generating random numbers in Verilog

Online resources

The Verification Academy provides a comprehensive learning resource from basics to very advanced materials.

Additional materials

A book that covers some of the course is:

Janick Bergeron
Writing Testbenches: Functional Verification of HDL Models
First Edition, Kluwer Academic Publishers, 2000, ISBN: 0-7923-7766-4
Second Edition, Kluwer Academic Publishers, 2003, ISBN: 1-4020-7401-8 

In May 2005 a comprehensive textbook on Design Verification was published:

Bruce Wile, John Goss and Wolfgang Roesner
Comprehensive Functional Verification
Elsevier, 2005, ISBN: 0-12-751803-7 

Both books are available in the QB library.

Credits

Parts of the lecture notes contain material from the book “Writing Testbenches: Functional Verification of HDL Models” by Janick Bergeron and from lecture slides developed at IBM, the University of Pittsburgh, Penn State University, North Carolina State University and Ohio State University. The HDL for the assignments has been developed at IBM - special thanks go to Bruce Wile for his support and encouragement.