Espen Tallaksen, CEO, EmLogic
According to the 2024 Wilson Research Group FPGA functional verification trend report, around half the development time for an FPGA is spent on verification, particularly on projects involving a substantial amount of newly developed design. Even worse, almost half of that time involves debugging of both the Design Under Test (DUT) and the Testbench itself, which is, in my opinion, far too large a percentage of the total development time. Yet I think it is possible to significantly reduce this, with only minor adjustments and no extra cost.
For an FPGA design, we all know that the architecture, all the way from the top to the micro-architecture, is critical for both FPGA quality and development time, and this also applies to the testbench. The Universal VHDL Verification Methodology (UVVM) was developed to solve this and can cut verification time while at the same time improving FPGA quality.
A free and open source VHDL verification library and methodology, UVVM provides the best VHDL testbench approach possible. With its straightforward and powerful architecture, it allows designers to build their own test harness and test cases much faster than ever before. Equally important, it has a unique reuse structure, making it a game changer for efficient FPGA development and testbench reuse.
This has led to the fast adoption of UVVM in the FPGA community. Currently, more than 27% of all FPGA designers worldwide use UVVM, and the number is much higher when considering VHDL designers only. Since 2017, we have closely cooperated with the European Space Agency (ESA) on improving and extending its functionality, and in March last year, we started on our third ESA UVVM project, providing even more verification and debug functionality.
Enabling efficiency and quality
Most designers know that for FPGA design, there is a strong correlation between efficiency and quality and the following design characteristics: Overview, Readability, Modifiability, Maintainability and Extensibility. These are equally important for testbenches and verification, and debuggability is also essential. Finally, reusability from one project to another is always important, but for verification, there is also a huge improvement potential between module testbenches in a single project, and also from module testbenches to the top level testbenches.
Improving architecture and simplicity
In order to achieve these improvements, you need a really good testbench architecture, not just at the higher level, but all the way down to testbench micro-architecture. To achieve good efficiency and quality, only one critical aspect is missing: simplicity.
For a complex FPGA, verification is seldom simple, but the key is “as simple as possible for the tasks where you spend most of your time”. For verification that would be to make all the different test cases, i.e. writing the various test sequences. The other main aspects of making a testbench are the test harness and any potential verification support procedures, processes and entities, all of which UVVM focuses on.
What UVVM offers
UVVM provides VHDL users with a methodology and library allowing a stepwise evolution of their testbenches for simple, via medium, to complex DUTs, so that the user can take the next step in verification complexity only when needed. From day one, it offers the following:
- A testbench infrastructure with basic commands for any VHDL testbench
- BFMs (Bus Functional Models) for many common FPGA peripherals/interfaces
- Specification Coverage for Requirements Tracking
- A very structured testbench architecture for more advanced verification challenges
- VHDL Verification Components (VVCs) for many common FPGA peripherals/interfaces
- A very structured VVC architecture that allows BFMs to be controlled simultaneously and in a very controlled manner
- Transaction-level modelling (TLM) for high-level control of the testbench
- Advanced and Optimized Randomization for Constrained Random
- Functional coverage
- Various other verification support modules like Scoreboards, Error injector, Watchdogs, etc.
Fig 1. Efficiency and quality enablers
Starting out with UVVM
A good starting point for using UVVM is to evaluate what is always required for any good testbench, independent of DUT complexity:
- Logging – with good messages
- Alert handling – with good messages
- Checking values and time aspects
- Waiting for something to happen
- Randomization (not always, but often)
We can exemplify this by looking at the most important testbench functionality for a very simple module like a basic interrupt controller with N interrupt sources, a resulting interrupt to the CPU and a register interface (SBI = Simple Bus Interface) for software access, as shown in Figure 2:
Fig 2. A simple testbench for a simple DUT
In every testbench, we also need a clock controller, and in UVVM you can choose between several variants, with the simplest version as follows:
This is just a simple procedure call that you put into your testbench architecture. When putting a procedure call directly in the architecture and not inside a process, this will be a so-called concurrent procedure, which works exactly like a process. Thus, clock generator() works exactly like a full clock process. There are lots of clock generator variants available in UVVM. You can then write your test sequencer inside your test process as shown below:
All the commands here are self-explanatory, which makes it easy to read and modify. The message provided as the last parameter is what you would normally write as a comment, but including it in the procedure call allows the message to be written to the transcript/log if something fails, or as a positive acknowledgement if you want that. The resulting output from the above commands would be as shown below. You can also include a log prefix, message ID and scope in this log output:
All the commands shown above are available from the UVVM Utility Library, a testbench infrastructure library with lots of very useful functions and procedures.
In addition to the ones mentioned above, there are similar, very simple-to-use functions for string handling, randomization, waiting for signal stability, flags and synchronization mechanisms, normalization, verbosity control, etc.
BFM procedures
The next logical step would be to use Bus Functional Model (BFM) procedures for accessing the various DUT-internal software accessible registers. In the example below, we first write something to the Interrupt Trigger Register (ITR) and then read back the Interrupt Request Register (IRR) and check that the value is as expected:
The result would then be as follows:
In your test sequencer you can use all these procedures and many others to check various functionality inside your module, and finally you can write out an alert summary. You can then use this summary or the provided status summary shared variable as input to your regression testing tool.
BFM procedure limitations
BFM procedures are great for accessing interfaces, as you only need to call a procedure. All of the protocol details, signal wiggling, sampling, etc, are then handled for you and you don’t even have to understand the interface or protocol. Also, it allows changes to the protocol and interface without changing the BFM procedures or the calls from the sequencers.
There is, however, one major limitation with BFMs: they are blocking. That means when executing the BFM, nothing else can be done inside the process in which the BFM is executed. Thus, when executing a BFM inside a test sequencer, the test sequencer cannot do anything else until the interface access is finished. So, for instance, a UART test sequencer cannot transmit data into the DUT RX input at the same time as reading a previously received byte via the CPU interface. This is a very typical error-prone scenario and a testbench should definitely test simultaneous activity on all channels.
Designers who are aware of this typically handle the three independent UART interfaces (RX, TX and CPU interface) from three different test processes. The problem with that approach is that in order to find the very error-prone cycle-related corner cases in such scenarios, you need to carefully control the interactivity on these interfaces. The normal way to handle this is to apply various synchronization mechanisms between these processes, typically by sending trigger signals or semaphores back and forth. This might seem like a structured approach, but you very soon lose the overview. This means you can forget about readability, maintainability, debuggability and reusability. It is much easier to control everything from one single brain – in this case a single sequencer, provided you have the right testbench architecture. Here, UVVM is a major step forward.
Simple control of multiple interfaces
UVVM’s VHDL Verification Component (VVC) system allows simultaneous activity on multiple interfaces to be controlled in a very structured and simple manner, by distributing, in zero time, the execution of the BFM procedures to VVCs. UVVM even allows for delays to be inserted before or after any BFM execution.
This means the test sequencer has full control over the complete verification environment, and you can easily see what will happen in your testbench at any time by just looking at the test sequence in one single process, and not multiple processes.
Switching from BFM to VVC
Seen from the test sequencer, the switch from using BFMs directly to using VVCs is quite simple.
You just use a slightly different set of procedures, where the most important difference is the added target parameters. These target parameters say which verification component will handle the actual execution of the BFM, so in fact, the VVC command is just a non-time-consuming distribution of a BFM command to the given VVC. And this non-time-consuming distribution means that the test sequencer is not blocked, but can distribute commands to multiple interfaces simultaneously and then even do something by itself in parallel.
Figure 3 shows a BFM-based testbench to the right, where the test sequencer has direct access to the interfaces of the DUT, and uses BFMs to access these, one at a time. To the left, a VVC-based testbench is shown. Here, the VVCs are connected directly (port-mapped) to the various interfaces of the DUT, and the test sequencer sends non-time-consuming commands to the various VVCs. Note that the VVCs will then immediately start executing the corresponding time-consuming BFM procedures towards the DUT.
Fig 3. VVC vs BFM
The test sequencer thus has full control of what is happening on the DUT’s interfaces and may use the await_completion() and insert_delay() commands to synchronize to any VVC’s BFM execution. await_completion() will stall the test sequencer until a given VVC has finished executing either all its commands or just a given command. This way, the test sequencer has full overview of the testbench status and knows when to issue commands to the various VVCs. The insert_delay() command allows the test sequencer to offset or skew VVC interface handling with respect to each other, and thus target cycle-related corner cases.
Overview, readability, maintainability, modifiability and extensibility
If your DUT is quite simple, you can probably manage with BFMs only. The VVCs are intended for medium to high complexity modules and FPGAs (or ASICs), but note that complexity here is seen from a verification point of view. As such, even a simple UART would benefit from using VVCs. This is, of course, also a question of quality. If you can accept a UART byte error every 10,000 bytes, then you can lower your verification level and effort. If not, VVCs will help you a lot in detecting those types of bugs.
For simple testbenches, the use of the UVVM Utility Library and BFMs would be sufficient, and if not, then at least a very good starting point. You might think that the improvement potential is not that big for a simple testbench, but even for a small project you could easily have a saving potential of 50-300 hours (or in the range 10-30% or more on the total project time), and at the same time improve the quality. For more complex modules or FPGAs, the savings potential could be much higher, even percentage-wise.
AXI-Stream VVC example
The AXI-Stream interface is used in many FPGA projects today and when making a testbench for a DUT with one or more AXI interfaces, you can use the VVC approach to detect potential problems.
Figure 4 illustrates how the test sequencer issues two commands to the test harness: the axistream_transmit() to the master VVC, and the axistream_expect() to the slave VVC.
These two commands are issued at the same time (only delta cycles apart) from the test sequencer to the VVCs. Transmission will then start at the same time from the master VVC towards the DUT, whereas the slave VVC will be waiting for data to arrive out of the DUT, and then check the actual data received against the expected. The protocol is then obviously also confirmed.
Beware, however, because while the AXI stream protocol allows feed-forward control and backpressure, experience shows that this mechanism can be error-prone. For that reason, UVVM has introduced both directly and randomly controlled manipulation of the ‘valid’ and ‘ready’ signals in the AXI protocol, for deactivation at any given position and for any number of clock cycles. According to many UVVM users, this feature has enabled lots of bug detection in their design. UVVM also has the largest number of free Open Source VHDL BFMs and VVCs available.
Taking it one step further
In future articles here in the FPGA Horizons Journal, we will look in more detail at selected important features of UVVM. The next article will be on Requirements Tracking and the Requirements Traceability Matrix. In UVVM, we call this functionality Specification Coverage, and lots of FPGA designers have used this functionality in UVVM to qualify their FPGA for mission-critical or safety applications like, for instance, DO-254, the safety standard for the development of aviation software and hardware.
