As FPGA-based System-on-Chips (SoCs) evolve, where the combination of programmable logic (PL) and multiple CPU cores make them suitable for embedded processing and Linux, a review of the tooling used should also be considered.
Various open-source IP core projects exist with verified standard HDL components, but are limited to the PL and fabric domain. Countless developers have also come up with scripts to keep memory-mapped registers in sync between CPUs and FPGAs. At the same time, commercial tools help with the mathematical algorithms in PL, and FPGA vendors provide utilities such as Linux device drivers or HLS compilers.
This leaves SoC developers aiming to deliver feature-rich, portable solutions with a fragmented tooling landscape. Gaps ranging from HDL wrappers to device drivers, and from scheduling mechanisms to user/M2M interaction concepts need to be filled manually. This takes substantial implementation effort to get from the initial system design idea to a first working prototype.
It also comes on top of that required for the project’s core functionality, multiplies with the number of project variants, and continues along the project lifecycle with multiple development environments needing to be kept up to date.
How the Whiznium framework unifies FPGA‑SoC development
WhizniumSBE (Service Builder’s Edition) and WhizniumDBE (Device Builder’s Edition) form an open-source development framework for embedded software that attributes equal importance to the CPU and programmable logic portions of FPGA-SoCs. This helps in the implementation of well-structured, feature-rich applications with flexible system partitioning.
The methodology behind both tools is a model-based design complemented by automated source code generation. In addition, it retains the flexibility to integrate external artifacts from other frameworks and tools.
The model description, in the form of text files, serves as a single source of truth, greatly simplifying project maintainability. Importantly, the source code generator consistently ensures high code quality across all project aspects. Whiznium spans the embedded full stack, from the fabric level across multi-threaded embedded Linux daemons, all the way to web UIs and API device access.
Model structure
Whiznium’s model files follow a consistent header/content line approach, establishing a hierarchical structure by indentation. Each model file covers one distinct gate- or firmware aspect, with terminology already familiar to FPGA and embedded engineers.
WhizniumSBE model files cover:
- Initialization data
- Project definitions
- Database structure
- UI layout
- Job definitions
- Job tree behavior
- Deployment information
WhizniumDBE model files describe:
- Project structure
- Module hierarchy
- Command sets
- Data flows and algorithms
- Fine structure elements such as generics, ports, processes, and FSMs
- Deployment information
Software design principles
To ease the transition to a Whiznium-backed development workflow, only well-established programming languages are employed, with both Whiznium tools written in C++, backed by SQL databases. They generate generic, standard-conforming, human-readable VHDL, C++ and XML/JavaScript, while only relying on few mainstream open-source libraries.
Another cornerstone of the development code process is the ‘all code in plain sight’ approach. For example, the journey of web UI HTTP requests to their handling in C++ code and up to reply generation is easily traceable in the WhizniumSBE project code.
An equivalent example in WhizniumDBE is the traceability of the CPU-side command flow through the stages of bytecode encoding, device read/write and FPGA-side decoding, and handshake generation. This approach also allows users to transition away from Whiznium at any point in a project’s lifecycle if needed.
How Whiznium models and code evolve together
When a project is first generated, Whiznium delivers a comprehensive source code tree derived from the model description, the contents of which can be compiled and synthesized out-of-the-box.
However, the source code tree and the contents of its files are intended to be complemented manually. Whiznium uses the simple yet scalable concept of insertion points – manual sections delineated by tag lines (comments in the respective programming language) in code files, to ensure manual code modifications are carried forward as the code generation process continues.
Whiznium augments rather than replaces the familiar FPGA-SoC development workflow. Code editing and debugging in tools such as VS Code and FPGA vendor IDEs is complemented by altering the model description as the project evolves.
The process of running Whiznium on a project’s existing source code tree is called iteration. During iteration, Whiznium combines the information from the updated model files with internal derive-and-generate functionality to arrive at a full project representation in its internal database.
From that point, a set of template files which include a rich user-extensible library of parameterized IP cores help it to write an updated source code tree, while extracting and re-inserting the manual components of the existing source code tree. The helper tools for WhizniumSBE and WhizniumDBE also automate the iteration process, making it faster and smoother.
Demonstrating Whiznium in practice
To show how Whiznium works in a real engineering workflow, the framework is accompanied by a complete computer vision demonstrator. This demonstrator is a small, self contained FPGA-SoC system built to exercise the full Whiznium toolchain, from model files through gateware and firmware to a working web UI. It serves as a hands-on example showing how Whiznium’s model-based approach translates into hardware control, data processing, and system-level integration on an actual device.
RTL-side handling of high bandwidth camera data
The camera sensor on the demonstrator delivers full-resolution raw frames of at least 2,592 x 1,944 pixels. After de-mosaicing, this becomes a 1,296 x 972 resolution RGB image. At up to 60 fps, the resulting data rate still exceeds 200 MB/s, which is excessive for displaying preview images in the web UI.
To slash the data rate on the RTL side, the WhizniumDBE-backed project uses a decimation module called decim. This takes a single configurable edge parameter and averages over an edge x edge block of raw pixels to generate a lower resolution preview. This is achieved using a sophisticated load/store unit, a block RAM row buffer and a fractional multiplier that approximates the integer division required for averaging.
To achieve basic control of the decim RTL module from the CPU side, it is declared as a controller in the IexWdbeCsx model file. Controllers can be equipped with commands like config(rgbNotGray[bool]) for configuration, set(rng[bool]) to start or stop operation, and getInfo() which returns tixVState[uint8] and tkst[uint32] for status polling.
WhizniumDBE uses this model information to generate a project-specific C++ library exposing these commands to user space and handles bytecode encoding on the CPU side, along with RTL decoding and handshake logic on the FPGA side.
In WhizniumDBE-backed projects, buffer transfers can also be attributed to controllers. They are not copy-free in the same way as DMA but retain the interface-agnostic characteristic of command passing using, for example, SPI or UART instead of AXI-Lite. This feature is used to read out the 38 kB pvwbuf buffer, which is implemented in block RAM as a sub-module of decim.
CPU-side preview image acquisition and delivery
On the CPU side, the WhizniumSBE-backed project is organized into a collection of C++ classes or jobs, with each job having a clearly defined responsibility for either a hardware feature or UI/M2M session control. At runtime, job objects are assembled into an interconnected hierarchy called the job tree. This approach ensures non-interfering hardware access, information sharing and hardware abstraction when needed.
For preview image acquisition, a separate thread inside the job JobWzskAcqPreview to handle configuration, status polling and data transfers from the FPGA side. The buffer transfer target is a three-item result buffer which is initialized once to avoid dynamic memory allocation at run-time.
Once a frame acquisition completes, notification is handled via an external call which is then passed up through the job tree to the UI component responsible for preview display. Each result item is protected by a mutex so the acquisition thread can keep acquiring new preview frames while older ones are being processed upstream in the job tree without causing memory access conflicts.
To display preview images in the web UI, WhizniumSBE auto-generates the infrastructure for message-passing between the multi-threaded daemon and the web UI using HTTP(S). Server-side, this communication is handled through dispatches, which are C++ objects serialized to XML or JSON, with Base64 encoding used for binary data. Standard UI controls such as text boxes, buttons and sliders are derived automatically.
For image display, a custom UI control is declared in the WhizniumSBE model description along with a custom dispatch type. When the acquisition job signals that a new frame is available, this dispatch is populated via straight copy from the FPGA-delivered data and scheduled for transfer to the web UI.
Finally, in the web UI, which performs continuous long-polling to the server, the dispatch arrives. Its RGB content is unpacked into a byte array, after which a custom JavaScript routine updates the HTML5 canvas to display the new preview image.
Conclusion
With Whiznium, a comprehensive open-source development framework for the embedded domain has been introduced. Whiznium’s methodology of model-based design, complemented by automated source code generation, provides the missing ‘glue’ in the currently fragmented FPGA-SoC developer tool landscape, resulting in faster, more coherent results with simplified maintainability.
If you’d like to get started with Whiznium, visit https://github.com/mpsitech where you’ll find the source code repositories, and also https://mpsitech.github.io/Whiznium-Knowledge-Base for deep-dive articles and pointers towards comprehensive documentation and handy reference pages for WhizniumSBE and WhizniumDBE.
Contributions to the Whiznium open-source projects from the community are welcome. We’re particularly interested in open discussions on best coding practices, which could result in those being incorporated in the automated source-code generation; how the current VHDL-only PL-side code generation could be complemented by (System)Verilog output; and wrappers around more vendor standard IP cores like MIPI DSI, PCIe, etc.
