(And why it’s suprisingly hard to kill one)

FPGAs in the cloud have changed the way we think about hardware security. In the old days, if you wanted to attack an FPGA, you needed a lab bench, oscilloscopes, EM probes, and a lot of patience to try and retrieve cryptographic keys. With FPGAs now available in cloud infrastructures, you just need a cloud account. And once you can run your own design on someone else’s hardware, it opens the door for remote attacks, mostly denial-of-service, but also the leakage of keys and metadata.

The obvious question also arises: Can you do nasty things to the equipment there?

This triggered a wave of research and related work, including ours in the Novel Computing Technologies group at Heidelberg University. But as we pushed deeper, something unexpected happened. The story stopped being just about attacks and became a study in robustness. We tried very hard to break datacenter-class FPGAs. We stressed them, overheated them, aged them, and crashed them. And they survived far more than we thought possible.

This article walks through both sides of that journey: the attack primitives you can build on an FPGA, and the surprising resilience of datacenter FPGAs when you subject them to extreme stress.

How to turn an FPGA into a ring oscillator

An FPGA is a grid of configurable logic blocks (CLBs) with a network of multiplexers and routing channels which provide the infrastructure for switching different components together. Inside each CLB you’ll find Lookup Tables (LUTs), flip- flops, and local routing. And the logic itself is created by the LUTs, which are the easiest form of function generators. It’s just a tiny memory where you say for a couple of input bits what the result should be.

With that alone, you can build almost anything, including a malicious circuit like a ring oscillator. Feed the output of an LUT back into its input with the right truth table and it oscillates. And once you have a ring oscillator, you have a temperature- and voltage-dependent sensor. You also have a primitive that cloud providers really don’t want you to deploy because it can cause interference or even create a security concern in a shared environment.

Providers, for example, typically scan for ring oscillators. But if you hide one behind a transparent latch, it passes Design Rule Checks (DRCs) just fine. And once it’s on the chip, it can spin at 1.6–1.8 GHz. With a simple time-to-digital converter, you can measure its frequency with sub-10 picosecond accuracy. That gives you a remote oscilloscope, if you like, which is perfect for side-channel analysis. It also, of course, could be used for side-channel attacks, although this is not a significant industry threat. It’s primarily explored in academic work.

It is, though, where the attack story begins.

Denial-of-Service by design

Side channels are fun, but denial-of-service is where things get interesting and there are different ways to approach it.

Short-circuits through corrupted bitstreams

Multiplexers inside an FPGA are implemented using pass transistors or transmission gates. Under normal operation, only one input is ever selected. But if you activate multiple inputs at the same time and drive some inputs to 1 and some to 0, you create a short circuit. I discovered this accidentally while generating my own bitstreams and ended up damaging a chip.

That’s why I started this whole research exercise, trying to understand what was going on. What I found was that if I activated all MUX inputs on an FPGA, putting all configuration bits to 1, and then applied a specific input pattern, the device drew up to 7 mA extra. That doesn’t sound like a lot, but an FPGA has an enormous number of LUTs – hundreds of thousands eventually – and they have multiple inputs, so it all adds up.

Power-hammering with shift registers

Or take something simpler: a shift register. On a datacenter FPGA you have something like 20–30 million flip-flops through dedicated shift register primitives, and you can initialize them with a 1010 pattern in a ring configuration and shift it at around half a gigahertz. This creates a much larger power draw of something like 2 kW, and you can control it by increasing load steps on a per-cycle basis, which in turn creates voltage droops and sets the stage for a fault- injection attack. For instance, a single cycle shift will cause a voltage drop which may result in some parts of the remaining user design not being able to switch fast enough.

If a soft CPU is implemented and a compare instruction is executing, that droop gives you the window you need to activate the attack. For example, you can insert a fault by not giving the circuit enough time to set the condition flag correctly, causing the corresponding branch to take the wrong direction. You get the idea.

Higher frequency oscillation using glitch amplification

Another fun one is glitch amplification. We start with a toggle flip-flop that switches once per clock cycle. If I route that signal through two paths with different latencies to an XOR gate, each time my signal arrives at the XOR inputs with a slight offset, it will toggle the output. That allows me to generate faster switching activity.

Even if my signal isn’t fast enough, I can make it faster. Because the XOR output now toggles twice per input transition, I can route that output back into the clock input and get a self-running oscillator at ~0.5 GHz that burns a surprising amount of power. This principle can be pushed even further by driving multiple global clock trees with phase-shifted clocks and XOR-ing them together in an LUT. By using four global clocks overclocked to about 2 GHz, I could generate a toggle rate of ~8 GHz at an LUT (XOR) output.

These are all attack primitives that can be deployed in the cloud. But they also became tools for something else: pushing the hardware to its limits.

Creating faster and hotter ring oscillators

We looked into all kinds of ring oscillators to see how fast they can spin, because fast ones are specifically good for side-channel analysis and eventually also for power draw. We saw that most are equal, but a few are more equal than others, with one that spun at a frequency of around 6 GHz. We did this for 2,000 ring oscillators in parallel to better measure the impact and saw an average power draw of around 2 mW per LUT, or 16 mW for a cluster of eight.

The key to creating fast oscillators is to minimize latency, both in the routing and inside the LUT that implements the inverter. A ring oscillator’s frequency is set by how long it takes for a transition to propagate through the logic and back around the loop, so the shorter that path, the faster it spins.

If you look at how LUTs are physically implemented, it’s basically a multiplexer tree. If I switch something at the base of the tree, which is at entry point 10 on the left of Figure 2, it has to traverse all the way through the LUT. But if I use entry point 15, the path in red goes straight to the top of the tree and goes a little bit faster. The 6 GHz oscillator we saw above uses this path, which is just 41 picoseconds from input to output. And this also has a super-fast path from the output back to the input.

Now let’s render cloud FPGAs useless

Once we understood how to build fast and power‑hungry oscillators, the next step was obvious: take them to the cloud and see what happens. Cloud instances are virtualized to the point where you don’t know which physical FPGA you’re talking to. You do, however, have the ability to run your own design and that was enough.

By simply running a set of ring oscillators in the user logic and measuring their speed, we could build a so-called physically unclonable function – a PUF. Depending on process variations, each of these oscillators spin at their own frequency which acts like a device fingerprint that reveals the cloud instance. And even better, the speed of the oscillators gets biased if the FPGA instance was in use before. With this, we not only know which physical FPGA instance we are talking to but also if that instance was in use before.

We took our thinking and applied it to an FPGA with the same specifications used by cloud providers, generating a design with kilowatts of power- hammering potential. It passed all the checks, meaning it could be deployed in the cloud, even though we only used a much smaller 360 W version for actual cloud experiments.

To test attacks, we firstly contacted the cloud service provider and asked for permission to run experiments … which they gave us. We then repeatedly launched and released FPGA instances in the cloud and then crashed them with a 360 W power-hammering design. We wanted to do something where we made a statement without creating real harm. When we didn’t crash the instance, we’d get the same instance back almost immediately. When it did crash, the turnaround time jumped to an hour or more. We continuously requested instances, released them, requested them, released them and did this for over 100 instances before we were finally detected and stopped by the cloud provider.

We didn’t receive feedback from the cloud provider, but the long downtime suggests that some instances may have required a cold start, etc., for bringing them back to life.

Okay, so let’s try and kill an FPGA

The next phase of the work was not just hammering cloud FPGAs, but understanding how close you can get to the physical limits of a datacenter FPGA. The real issue was not deploying kilowatts of power because there is some overpower protection logic that will kick in and protect the chip. It was more like, could we deploy 100 W on the tip of a needle? Could we fry a chip with a hotspot? Or at the very least damage or age it substantially?

That would take two steps – finding a way to assess the damage, and then doing the damage.

Assessing the damage

We didn’t have a microscope, so we had to think differently. We needed a way to see whether the chip was slowing down, aging, or behaving differently under extreme stress, but from the inside. Instead of looking at the silicon, we looked at timing. The performance of a chip is basically defined by how long a signal takes to travel between two flip-flops. If the silicon ages or heats unevenly, that delay changes. And if you can measure that delay precisely, you can literally watch the chip degrade in real time.

We used two clocks that we could phase-shift against each other, and toggled a signal from zero to one and sampled it with the shifted clock. First, we read zeros, then it started flickering, and eventually it became stable. By cascading the clock managers in a clever way, we got phase-shift steps of about a picosecond, which was enough to characterize what was happening on the chip.

Doing the damage

In order to drive maximum power to our hotspot, we turned to a ring oscillator again. We used all of the possible outputs we could and routed the signal through other LUTS to get maximum fan out to drive the 6 GHz we could generate through everything on the chip. We played around with the design to maximize power consumption, and that gave us around 100 mW per CLB. If you scale that up, that’s something like 15 kW. And remember, the DSP blocks and block RAMs were still there, untouched.

We also looked at aging by examining the difference between rising and falling edges, because transistors age differently. We ran experiments for two to three weeks. After day two, we saw a jump in delay of around 0.3 picoseconds, but the aging wasn’t equally distributed. Most wires don’t age, or age just a little bit. But a few are real outliers. Still, none of them were fried, and some of the aging even gets better over time, so the chip could actually self-heal.

Those early results were on a smaller FPGA, something like 20 W which I thought would fry the chip. But then we turned to a datacenter FPGA and went all in. We created a 135 W hotspot in just 1.2% of the device and attacked the chip for three weeks. The average performance degradation was 1.39%, which doesn’t sound like a lot but, interestingly, the distribution was very wide. A few wires aged by 70% and were still alive but we had only a few wires that aged more than 50%. 

What does all of this mean in practice?

If you have an aging and slow path in an FPGA, that alone won’t make everything slower. A critical path is made up of multiple segments. If only one of them is a slow outlier, it only impacts you to some extent. And not all paths in your design are critical. A non-critical path can slow down without biting you. In reality, the performance reduction we saw probably translates to something like 15–20%, which would mean a 500 MHz design could still operate safely at around 425 MHz.

Truth is, damaging FPGAs is much harder than I thought. I would have bet a crate of beer that 20 W would do the trick, and even 135 W didn’t. One reason is that an FPGA has a lot of dark silicon with about a gigabit of non-toggling configuration bit cells and a large number of multiplexers that are pretty big, and you only have one path that you route through, with the rest sitting idle. Silicon is also an incredibly good heat spreader.

Where this leaves us

Cloud FPGAs are powerful, flexible, and if you push them in the wrong ways capable of leaking information, inducing faults, crashing instances, and aging silicon. The attack surface is real. You can build oscillators that double as remote sensors, hammer power rails, and create damaging hotspots.

But the deeper we pushed, the more the story changed. Datacenter FPGAs don’t die easily. Even under 6 – 8 GHz oscillation, multi-kilowatt-equivalent stress patterns, and weeks of concentrated thermal and electrical abuse, the devices degraded only modestly on average. A few wires aged dramatically, but the chips kept functioning.

So the conclusion is more nuanced than the attacks alone suggest. Cloud FPGAs can absolutely be misused, and providers need better scanning, tighter isolation, and stricter control over clocking resources. But actually rendering the hardware permanently useless and killing the device is far harder than expected. Even aggressive, sustained, high density stress patterns only nudged performance rather than destroying it.

In the end, the real lesson isn’t how to break a cloud FPGA, but to make them robust and prevent such attacks.