Unconventional Uses of FPGAs

If you work on hardware long enough you run into all kinds of things that make you realize just how little you know.

I had an interesting case around fifteen years ago. I had designed a real time image/video processor using a large Xilinx FPGA at the center of it. The system worked very well, except that it would have problems when flown on a helicopter. Actually, only certain kinds of helicopters, with certain types of cameras.

I scratched my head for months trying to solve this riddle.

Months later, while working on a completely unrelated problem, I touched the FPGA and burned my finger. It was surprisingly hot. This thing ran cool as can be, even without a heatsink.

What happened? At first I thought I had a bad board, so I swapped it out. Sure enough, it got very hot, very quickly.

Huh?

That's when I realized I was using footage from one of the helo's. For some reason my brain made the connection. I instantly knew why we were having problems with helicopter-mounted cameras and why I burned my finger: Vibration.

CMOS consumes power on state transitions, not so much once you reached the final state (low or high). More transitions per unit of time means more power. If the transitions are high speed edges (which is the case for real-time image processing) even worse.

The helicopters in question did not have motion-stabilized cameras. They were hard-mounted to the verticals of the skids and that was that. Which means that every bit of vibration from the helo made it onto the camera. Add that to the helo travelling and you have a perfect storm: Every pixel was changing on every frame at 120 frames per second. The translation was a massive increase in power consumption and heat dissipation. Hence my burned index finger and the problems we had with helo-mounted systems.

So, yeah, the power consumed by an imaging system is a direct function of the rate of change per unit of time of pixel data. A still frame consumes nothing when compared to a busy sequence where every pixel is changing. Logical, yet not obvious when you are not thinking in those terms.

You have to burn your finger (or knock stuff off your desk) to learn this kind of thing.

Stories like these are my catnip. The 500 mile email is another great one.

How did you end up debugging/reproducing? I conjured a picture in my head of an engineer comically exasperated and squeezing the controller followed by an “Aha!”

This is definitely one of the reasons I wrote this article. To be able to see the cause/effect when a problem is very convoluted is something difficult to learn and difficult to teach. You just have to consume a lot of second hand stories and experience a few yourself.

A story I have heard several times in different instances is some light sensitivity, I think this happened to the Raspberry Pi team (PMIC transient induced by camera flash).

Eagerly awaiting Mordechai's paper on this one.

Can't do same with ASICs.

You can make latches with ASIC perfectly well. Or even more complex memory elements.

Can't do same with ASICs.

shrug I can't really say anything except that's not the only way to accomplish that sort of thing. And if it's a common enough use case that you need to build it into millions of pieces of ammunition, it doesn't explain why economies of scale never kicked in or a more purposeful custom part was never designed.

Limitless military budgets, by contrast, explain quite well why this /hasnt/ happened.

The cost of making a chip is proportional to the size of the chip and the process node, and that's the end of the story. FPGA and CPU quite literally cost the same to manufacture; it doesn't explain why the one categorically is priced 10 times higher than the other by die area, even in the long tail of the supply chain.

The FPGA in my scope probe that is dutifully implementing the functionality of an EEPROM while loading its configuration from an even larger EEPROM exists in a state of extreme disagreement to your comment.

Yeah obviously FPGA's are unquestionably awesome devices, but IMO the real problem with them is they have no long tail or economies of scale compared to other device types. Having worked with them personally, I mostly believe this is because there aren't really any great cross vendor standards analogous to a CPU's ISA. The lack of interoperability absolutely destroys the ability of competition to benefit customers. I'd like to be able to buy a modern $5 part with capabilities equivalent to something like a Cyclone IV, but there's nothing even remotely close.

I kind of empathize where they are coming from, my electronics version of "this meeting could have been an email" is "this fpga could have been an arduino".

I feel like we're constantly spoiled by the clock speed of computers, they are good enough for so many things. FPGAs can make simple things a bear to implement, and you will just get there faster.

But there's a bunch of things where FPGAs are like discovering fire, all the discrete digital logic you ever wanted in the palm of your hand (until you have to meet timing).

FPGAs aren't inherently secure. I'd assume that a nation-state-level actor would have the ability to extract the bitstream from an operating FPGA which would invalidate both of your statements if additional protections and security considerations are not met in the hardware design. But you are correct that an FPGA readout is a far more difficult attack than, say, bypassing the read protection on a microcontroller's flash. At the same time, it might be easier to attack an FPGA than a modern CPU's secure area/secure enclave.

That is all to say the security is not coming the FPGA but from a systems design process that prioritizes security. There are a number of solutions to the problem which may or may not involve FPGAs but when you scramble it all up along with the preverse incentives driving the pricing/costing of these contracts, it's pretty clear that good+expensive solutions are chosen more often than good+inexpensive solutions.

Isn't that how those Fortezza crypto cards worked? The chips leave the factory unclassified, then the NSA loads their classified Suite A algorithms onto them?

Probably compared to micro controllers that are getting fast enough to do a lot of what used to take custom hardware or FPGAs

The Fujitsu device is a synchronous digital architecture. I'm using asynchronous oscillators, more like Lo et. al. (https://www.nature.com/articles/s41593-024-01607-5)

I personally think it's super cool that we can replicate analog coupling dynamics using asynchronous digital logic!

This is a personal project aiming to replicate analog coupling dynamics with asynchronous digital circuits FPGAs, not a claim that we can solve NP complete problems with a fundamental speedup.

Also, these sorts of Ising architectures get their speed-up are based on oscillator interaction, not parallelism. It's relevant to this article because it leverages oscillators on FPGAs in a nifty way :)

Usually one builds in a better temperature sensor and calculates a correction function of temperature.

Yes, FPGA-based, and no ring oscillators. AWS doesn't allow you to use latches or comb loops at all, but has the same area efficiency as the best RO-based designs. We'll publish eventually.

For the FPGA users out there - we will be packaging a few shapes of RNGs as IP in the near future, but this kind of chip/FPGA IP needs a LOT of very subtle verification, which made a lot more sense to do for full cards first.

It's not uncommon in sensitive analog systems to do something like this, ~but I haven't seen anyone doing it with an FPGA yet~. Very old analog systems (that were a lot more temperature-dependent than modern analog circuits) often were contained in small ovens, and an intermediate step between a normal oscillator and an atomic clock today is to use an oven-controlled oscillator. In both cases, they keep the ambient temperature on the analog devices at 40-50 C to make sure that they operate at a stable temperature regardless of ambient temperature.

Edit: See the sibling comment - someone has found a reasonable circumstance to do it!

As a musician I absolutely love my "so small/light forgot I had it in my backpack"-500W Bass amp, compared to a 36 kg tube one. Class D is the shit.

I will give Chord some credit for the idea of running the DAC super fast. I play music and occasionally record myself, and I can tell you that when you look at the waveforms, there is definitely a component of the human ear that can distinguish time intervals faster than 1/(22 kHz) despite being unable to hear frequencies over about 15-20 kHz (eg very tight flams in drum parts are audible). DSD had this idea, and I am 100% on board with it. Despite being a gold standard because of how it's produced, lossless CD audio cuts off this "high-frequency" component.

However, the actual analog parts of Chord's system around the fast DAC and the upscaling circuit seemed to be sort of suspect to me, from the handling of reference clocks to the nature of the output amplifiers. I'm glad ASR has confirmed those suspicions before I bought one.

At some point, I started a project making a multibit DAC running at 1 Msps with the same kind of FPGA upscaling (although ADI's SHARC DSPs are amazing) but a much more conventional frontend and better handling of clocks, but I realized it was too complicated to do while also doing all the other things I was doing at the time (and way too complicated now). The idea is still on the shelf for later (although it's probably already obsolete as a concept - delta-sigma is really good).

Agreed. I personally think it is the ultimate piece of equipment for sound reproduction from computers or personal recordings, unless one has a set of very specialized cans that it could even drive. But of course sound sensitivity is subjective and there are limits to what some people care about so their may exist cheaper options with some comparable quality, but at some point I gave up to quest with that little gadget.

20 years ago... This paper caused me to go all in on evolutionary algorithms, and eventually burn out because I could never get any kind of promising result out of it. Wrong choice I guess!

Wow. 36 years ago, I was super excited about neural networks, but lacked the skill to get good results fast enough; meanwhile, a very talented colleague was getting really interesting results from evolutionary algorithms.

It can be hard to move forward all alone.

Yeah, I tried my hand at some evolutionary algorithms and found it's all about the fitness function. I'd spend hours and hours refining it and eventually realized that you're really just optimizing the function to fit what you want to see. Kinda took the magic out of it for me.

Thank you very much. Do you have general suggestions of where to look? Do I "just" need to learn analog circuit design?

Links/pointers? I know only of Achronix' original Speedster series, which was alas abandoned for a conventional architecture ("Power" was the reason I was told).

I edit my comment, but pre FPGA, much of our missile guidance electronics was entirely analog, and thus observable if your buddy say, misplaced one in the side of a mig.

Now the bitstream can be downloaded as needed, and bit rots quickly after power loss.