AMD Patent Reveals Hybrid CPU-FPGA Design That Could Be Enabled by Xilinx Tech

I can&#x27;t help but think most commentators haven&#x27;t actually read the article or the patent. This isn&#x27;t about having an FPGA embedded into the CPU or near the CPU, it&#x27;s about having a programmable FPGA like execution unit that can be programmed to be say a 4-bit floating point adder, or any other weird execution unit one might need.<p>Why is this important? Have a program that does a lot of integer multiplications? Let&#x27;s program all of these programmable execution units to multiply integers on the fly, etc. Now your integer multiply throughput is higher, as per the current program&#x27;s needs.<p>Have lots of weird old x86 instructions you are forced to support but no one actually uses? Don&#x27;t waste transistors on them just program an execution unit to execute that instruction on the fly, etc.<p>I think it&#x27;s great, and that most people are missing the point.

How patents happen:<p>An alarm goes out: our company has fewer patents than company X! In fact, we have the smallest patent hoard of our competitor group. If they sue us we might not have enough patents to sue them back! We must have more patents! Everyone who gets a patent gets a bonus! ( Exit CEO, trailing exclamation marks. All the engineers file their pet idea as a patent, hoping management will be interested in building it).<p>(Some years later) Okay, some of those patents we filed are a bit silly. But at least we now have a huge, intimidating patent pile! No one will dare sue us now! Mua ha ha! But let&#x27;s be a bit more careful what we give those patent bonuses for. (Meanwhile at company X: our company has fewer patents than company Y!...)<p>The above is a true story, happened to me. Well, apart from the moustache twirling. My name is on some not very practical patents. So I&#x27;m not very convinced by stories which read the tea leaves from patents as to what a company intends ( Or economists trying to infer innovation rate from patent filing rate). Another problem is that the patent office is slow. Unless the company is General Fusion, most probably the product will be out before the patent.

Everything described in the article sounds exactly like some of the Virtex*-FX products from more than 10 years ago.<p>For instance, the Virtex4-FX had either one or two 450MHz PowerPC coresembedded in it, where you could implement 8 of your own additional instructions in the FPGA. This is effectively now a CPU where you can extend the instruction set, and design your own instructions specific to your application. For example, you might make special instructions using the onboard logic to accelerate video compression, or math operations; I know of one application that was designed to do a 4x4 matrix multiply per cycle.<p><a href="https:&#x2F;&#x2F;www.digikey.com&#x2F;catalog&#x2F;en&#x2F;partgroup&#x2F;virtex-4-fx-series&#x2F;76285" rel="nofollow">https:&#x2F;&#x2F;www.digikey.com&#x2F;catalog&#x2F;en&#x2F;partgroup&#x2F;virtex-4-fx-ser...</a> <a href="https:&#x2F;&#x2F;www.xilinx.com&#x2F;support&#x2F;documentation&#x2F;data_sheets&#x2F;ds112.pdf" rel="nofollow">https:&#x2F;&#x2F;www.xilinx.com&#x2F;support&#x2F;documentation&#x2F;data_sheets&#x2F;ds1...</a>

I hate to be that bucket of cold water, but there&#x27;s <i>multiple</i> reasons FPGAs haven&#x27;t been successful in package with CPUs. Firstly, the costs of embedding the FPGA - FPGAs are relatively large and power hungry (for what they can do), if you&#x27;re sticking one on a CPU die, you&#x27;re seriously talking about trading that against other extremely useful logic. You really need to make a judgement at purchase time whether you want that dark piece of silicon instead of CPU cores for day to day use.<p>Secondly, whilst they&#x27;re reconfigurable, they&#x27;re not reoconfigurable in the time scales it takes to spawn a thread, it&#x27;s more like the same scale of time to compile a program (this is getting a little better over time). Which makes it a difficult system design problem to make sure your FPGA is programmed with the right image to run the software programme you want. If you&#x27;re at that level of optimization, why not just design your system to use a PCI-E board, it&#x27;ll give you more CPU, and way more FPGA compute and both will be cheaper because you get a stock CPU and stock FPGA, not some super custom FPGA-CPU hybrid chip.<p>Thirdly the programming model for FPGAs are fundamentally very different to CPUs, it&#x27;s dataflow, and generally the FPGA is completely deterministic. We really don&#x27;t have a good answer for writing FPGA logic to handle the sort of cache hierarchy, out of order execution that CPUs do. So you&#x27;re not getting the same sort of advantage that you&#x27;d expect from that data locality. It&#x27;s very difficult to write CPU&#x2F;FPGA programs that run concurrently, almost all solutions today run in parallel - you package up your work, send it off to the FPGA and wait for it to finish.<p>Finally, as others have said - the tools are bad. That&#x27;s relatively solvable.<p>For me, it boils down to this, if you have an application that you think would be good on the same package as a CPU, it&#x27;s probably worth hardening it into ASIC (see: error correction, Apple&#x27;s AI stuff). If you have an application that isn&#x27;t, then a PCI-E card is probably a better bet - you get more FPGA, more CPU and you&#x27;re not trading the two off.

Everyone seems to be talking about accelerated instructions but how about I&#x2F;O?<p>FPGAs are awesome at asynchronous I&#x2F;O and low latency. We could implement network stacks, sound and video processing, etc... It can start a TLS handshake as soon as the electrical signal hits the ethernet port, while the CPU is not even aware of it happening. It can timestamp MIDI input down to the microsecond and replay with the same precision. It can process position data from a VR headset at the very last moment in the graphics pipeline. Maybe even do something like a software defined radio.<p>Basically every simple but latency-critical operations. Of course, embedded&#x2F;realtime systems are a prime target.

A killer tech for this would be a framework that automatically reprograms the FPGA and offloads the work if it makes sense. For example - running k-means? Have your FPGA automatically (with minimal dev effort) flash to be a Nearest Neighbor accelerator.<p>The problem is finding a way to make that translation happen with minimal dev effort, as software is written rather differently from hardware.

The main reason I am interested in this acquisition is a (faint) hope that they open some specs up to help projects like SymbiFlow.

For decades, the FPGA vendors have had this fever dream of &quot;an FPGA in every PC&quot; -- either as an add-on card, or as part of the chipset on a motherboard -- that would enable a compiler or operating system to seamlessly accelerate arbitrary tasks on demand.<p>In my opinion, the problem has always been their software: the FPGA vendor tools are slow, bloated monstrosities. The core of these tools are written by the big three EDA vendors (Cadence, Synopsys, and Mentor Graphics) rather than the FPGA vendors themselves. The licenses include ridiculous, paranoid restrictions [1] and force the FPGA vendors to keep their bitstream formats and timing databases secret [2] in order to prevent competition from other tool vendors. Most FPGA vendors didn&#x27;t see this as a problem, but even the ones that did didn&#x27;t have much of a choice, because the tool market is a cartel.<p>Thankfully, we now have an open source toolchain [3] with support for a growing number of FPGA architectures [4], and using it vs. the vendor tools is like using gcc or llvm vs. a &#x27;90s era, non-compliant C++ compiler. It even has a real IR that isn&#x27;t Verilog, which has made it easier to design new HDLs [5].<p>I don&#x27;t see how a dynamic FPGA accelerator platform can be even remotely viable without this. It&#x27;s the difference between a developer getting to choose between one of a few dozen pre-baked designs that lock up the entire FPGA (and needing to learn how to shovel data into it), vs. a compiler flag that can give you the option of unrolling any loop directly into any inactive region of FPGA fabric.<p>It would be quite the cherry on top to see AMD build something interesting in this space. But unless they&#x27;re willing to fully unencumber at least this one design, I think the effort is likely to fail. The open source guys are chomping at the bit to make this work, and have been making real progress lately. Meanwhile, the EDA vendors have been making promises, failing, and throwing tantrums for the last 20 years. It&#x27;s time to write them off.<p>[1] <a href="https:&#x2F;&#x2F;twitter.com&#x2F;OlofKindgren&#x2F;status&#x2F;1052822081652617221?s=20" rel="nofollow">https:&#x2F;&#x2F;twitter.com&#x2F;OlofKindgren&#x2F;status&#x2F;1052822081652617221?...</a><p>[2] Imagine trying to write an assembler without being allowed to see the manual that tells you how instructions are encoded. It&#x27;s like that, but the state-space is hundreds to thousands of bytes in multiple configurations rather than a few dozen bits.<p>[3] <a href="https:&#x2F;&#x2F;github.com&#x2F;YosysHQ&#x2F;yosys" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;YosysHQ&#x2F;yosys</a><p>[4] <a href="https:&#x2F;&#x2F;symbiflow.github.io&#x2F;" rel="nofollow">https:&#x2F;&#x2F;symbiflow.github.io&#x2F;</a><p>[5] <a href="https:&#x2F;&#x2F;github.com&#x2F;m-labs&#x2F;nmigen" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;m-labs&#x2F;nmigen</a>

This looks a bit like the old (2000s) work of Leopard Logic or Tensilica. Exciting stuff.<p>One important note (based on some comments here): generally, these in-CPU FPGAs have <i>very</i> fast reconfiguration. Not sure if it&#x27;s 1, 10 or 100 cycles but it&#x27;s not milliseconds. Actually, (in past examples) configuration might take milliseconds but it would load a number of planes of configurations: plane 0 might be MP3 audio device; plane 1 might be MPEG2 video device. Then reconfiguration is: switch to plane 1.<p>This AMD proposal looks like it&#x27;s much more tightly integrated into the CPU so it&#x27;s got to be even faster. Combine that with the deep knowledge of processor internals you&#x27;ll have to have to code for this thing and I&#x27;m having a hard time seeing you and me having much luck tinkering. This is probably 99.99% data center with gnarly NDAs and field support.

Sounds like spending a few hours a month learning an HDL could be a good long-term career decision.

this approach is not new, and has been toyed around since the 1960 (!), see G. Estrin&#x27;s work on adaptive architectures for example.<p>i got to know about this as part of PRISM (processor reconfiguration through-instruction set metamorphosis) work in the early 90&#x27;s. there is a very cool paper by the same name. check it out !<p>ps : PRISM Paper (<a href="http:&#x2F;&#x2F;class.ece.iastate.edu&#x2F;tyagi&#x2F;cpre583&#x2F;documents&#x2F;prism.pdf" rel="nofollow">http:&#x2F;&#x2F;class.ece.iastate.edu&#x2F;tyagi&#x2F;cpre583&#x2F;documents&#x2F;prism.p...</a>)

I could never stop wondering why is this not a norm yet. Why doesn&#x27;t every computer have an FPGA.

How fast can an FPGA be reprogrammed? If I close my FPGA accelerated machine learning training algorithm, and then open a PC game, would it be feasible to load the new gaming-oriented instructions in ~10-30&quot; that a PC game takes to open?

Lisa Su is a fantastic CEO. Time will tell what the impact of AMD’s acquisition of Xilinx will be (should it close), but this shows the strategy and execution behind Su and team.<p>While a lot of acquisitions don’t pan out, this seems great.

This is exciting! Would be cool if it could access some sort of gpio as well!

About *!$% time! I was hoping Intel would do something like this when they acquired Altera a few years back. Does anyone know why Intel acquired Altera?

This seems more appropriate for GPUs than for CPUs where it’s high throughput and you can eat the latency cost of reconfiguring the node.

Xilinx already have ARM cores in their FPGAs so I wonder which way they&#x27;ll go - I&#x27;d honestly prefer a neoverse core than an X86

I think neural networks and AI might be a good application area for this.

Computronium

Yet another patent that should never have been granted.<p>SoC have been a thing for a long time. SoC = CPU + FPGA on a single chip.<p>Looking at the patent, the list of 20 claims is absurd. The title says it all &quot;... PROGRAMMABLE INSTRUCTIONS IN COMPUTER SYSTEMS&quot;, they&#x27;re trying to patent anything that can run or dispatch instructions.