Data centre and acceleration

Overview

An FPGA in a data centre is a PCIe card or a network card with a large device on it, programmed to do one thing faster or cheaper than the server’s processors. The successful cases share a shape: a stream of data that must be handled at line rate, a function that changes rarely, and a reason a GPU does not fit (latency, power, or the data never needing to reach memory at all). This hub is for engineers evaluating or building such accelerators, and for the people who have to decide whether to pay for them.

Microsoft’s Catapult project, which put an FPGA in every Azure server from 2015, made the idea respectable. The decade since has sorted it into three durable niches and several that did not last. The durable ones are network processing (SmartNICs, firewalls, telemetry), ultra-low-latency finance, and specialised streaming offload such as compression, encryption, search and some database operators. The ones that did not last were general-purpose compute offload in competition with GPUs, which won on software.

Key ideas

The data path, and why the host is the bottleneck

A PCIe Gen4 x16 link carries roughly 32 GB/s each way, Gen5 doubles it, and a 400 Gb/s Ethernet port needs 50 GB/s. Every byte that crosses the link costs host memory bandwidth, a DMA descriptor and often an interrupt. The best accelerators therefore keep data on the card: packets come in on the network port, are filtered, parsed or encrypted in the fabric, and leave on the other port or go to host memory already reduced. Host-FPGA DMA engines (AMD XDMA and QDMA, Altera’s PCIe multi-channel DMA, and open designs such as Corundum’s) manage descriptor rings and scatter-gather lists so the processor sees a block device or a NIC rather than a PCIe BAR. RDMA and RoCE v2 move data between hosts without involving either processor, and an FPGA can terminate RoCE in hardware. CXL, which runs over the PCIe physical layer, lets an FPGA share coherent memory with the host at the time of writing on Agilex 7 and Versal parts, which removes the copy for memory-expansion and near-memory compute cases.

Latency: the finance case

High-frequency trading is the purest FPGA application. A market data feed arrives on Ethernet, the FPGA decodes it, applies a pricing rule and sends an order, and the whole path from wire to wire takes under a microsecond on the best designs, with tens of nanoseconds of jitter. No processor and no kernel can match that, because the FPGA has no cache, no scheduler and no interrupt. The engineering is in cut-through processing (start on the header before the packet has arrived), in avoiding any serialisation, and in the Ethernet PHY itself, where a vendor’s hard MAC may add more latency than the whole trading logic. The same skills transfer to telecom fronthaul, industrial control and test equipment.

The economics

A large data-centre FPGA card costs several thousand USD, draws 75 W to 225 W and needs an engineer who costs more than the card. It pays off when it replaces several servers, when it enables a latency or line-rate target that nothing else can hit, or when it moves a function (encryption, compression, RAID parity) off processors that are then sold as compute. It does not pay off when the function could run on a GPU with a mature library, when the workload changes faster than a hardware team can follow, or when the volume is high enough that an ASIC (or a merchant DPU from NVIDIA, AMD Pensando or Marvell, which are themselves FPGA-prototyped ASICs) is cheaper. Hyperscalers who use FPGAs do so because they can amortise the engineering over a hundred thousand servers. Most smaller operators are better served by buying the finished card from a vendor who did that work.

Where to start

The building blocks are FIFOs, pipelines and clock-domain crossings at high clock rates, so /learn/beginner/09-memories-block-ram-and-fifos/ and /learn/beginner/04-registers-clocks-and-state/ are the foundations, followed by /topics/high-speed-io/ for the transceivers and /topics/high-level-synthesis/ for the Vitis and oneAPI kernel flows. A first hands-on step is a Vitis vector-add example on an Alveo card or on a cloud FPGA instance, then the Corundum NIC on a board you own. An advanced track on networking and acceleration is coming.

Tools and vendors

The AMD Vitis and Altera oneAPI flows let software engineers write kernels in C++ and SYCL, with a prebuilt shell handling PCIe, DMA and memory. They are the fastest way to a working accelerator. They are also several times less efficient than RTL for the fast path, so most production cards mix a C++ control plane with hand-written RTL on the wire.

Common mistakes

• Building an accelerator for a function that is bandwidth-bound on the PCIe link, so the card is slower than the host doing it in place.
• Measuring kernel time and ignoring DMA setup, driver overhead and the copy to pinned memory.
• Choosing an FPGA for a workload with mature GPU libraries and no latency requirement.
• Ignoring the shell. The vendor’s static region can take a third of the device and all of the best transceivers.
• Underestimating driver and deployment work, which outweighs the RTL on most projects.
• Forgetting thermal limits. A 225 W card in a 1U chassis needs the chassis airflow to be designed for it.

Going deeper

• Putnam et al., A Reconfigurable Fabric for Accelerating Large-Scale Datacenter Services, ISCA 2014, the Catapult paper and project page.
• AMD PG195, DMA/Bridge Subsystem for PCI Express (XDMA), the host-FPGA DMA reference most cards still use.
• AMD Alveo adaptable accelerator cards and the Vitis unified software platform.
• Intel oneAPI for FPGA, the SYCL flow for Altera devices (the pages were moving to altera.com at the time of writing).
• Compute Express Link consortium, for the CXL specifications.
• Corundum, an open-source 100G NIC design with a complete driver and DMA engine.