Clock domain crossing without tears

Almost every real FPGA design has more than one clock. A 100 MHz processing pipeline takes samples from an ADC clocked at 61.44 MHz, feeds a 125 MHz Ethernet MAC and reports status over a 50 MHz AXI bus. Each boundary between those clocks is a clock domain crossing (CDC), and each one is a place where static timing analysis gives up. This lesson is for anyone who has passed the STA lesson and now needs to move bits between clocks that have no fixed relationship. It covers the physics of why a crossing can fail, the three structures that cover nearly every case, and how to tell the tools which paths you have made safe.

Why a flip-flop cannot always decide

A flip-flop needs its input stable for a setup time before the clock edge and a hold time after it. Inside a single domain, STA proves that this always happens. Across unrelated clocks there is no such proof. Sooner or later a data edge lands inside that window, and the flop’s internal latch is left balanced between 0 and 1. That state is called metastability. The output is neither a clean 0 nor a clean 1, and it takes an unpredictable time to fall one way or the other.

The decay is exponential. A flop that is metastable at the clock edge has a probability of still being undecided after a time t that falls as e^(−t/τ), where τ (tau) is a property of the flop, typically 10 ps to 50 ps on a modern process. The rate at which a violation happens in the first place is T0 · f_clk · f_data, where T0 is an effective capture window (around 100 ps), f_clk is the sampling clock and f_data is the rate at which the input changes. Putting the two together gives the standard estimate:

MTBF = e^(t_r / τ) / (T0 · f_clk · f_data)

where t_r is the resolution time the design leaves between the first flop’s clock edge and the moment its output is next needed. The whole game of CDC design is to make t_r large.

The numbers are startling. Take τ = 25 ps, T0 = 100 ps, a 200 MHz destination clock and an input that toggles 50 million times per second:

Resolution time available	e^(t_r/τ)	MTBF
0.5 ns (output feeds logic in the same cycle)	about 5 × 10^8	about 8 minutes
4.5 ns (output goes straight to a second flop)	about 1 × 10^78	about 3 × 10^70 years

Those exact values are illustrative. Each vendor characterises τ and T0 per device, and Vivado will compute the figure for you with report_synchronizer_mtbf. The shape of the result does not change: feed a metastable output into logic and your design fails during the demo, give it a whole clock period to settle and it will outlive the universe.

Rule one: a single bit crosses through two flops

The two-flop synchroniser is the whole of the previous section in hardware. The first flop absorbs the metastability. Nothing but the second flop looks at its output, so the resolution time is a full clock period minus the second flop’s setup time and the routing between them. That routing is why the two flops must be placed next to each other. On AMD devices the ASYNC_REG attribute tells Vivado to keep them in the same slice and to treat the first flop as a legitimate CDC target. Other vendors have equivalents, listed in our reset and CDC patterns reference.

module sync_bit #(
    parameter integer STAGES = 2
) (
    input  wire clk,
    input  wire d_async,
    output wire q
);
    (* ASYNC_REG = "TRUE" *) reg [STAGES-1:0] shift;

    always @(posedge clk) begin
        shift <= {shift[STAGES-2:0], d_async};
    end

    assign q = shift[STAGES-1];
endmodule

The same thing in VHDL-2008:

library ieee;
use ieee.std_logic_1164.all;

entity sync_bit is
  generic (
    stages : positive := 2
  );
  port (
    clk     : in  std_logic;
    d_async : in  std_logic;
    q       : out std_logic
  );
end entity;

architecture rtl of sync_bit is
  signal shift : std_logic_vector(stages - 1 downto 0) := (others => '0');
  attribute async_reg : string;
  attribute async_reg of shift : signal is "TRUE";
begin
  process (clk)
  begin
    if rising_edge(clk) then
      shift <= shift(stages - 2 downto 0) & d_async;
    end if;
  end process;

  q <= shift(stages - 1);
end architecture;

Three things about this module matter more than its size suggests.

• It has no reset. A reset on a synchroniser adds a mux or a reset pin that can itself become a CDC problem, and the flops settle to the right value within two cycles anyway. If your coding standard insists on one, make it synchronous and in the destination domain.
• Its input must come straight from a flop in the source domain. If the input is a combinational function of several source flops, it can glitch between them. A glitch sampled by the synchroniser becomes a spurious pulse that no second flop can remove.
• It only works for signals that stay put. A level that changes slower than about three destination cycles will always get through. A one-cycle pulse from a faster clock may be missed entirely. For pulses and for data words you need a protocol.

Rule two: a word crosses with a handshake

The simplest safe way to move a multi-bit value is to freeze it in the source domain, wave a single flag across, let the destination copy the frozen value and wave a flag back. The flag is a toggle rather than a pulse so it survives any ratio of clock rates. The module below carries a word of any width in that way, with the two synchronisers written out inline so the file stands alone.

module cdc_handshake #(
    parameter integer WIDTH = 16
) (
    input  wire             clk_src,
    input  wire             rst_src,     // synchronous to clk_src
    input  wire [WIDTH-1:0] data_src,
    input  wire             valid_src,
    output wire             ready_src,

    input  wire             clk_dst,
    input  wire             rst_dst,     // synchronous to clk_dst
    output reg  [WIDTH-1:0] data_dst,
    output reg              valid_dst    // one-cycle pulse per word
);
    reg              req;        // clk_src, toggles once per word
    reg  [WIDTH-1:0] data_hold;  // clk_src, frozen while a word is in flight
    reg              ack;        // clk_dst, copies req once the word is taken

    (* ASYNC_REG = "TRUE" *) reg [1:0] ack_meta;   // ack crossing into clk_src
    (* ASYNC_REG = "TRUE" *) reg [1:0] req_meta;   // req crossing into clk_dst

    wire ack_sync = ack_meta[1];
    wire req_sync = req_meta[1];

    assign ready_src = (req == ack_sync);

    always @(posedge clk_src) begin
        if (rst_src) begin
            req       <= 1'b0;
            data_hold <= {WIDTH{1'b0}};
            ack_meta  <= 2'b00;
        end else begin
            ack_meta <= {ack_meta[0], ack};
            if (valid_src && ready_src) begin
                data_hold <= data_src;
                req       <= ~req;
            end
        end
    end

    always @(posedge clk_dst) begin
        if (rst_dst) begin
            ack       <= 1'b0;
            req_meta  <= 2'b00;
            data_dst  <= {WIDTH{1'b0}};
            valid_dst <= 1'b0;
        end else begin
            req_meta  <= {req_meta[0], req};
            valid_dst <= 1'b0;
            if (req_sync != ack) begin
                data_dst  <= data_hold;
                ack       <= req_sync;
                valid_dst <= 1'b1;
            end
        end
    end
endmodule

Walk through one transfer. The source captures a word into data_hold and flips req on the same edge. The toggle takes two or three destination cycles to come out of req_meta. When it does, the destination copies data_hold, pulses valid_dst and sets ack equal to the new req. That toggle takes two or three source cycles to return, and only then does ready_src rise again. The word was frozen for the whole exchange, so the destination copied a stable value. The cost is throughput: roughly two destination cycles plus three source cycles per word, so a handshake carries one word every five or six cycles when the clocks are similar. For control registers and status values that is fine. For sample streams it is not, which is why the third structure exists.

There is one subtlety hiding in the walkthrough. The word and the toggle leave the source on the same edge, and the word must have arrived at the destination flops before the toggle emerges from its synchroniser. Nothing in the logic guarantees that. Routing does. The constraints section shows how to bound that path so the guarantee is real.

Rule three: a stream crosses through an asynchronous FIFO

An asynchronous FIFO is a dual-port memory with a write pointer living in one clock and a read pointer living in the other. The memory itself is not a problem: block RAM has independent port clocks, and a location is never written and read at the same time if the flags are right. The problem is the flags. The write side needs to know the read pointer to decide whether it is full, and the read side needs the write pointer to decide whether it is empty. Each pointer is a multi-bit bus that has to cross a clock boundary, and the previous callout says buses cannot cross bit by bit.

Gray code resolves that. In Gray code, consecutive values differ in exactly one bit. If the pointer only ever increments by one, the bus only ever changes one bit at a time. A set of independent synchronisers can then land on either the old value or the new value but nothing else, and both are legitimate pointer positions. The crossing may be a couple of cycles stale, and the resulting flag may say full or empty a little early, but it will never say the opposite of the truth. A FIFO that reports full when it has one slot left is safe. A FIFO that reports not-full when it is full corrupts data.

Two more decisions make the flag logic clean. First, the pointers carry one bit more than the address needs. With a 16-word memory the address is 4 bits but the pointer is 5. The extra bit records how many times the pointer has wrapped, so “pointers equal” means empty and “pointers equal except for the wrap bit” means full, with no ambiguity between the two. Second, in Gray code, “equal except for the wrap bit” is not a single bit flip. It comes out as the top two bits inverted and the rest equal, which is what the full expression below tests.

module async_fifo #(
    parameter integer WIDTH     = 8,
    parameter integer ADDR_BITS = 4      // depth = 16 words
) (
    input  wire             clk_wr,
    input  wire             rst_wr,      // synchronous to clk_wr
    input  wire             wr_en,
    input  wire [WIDTH-1:0] wr_data,
    output wire             full,

    input  wire             clk_rd,
    input  wire             rst_rd,      // synchronous to clk_rd
    input  wire             rd_en,
    output reg  [WIDTH-1:0] rd_data,
    output wire             empty
);
    localparam integer P = ADDR_BITS + 1;      // pointer carries one wrap bit

    reg [WIDTH-1:0] mem [0:(1 << ADDR_BITS) - 1];

    // Write side: binary pointer for addressing, Gray copy for crossing.
    reg  [P-1:0] wr_bin, wr_gray;
    wire         do_wr        = wr_en && !full;
    wire [P-1:0] wr_bin_next  = wr_bin + {{(P-1){1'b0}}, do_wr};
    wire [P-1:0] wr_gray_next = wr_bin_next ^ (wr_bin_next >> 1);

    // Read side, the same shape.
    reg  [P-1:0] rd_bin, rd_gray;
    wire         do_rd        = rd_en && !empty;
    wire [P-1:0] rd_bin_next  = rd_bin + {{(P-1){1'b0}}, do_rd};
    wire [P-1:0] rd_gray_next = rd_bin_next ^ (rd_bin_next >> 1);

    // Each Gray pointer crosses through its own two-flop synchroniser.
    (* ASYNC_REG = "TRUE" *) reg [P-1:0] rd_gray_w1, rd_gray_w2;   // rd pointer seen in clk_wr
    (* ASYNC_REG = "TRUE" *) reg [P-1:0] wr_gray_r1, wr_gray_r2;   // wr pointer seen in clk_rd

    always @(posedge clk_wr) begin
        if (rst_wr) begin
            wr_bin     <= {P{1'b0}};
            wr_gray    <= {P{1'b0}};
            rd_gray_w1 <= {P{1'b0}};
            rd_gray_w2 <= {P{1'b0}};
        end else begin
            wr_bin     <= wr_bin_next;
            wr_gray    <= wr_gray_next;
            rd_gray_w1 <= rd_gray;
            rd_gray_w2 <= rd_gray_w1;
            if (do_wr) mem[wr_bin[ADDR_BITS-1:0]] <= wr_data;
        end
    end

    always @(posedge clk_rd) begin
        if (rst_rd) begin
            rd_bin     <= {P{1'b0}};
            rd_gray    <= {P{1'b0}};
            wr_gray_r1 <= {P{1'b0}};
            wr_gray_r2 <= {P{1'b0}};
            rd_data    <= {WIDTH{1'b0}};
        end else begin
            rd_bin     <= rd_bin_next;
            rd_gray    <= rd_gray_next;
            wr_gray_r1 <= wr_gray;
            wr_gray_r2 <= wr_gray_r1;
            if (do_rd) rd_data <= mem[rd_bin[ADDR_BITS-1:0]];
        end
    end

    // Empty: the read pointer has caught up with the (stale) write pointer.
    assign empty = (rd_gray == wr_gray_r2);

    // Full: pointers differ only by one wrap. In Gray code that is the top
    // two bits inverted and everything below them equal.
    assign full = (wr_gray == {~rd_gray_w2[P-1:P-2], rd_gray_w2[P-3:0]});
endmodule

Notice that wr_gray and rd_gray are registers, not combinational conversions of the binary pointers. The Gray value that crosses must come straight from flops, for the same glitch reason as rule one. The binary pointers still exist because they are what addresses the memory and what the increment operates on. Converting Gray back to binary on the far side is not needed for the flags, which is why the compare is done in Gray.

The flags are conservative by about two cycles of the far clock. Right after a write, the read side still sees the old write pointer and may report empty for two more read cycles. That costs a little latency and a slot or two of effective depth, and nothing else. The deeper treatment of FIFO sizing in the memories lesson applies unchanged.

Telling the tools what you did

By default, Vivado and Quartus Prime time every path between every pair of clocks, including unrelated ones. The report for a crossing between 200 MHz and 61.44 MHz will show a nonsense requirement of a few hundred picoseconds and a huge violation. Beginners fix that with a blanket set_false_path and move on. That removes the error and also removes every check on the crossing, including the two that matter: the data word in a handshake must beat its own flag, and the bits of a Gray pointer must not drift apart by more than a cycle. The constraints in the XDC and SDC cheat sheet let you keep those checks.

create_clock -name clk_src -period 5.000 [get_ports clk_src]
create_clock -name clk_dst -period 16.276 [get_ports clk_dst]

# Single-bit synchroniser inputs: no timing relationship, and ASYNC_REG keeps the flops adjacent.
set_false_path -to [get_cells -hierarchical -filter {ASYNC_REG == TRUE && PRIMITIVE_SUBGROUP == SDR}]

# Handshake data: must arrive within one destination period so it beats the toggle.
set_max_delay -datapath_only 16.276 \
    -from [get_cells u_hs/data_hold_reg[*]] \
    -to   [get_cells u_hs/data_dst_reg[*]]

# Gray pointer crossing: bound the delay and the skew between bits.
set_max_delay -datapath_only 16.276 \
    -from [get_cells u_fifo/wr_gray_reg[*]] \
    -to   [get_cells u_fifo/wr_gray_r1_reg[*]]
set_bus_skew 16.276 \
    -from [get_cells u_fifo/wr_gray_reg[*]] \
    -to   [get_cells u_fifo/wr_gray_r1_reg[*]]

The key phrase is -datapath_only. It tells the analyser to ignore the clock relationship and check only the cell and routing delay from launch flop to capture flop. That is exactly the physical quantity you care about. The set_bus_skew constraint, available in Vivado since 2017.1, adds the second check that the fastest and slowest bits of the bus differ by less than one period, which is the condition under which the Gray trick holds.

One trap. In Vivado, set_clock_groups -asynchronous takes priority over set_max_delay, so if you also declare the two clocks as an asynchronous group the bounded paths become unbounded again. Choose one strategy. For a design with handshakes and FIFOs the per-path constraints above are the safer choice. Quartus Prime treats the same exceptions similarly but with its own priority rules, so check the Timing Analyzer documentation for your version.

CDC lint: let a tool find what you missed

Structural rules are easy to state and easy to break in a 40,000-line design. Every mainstream vendor flow now has a CDC check. In Vivado, report_cdc after placement lists every crossing, classifies it (two-flop synchroniser, handshake, Gray-coded bus, FIFO primitive, unconstrained) and assigns a severity. The critical ones to chase are a bus crossing with independent synchronisers that is not Gray-coded, a synchroniser whose input comes from combinational logic, and a reset that crosses without synchronisation. Quartus Prime Pro has a CDC viewer in the Timing Analyzer that does the same job. Beyond vendor tools, Siemens Questa CDC and Synopsys VC SpyGlass CDC perform formal checks that also catch protocol errors, such as a handshake where the source changes the data before the acknowledge returns. They are expensive and, at the time of writing, there is no open-source equivalent that goes beyond structural matching.

Treat the report like a compiler warning list. Zero critical findings is the bar, and every warning should be either fixed or waived with a written reason.

Exercise

Write a testbench for async_fifo with a 200 MHz write clock and a 61.44 MHz read clock, following the structure from the testbench lesson. Push 64 incrementing bytes with the write side running flat out, read as fast as empty allows, and check the output order. Then watch the full flag and count how many of the 16 slots the writer actually gets to use before it stalls. Explain the number using the two-cycle synchroniser latency. Finally, reduce ADDR_BITS to 1 and explain from the full expression why the module no longer compiles.

Summary

• A flop sampling an unrelated clock will go metastable. The probability it is still undecided falls as e^(−t/τ), so giving it a whole cycle to settle takes the failure rate from minutes to longer than the age of the universe.
• One bit crosses through two adjacent flops, fed directly from a source flop, with no logic between them.
• A word crosses by freezing the data and waving a toggle across and back. A stream crosses through a FIFO whose pointers are Gray-coded so that only one bit changes per step.
• Never use a blanket false path on a crossing. Bound the data path with set_max_delay -datapath_only and the bus with set_bus_skew, and remember that an asynchronous clock group overrides both in Vivado.
• Run the CDC report every time and clear every critical finding. The structures are simple but the ways to misuse them are not.

Further reading

• C. E. Cummings and P. Alfke, “Simulation and Synthesis Techniques for Asynchronous FIFO Design”, SNUG 2002: http://www.sunburst-design.com/papers/CummingsSNUG2002SJ_FIFO1.pdf
• C. E. Cummings, “Clock Domain Crossing (CDC) Design and Verification Techniques Using SystemVerilog”, SNUG 2008: http://www.sunburst-design.com/papers/CummingsSNUG2008Boston_CDC.pdf
• R. Ginosar, “Fourteen Ways to Fool Your Synchronizer”, ASYNC 2003: https://webee.technion.ac.il/~ran/papers/Sync_Errors_Feb03.pdf
• AMD UG903, Vivado Design Suite User Guide: Using Constraints: https://docs.amd.com/r/en-US/ug903-vivado-using-constraints
• AMD UG906, Vivado Design Suite User Guide: Design Analysis and Closure Techniques: https://docs.amd.com/r/en-US/ug906-vivado-design-analysis