2.3.3 The canonical order and finality

An order with an official time does not have a place in the list at that moment. The stamps of far sites are on the fiber for tens of microseconds. Each card solves this with a watermark.

The watermark

watermark = card clock - lag      (the card updates it each 50 us)

The lag is more than the time of the slowest path between the sites, plus the time to broadcast a stamp. On a 10 km ring, the lag is approximately 0.1 to 0.2 ms (design target). The watermark means this: all stamps of orders with a time before this value are here now, or they will not come. The watermark never goes back.

Release in time order

Each time the watermark moves, the card releases all orders with an official time before the watermark:

  1. The order with the earliest time goes first.
  2. Orders in one simultaneous group go in the random order of the group (see 2.3.2).

Each released order becomes a CANON record. The list of CANON records is the canonical order. Each card calculates it from the same stamps with the same rule. Because of this, the canonical order is the same at each site. If the lists are different, the digests are different, and the batch is not final.

The card gives the canonical order to the matcher of its site through PCIe. The matcher executes it. Only execution goes through PCIe. All traffic between the sites goes through the ports of the cards.

The watermark moves along the time line and releases the orders behind it in time order. A late stamp is rejected. Five identical digests of six make the batch FINAL.
Figure 7. Release in time order, then five identical digests. Loop of 5 seconds.

LATE orders

If a stamp arrives after the watermark passed the time of its order, the card cannot put the order in its correct place. The card rejects the order as LATE. The order does not execute, and its nonce stays unused. The client sends the same order again.

Batches and finality

The matcher executes the canonical order in batches. For each batch:

  1. The matcher executes the orders and calculates the state root.
  2. The card calculates the digest of the batch: the orders, their order, the state root and the stamps that it used. The card signs the digest and broadcasts it one time.
  3. Each card checks the signatures of the other cards in hardware and counts the identical digests.
  4. When a quorum of identical digests exists, the batch is FINAL. With six sites, the quorum is five.

Why is this final? A different history also must have five signatures. Two sets of five among six sites share four sites. Four operators must sign two different statements each. Each double signature is public proof, and it costs the bond of its operator.

Three confirmations

Confirmation Meaning
ACK from a card The card accepted your order and stamped it. The order is not in the canonical order at this time.
Execution receipt Your order executed in a batch at a site. The batch is not final at this time.
FINAL and certificate A quorum of cards signed the digest. The certificate is the settlement record.

The time from the wire stamp to FINAL is approximately 0.3 to 0.4 ms in one metro area (design target).

results matching ""

    No results matching ""