2.1 The system

Part 2 describes the production system: the exchange that we build. Section 1.2 tells which parts run on the testnet today.

Sites

The exchange runs on a ring of sites in one metro area. Each site is in a different building, with its own power and fiber. Independent companies operate the sites. The full design has six sites and at least three operators. The first builds have three sites.

Six sites on a dual fiber ring in one metro area. Each site has a card, a matcher and a read server. Clients connect to two or three sites through passive taps.
Figure 3. Six sites on a metro ring. Every site is a door.

Each site has these parts:

Part Function
Sequencer card (FPGA) Owns the network ports. Stamps each packet when the light becomes bits. Signs observations and results. Calculates the canonical order.
Matcher A full replica of the exchange: order book, balances, positions and margin. It executes the canonical order. It runs in software first, and in a second FPGA for the highest rates.
Read server Receives each final batch from the card, one way. It serves the history to readers.
Fiber links Dual fiber to the other sites, approximately 10 km around the ring. The sites measure the delay of each link one time and sign it.
Clocks A clock synchronized through the fiber, and a rubidium atomic clock that checks it (see 2.2 and 2.6).

No main entrance

Every site is a door. A passive optical tap, which is glass without electronics, sends the light that arrives to the local card. A client sends the same signed order to two or three doors. The cards identify copies by the hash of the order. Because of this, no single door can block an order, and a door that delays an order shows itself in the time data.

Three broadcasts and one silence

  1. Deliver. Each door that receives an order forwards the packet to the other sites. This is approximately 50 µs across 10 km of fiber.
  2. State what you saw. Each card broadcasts one signed observation: the hash of the order and its wire stamp. Each card collects the observations and calculates the canonical order with the same public rule. The same inputs and the same rule give the same order at each site.
  3. Silence. Each matcher executes the canonical order alone. Six independent computations must agree, because the matcher is deterministic.
  4. State what you calculated. Each card broadcasts one signed digest of its results. When a quorum of identical digests exists, the batch is FINAL. With six sites, the quorum is five. With three sites, it is two of three, or all three.

There are no questions and no votes. There are only statements and arithmetic. The quorum certificate is the settlement record, and any person can verify its signatures. The time from the wire stamp to a final trade is approximately 0.3 to 0.4 ms (design target).

Where everything lives

  • The present is in the matcher of each site: the book, each balance and each position.
  • The past is on the read servers: each order, its signed stamps, its fills and the certificates. This stream is the chain. Any person can hold a copy, verify each signature and serve it again.
  • Money enters the same queue. Deposits and withdrawals are commands like all others: stamped, ordered and executed. No administrator path into the state exists.
  • Snapshots. Each 100 ms, the sites sign one root of the full state. A bridge contract on another chain accepts a root with five of six keys. A new or restarted site loads a snapshot and replays the stream to the present.

The life of one order

 1. client signs the order and sends it to two or three doors
 2. each door card: wire stamp at the port, session and rate checks   -> ACK
 3. each door forwards the packet to the other sites                   (~50 us)
 4. each card broadcasts its signed stamp
 5. each card: official time by the public rule, short wait, canonical order
 6. each matcher executes the canonical order                          -> receipt
 7. each card signs the digest of the batch and broadcasts it
 8. quorum of identical digests                                        -> FINAL
 9. read servers and replicas verify and serve the history             -> proof

Section 2.3 explains steps 2 to 8.

results matching ""

    No results matching ""