FPGAs and low latency trading - Williston Hayes - Optiver - FPL2020

Optiver
10 Sept 202019:56

Summary

TLDRThis presentation explores the intersection of FPGAs and low-latency trading, explaining how FPGAs can significantly reduce the time it takes to process trading decisions. The speaker from Optiver, a global market-making firm, discusses the architecture of an FPGA-based trading system and the unique design approach they use, focusing on minimizing latency and maximizing performance. The talk also covers the real-time impact of FPGA design on trading strategies and the dynamic, innovative solutions required in this high-paced financial environment.

Takeaways

  • πŸ˜€ The speaker was initially unaware of the intersection between FPGAs and low-latency trading until a job offer from Optiver sparked interest.
  • 🏦 Optiver is a global market-making firm founded in 1986, trading various financial instruments with their own capital and focusing on improving market liquidity.
  • 🌐 The transition from traditional trading pits to electronic trading in data centers has significantly impacted how trades are executed, with latency becoming a critical factor.
  • πŸš€ The importance of low latency in trading is highlighted by the 'first come, first serve' mechanism used by exchanges to handle simultaneous orders.
  • πŸ” Limitations of software-based trading systems include latency penalties from traversing the NIC, PCI Express, CPU, and memory, which FPGAs can help mitigate.
  • πŸ› οΈ FPGAs can reduce latency by handling trading logic close to the network, eliminating the need for data to pass through multiple system layers.
  • πŸ’‘ The internal structure of an FPGA-based trading system includes market data handling, trading logic, order formatting, TCP stack management, and output MAC encoding.
  • πŸ”§ Designing for FPGAs at Optiver involves avoiding pipelining to reduce latency, focusing on timing closure to save clock cycles, and custom designing IP for the hot path.
  • πŸ”„ The use of PCI Express in FPGA trading systems is for status and control, not for the critical trading logic, which remains within the FPGA for speed.
  • πŸ”‘ Optiver's approach to FPGA design is distinguished by a tight feedback loop, allowing for rapid testing and deployment of changes, with immediate visibility of their impact.
  • πŸ’‘ The trading environment at Optiver is characterized by a high pace, dynamic changes, and the need for innovative solutions to overcome challenges without introducing latency.

Q & A

  • What is the primary function of Optiver's trading systems?

    -Optiver's trading systems primarily function to improve the market by providing liquidity. They do this by buying and selling various financial instruments such as options, futures, ETFs, and stocks at prices they publish to the market.

  • How does Optiver's approach to using FPGAs in trading differ from other companies?

    -Optiver's approach to using FPGAs in trading is unique in that they focus on minimizing latency. They design their systems to be as fast as possible, avoiding pipeline registers in their critical path to reduce latency and ensure the first-come, first-serve mechanism in trading.

  • What is the significance of low latency in trading systems?

    -Low latency is crucial in trading systems because it allows for faster response times in the market. This is particularly important when multiple parties are competing to execute trades at the same time, as the first order to reach the exchange will be the one that is executed.

  • How does the shift from traditional trading pits to electronic trading impact the role of latency?

    -The shift to electronic trading has made latency a more critical factor. In electronic trading, orders are communicated through network packets, and the speed at which these packets are processed and sent to the exchange can determine the success of a trade.

  • What are the limitations of a software-based trading system in terms of latency?

    -A software-based trading system incurs latency due to the need to traverse the network interface card, PCI Express, CPU, and memory. Each of these components adds a delay, which can be significant in high-speed trading environments.

  • How does an FPGA-based trading system address the limitations of software-based systems?

    -An FPGA-based trading system addresses these limitations by placing the logic needed to make trading decisions as close as possible to the network, effectively reducing the overhead latency associated with PCI Express, CPU, and memory.

  • What is the role of the FPGA in the architecture of an FPGA-based trading system?

    -In an FPGA-based trading system, the FPGA handles tasks such as unpacking Ethernet protocols, filtering network traffic, decoding market data, and formatting orders. This allows for faster processing and decision-making compared to a software-based system.

  • Why is it important to filter out irrelevant network traffic in an FPGA-based trading system?

    -Filtering out irrelevant network traffic is important to ensure that only relevant market data reaches the trading logic. This prevents unnecessary processing of data that does not contribute to trading decisions, thereby saving processing time and reducing latency.

  • How does Optiver ensure that their trading systems are robust and error-free?

    -Optiver ensures robustness and error-free operation by implementing limit checking logic in their trading systems. This helps to prevent issues such as sending too many orders per second or sending orders with prices outside of a controlled range.

  • What are some of the challenges in designing an FPGA-based trading system?

    -Some challenges in designing an FPGA-based trading system include managing the encoding and decoding of market data and orders in market-specific formats, handling TCP/IP stack complexities, and ensuring that the system operates within tight timing constraints to minimize latency.

  • How does Optiver's approach to design and innovation in FPGA-based trading systems contribute to their competitive advantage?

    -Optiver's approach to design and innovation, such as avoiding pipeline registers, investing in the best devices, and hand-designing IP for the hotpath, contributes to their competitive advantage by allowing them to achieve lower latencies and more efficient trading systems.

Outlines

00:00

πŸ€” Introduction to FPGAs in Trading

The speaker begins by questioning the audience's familiarity with the relationship between FPGAs (Field-Programmable Gate Arrays) and low-latency trading. He shares his personal experience of being intrigued by an offer from Optiver, a trading firm, to work on an FPGA engineering role. The speaker outlines the agenda for the talk, which includes explaining trading, discussing the architecture of an FPGA-based trading system, and detailing the unique approach Optiver takes in designing FPGAs. Optiver is introduced as a global market-making firm that trades various financial instruments using its own capital and aims to improve market liquidity. The speaker also provides a brief introduction to himself, having been with Optiver for 11 years and having co-written the company's first FPGA-based trading system.

05:01

πŸš€ The Role of Latency in Electronic Trading

This paragraph delves into the significance of latency in electronic trading, especially when multiple sellers compete to trade with a single buyer. The speaker explains how exchanges handle such situations using a first-come, first-serve mechanism, emphasizing the importance of being the first to send an order back to the exchange. The limitations of a software-based trading system are also discussed, including the latency penalties incurred when data traverses through the NIC (Network Interface Card), PCI Express, CPU, and memory. The speaker then introduces the concept of using an FPGA to reduce this overhead latency by placing trading logic closer to the network, providing a more efficient path for trading decisions.

10:02

πŸ› οΈ FPGA-Based Trading System Architecture

The speaker provides an in-depth look at the internal workings of an FPGA-based trading system, starting with the reception of market data via 10 Gigabit Ethernet. The data is then processed through various stages, including unpacking Ethernet protocols, filtering irrelevant network traffic, and selecting specific instruments of interest. The payload, which contains the core order information, is extracted and passed to the trading logic block, where strategies are implemented. The speaker also discusses the complexities of encoding orders in a market-specific format and the challenges of working with TCP for order transmission, highlighting the importance of maintaining a fast and reliable critical path in the system.

15:03

πŸ”§ Optiver's Unique FPGA Design Approach

In the final paragraph, the speaker contrasts Optiver's FPGA design approach with that of other companies. He discusses the avoidance of pipelining to reduce latency, the pursuit of challenging timing closures to save clock cycles, and the preference for custom-designed IP over vendor-provided solutions to ensure low latency. The speaker also touches on the environmental aspects of working at Optiver, including the rapid feedback loop, the high-paced and dynamic nature of trading, and the need for innovative solutions to overcome design challenges without introducing latency. The presentation concludes with an invitation for further questions at the Optiver virtual booth.

Mindmap

Keywords

πŸ’‘FPGAs

FPGAs, or Field-Programmable Gate Arrays, are integrated circuits that can be programmed to perform various tasks. In the context of the video, FPGAs are used for their high-speed processing capabilities in low-latency trading systems, allowing for faster decision-making and order execution in financial markets. The script mentions that the speaker was initially puzzled by the use of FPGAs in a trading firm, but the video aims to shed light on their significance in this domain.

πŸ’‘Low Latency Trading

Low Latency Trading refers to the practice of minimizing the time delay, or 'latency', in trading systems to gain a competitive edge in the market. The lower the latency, the quicker the trading decisions can be made and executed. The video discusses how FPGAs can be utilized to achieve this by reducing the time it takes for market data to be processed and orders to be placed.

πŸ’‘Optiver

Optiver is a global market-making firm founded in 1986, based in Amsterdam. The company is involved in trading options, futures, ETFs, and stocks using their own capital. In the video, the speaker explains Optiver's role in the market, which includes providing liquidity by buying and selling financial instruments at published prices, and how they leverage FPGAs for their trading systems.

πŸ’‘Market Liquidity

Market Liquidity refers to the ease with which assets can be bought or sold in the market without affecting the asset's price. In the video, the speaker mentions that Optiver's goal is to improve the market by showing liquidity, meaning they are ready to buy or sell assets at the prices they publish, thus facilitating smooth trading.

πŸ’‘Trading System Architecture

The architecture of a trading system outlines the components and their interactions within the system. The script discusses the architecture of an FPGA-based trading system, emphasizing the placement of logic close to the network to reduce latency. This includes handling market data, making trading decisions, and formatting orders, all within the FPGA to achieve high-speed trading.

πŸ’‘Latency

Latency in the context of trading refers to the time it takes for a trade order to be executed after it has been placed. The video highlights the importance of reducing latency for competitive advantage. For instance, in situations where multiple sellers respond to a single buyer, the first order to reach the exchange has a higher chance of being executed, making latency a critical factor.

πŸ’‘Network Interface Card (NIC)

A Network Interface Card (NIC) is a hardware component that connects a computer to a network. In the video, the NIC is part of the traditional trading system's latency chain, where market data must pass through the NIC before reaching the CPU for processing. The use of FPGAs can bypass the NIC, reducing latency by processing data directly on the FPGA.

πŸ’‘10 Gigabit Ethernet

10 Gigabit Ethernet is a networking technology that provides high-speed connectivity, capable of transferring data at 10 billion bits per second. The script mentions that data centers provide 10 Gigabit network connectivity for trading systems to communicate with exchanges, emphasizing the importance of high-speed networking for low-latency trading.

πŸ’‘TCP/IP

TCP/IP, or Transmission Control Protocol/Internet Protocol, is a set of communication protocols used for transmitting data over networks. In the video, the speaker discusses the challenges of using TCP for trading systems due to its stateful nature, which includes managing sequence numbers and ensuring reliable data transfer, all of which can introduce latency.

πŸ’‘Custom Designed IP

Custom Designed IP refers to the practice of creating specialized intellectual property cores for FPGAs, tailored to specific needs rather than using pre-built, generic IP provided by vendors. The video explains that Optiver avoids using standard IP on their hotpath to ensure the lowest possible latency, opting for hand-designed solutions that are optimized for speed.

πŸ’‘High-Paced Environment

A high-paced environment is characterized by rapid changes and the need for quick responses. The video describes the trading environment as dynamic and fast-paced, with constant changes in market conditions and opportunities. This context requires trading systems to be adaptable and responsive, which is where the low-latency capabilities of FPGAs become crucial.

Highlights

Introduction to the intersection of FPGAs and low-latency trading, with a personal anecdote about joining Optiver.

Explanation of what trading is and how FPGAs fit into this domain.

Overview of Optiver as a global market-making firm and its role in improving the market.

Introduction of the speaker's background and experience with FPGA-based trading systems.

Historical context of trading from trading pits to electronic trading in data centers.

Importance of latency in trading and how it affects the trading process.

Limitations of software-based trading systems and the latency penalties involved.

Advantages of using FPGAs to reduce latency in trading systems.

Architecture of an FPGA-based trading system and its internal components.

Details on how market data is handled and processed in an FPGA-based system.

The role of filters in selecting relevant market data and ignoring unnecessary information.

Unpacking and understanding the structure of market data for trading decisions.

Implementation of trading logic within the FPGA and strategies for order execution.

Challenges in encoding orders in market-specific formats for exchange acceptance.

Importance of limit checking in trading systems to avoid costly mistakes.

TCP stack considerations and the challenges of state management in trading systems.

Optiver's unique approach to FPGA design, emphasizing speed and custom solutions.

Environmental factors that differentiate Optiver's approach to FPGA design and trading.

Innovative solutions and out-of-the-box thinking required for latency reduction in trading systems.

Invitation to the Optiver virtual booth for further questions and discussion.

Transcripts

play00:02

[Music]

play00:20

right well good evening

play00:21

um let me start with a question uh who

play00:24

have you out there have thought about

play00:26

the

play00:26

intersection between fpgas and low

play00:28

latency trading

play00:31

um i know i hadn't uh thought about it

play00:34

much

play00:35

i got a phone call about 11 years ago

play00:37

from optiver

play00:39

asking me if i was interested in an fpga

play00:42

engineering role

play00:43

and i remember thinking to myself what

play00:45

on earth is this company doing

play00:47

with with fpga they're a trading firm

play00:50

well

play00:50

in the next 15 minutes i hope to give

play00:52

you a little bit of insight into

play00:55

why and how we use fpgas in this domain

play01:00

a little agenda for this talk first i'll

play01:03

explain a little bit about what

play01:05

trading really is and that can give me a

play01:08

basis for explaining how fpgas actually

play01:10

fit in

play01:11

to this domain um i'll then talk a

play01:14

little bit about

play01:15

the architecture of an fpga-based

play01:17

trading system

play01:18

and finally i'll discuss a little bit

play01:20

about how the

play01:22

how we approach design for fpgas at

play01:24

optiver and how it's a bit different

play01:26

than

play01:26

how other companies which makes use of

play01:28

fpgas approach them

play01:32

a little introduction maybe first on

play01:35

optifer and myself

play01:36

optiver itself was founded in 1986 in

play01:39

amsterdam

play01:40

it's a global market-making firm it

play01:42

means we're a training firm

play01:43

we trade all kinds of things but

play01:46

predominantly options

play01:47

futures etfs stocks and that kind of

play01:50

thing

play01:50

and we do all of this with our own

play01:52

capital we don't have any clients

play01:54

this is all done internally

play01:57

and what's our goal here in the

play01:58

marketplace it's to improve

play02:00

the market and we do that by showing

play02:01

liquidity and what does that mean

play02:04

that means if if you want to buy

play02:05

something we'll sell it to you

play02:07

if you want to sell something we'll buy

play02:09

it from you but always at prices that we

play02:11

are

play02:11

publishing uh yeah to the to the market

play02:14

at large

play02:15

um we are a global company

play02:19

so we've got offices around the world

play02:21

and from those offices we can cover

play02:23

exchanges around the world um amsterdam

play02:27

of course which is well where i'm based

play02:29

but we also have offices in london

play02:31

chicago sydney and shanghai about me

play02:34

personally i've been with optiver for

play02:36

about 11 years now which maybe tells you

play02:38

something about how my phone call with

play02:40

them went 11 years ago

play02:42

um and in that time i've i co-wrote

play02:45

the first fpga-based trading system that

play02:48

we had

play02:49

and went on to design and implement and

play02:51

uh

play02:52

break uh numerous other systems in the

play02:55

last

play02:56

11 years to bring to one add new

play02:58

features and two to to bring

play03:00

these systems to new exchanges

play03:05

so what's trading really um yeah we see

play03:09

we have a photo here this is a trading

play03:11

pit this is the way that trading was

play03:13

done

play03:14

uh well and still is done to some degree

play03:16

for uh 400 years

play03:17

it's been done like this people yelling

play03:19

at each other and

play03:20

a cluster um and what's actually

play03:23

happening here is that

play03:24

one of these people will be saying

play03:26

something like i want to buy

play03:28

five shares of of apple for 10 euros

play03:31

and anyone around this person or at

play03:34

least with an earshot

play03:35

uh can can hear that and think oh is

play03:37

that a good deal or not

play03:39

and if it is then they can yell back i

play03:41

want to sell five shares to you

play03:43

and that's how a trade is done now

play03:47

in the last 25 years um this type of

play03:50

trading has largely been replaced with

play03:53

electronic trading

play03:54

and that what does that look like it

play03:56

looks like looks like this

play03:58

uh these days so this is a data center

play04:00

and

play04:01

what happens now is um an exchange

play04:05

a market can rent physical space um

play04:09

in in this data center where they can

play04:12

install their own servers their own

play04:13

network equipment and this kind of thing

play04:16

but so can market participants can also

play04:18

um

play04:19

rent space uh in in this in this data

play04:22

center and install

play04:23

their own equipment but how do we

play04:27

actually

play04:28

communicate here and that's that's one

play04:30

of the keys and

play04:31

and the old style it was by yelling in

play04:33

this new style electronic trading it's

play04:34

by

play04:35

sending ethernet packets so these data

play04:38

centers will provide

play04:39

10 gigabit network connectivity

play04:42

generally

play04:43

which you can use to communicate with

play04:45

the exchange

play04:47

so now when somebody wants to buy five

play04:49

shares of apple for

play04:50

10 euros they have to encode that

play04:53

information into an

play04:54

into a network packet and send it to the

play04:56

exchange's

play04:57

central server at that point the

play05:00

exchange can

play05:01

rebroadcast that information to all

play05:04

market participants now

play05:08

where does the latency angle come in

play05:10

where did yeah where did the lazy come

play05:11

in here

play05:12

so imagine that more than one person

play05:15

wants to sell to the single buyer

play05:17

imagine a thousand people

play05:19

want to sell to the single buyer yeah

play05:22

how do we solve that problem because

play05:24

from the exchanges point of view there's

play05:25

going to be a thousand

play05:26

orders all coming in at the same time

play05:29

here at the same time

play05:30

um and one of the ways that an exchange

play05:32

can solve that problem

play05:34

is by implementing a kind of first come

play05:36

first serve

play05:37

style mechanism and then you can start

play05:40

to see

play05:40

why the latency angle starts to matter

play05:43

which is yeah the first one to get their

play05:44

order

play05:45

back to the exchange will be the one who

play05:47

can affect the trade

play05:50

um so knowing that latency is

play05:53

important um what are the limitations

play05:57

of a software-based trading system um

play06:01

here we can see uh on the on the left

play06:04

we have our exchange who's going to be

play06:06

broadcasting information

play06:08

and then it's going to hit our server

play06:10

and on the server we're going to have a

play06:11

software application which

play06:12

you know needs to understand what's

play06:14

happening in order to take a trading

play06:16

decision

play06:18

but in order for the software

play06:19

application to see this we're going to

play06:21

have to first traverse

play06:22

the nic so this is a network interface

play06:24

card we're then going to traverse

play06:26

pci express and then we get into our you

play06:30

know our cpu and memory

play06:31

um where we're going to have you know

play06:33

our application running

play06:35

now in this chain um we're gonna be

play06:37

incurring a latency penalty

play06:39

crossing this guy crossing pci express

play06:42

but of course even software applications

play06:44

are gonna have their own

play06:46

variance and latency with respect to

play06:48

getting access to an

play06:49

actual cpu core imagine the colonel

play06:52

wants to start doing something on the

play06:53

processor you're running on and you get

play06:55

booted off

play06:56

that's going to incur a latency hit as

play06:58

well on the memory side

play07:00

imagine that the memory you're trying to

play07:01

access gets dropped out of cash

play07:04

and you're gonna have to go you know

play07:05

further away to get the information you

play07:07

actually need

play07:08

it's also gonna introduce a lot more

play07:10

variance in your in your latency

play07:12

because ultimately what we're trying to

play07:13

do here is get market data

play07:15

into your application take a trading

play07:17

decision and get back out again

play07:19

and there's a lot to accomplish there so

play07:22

how can fpgas

play07:23

help us in this next slide

play07:27

it doesn't look that much different but

play07:28

there is a key difference which is

play07:30

the nick has been replaced with an fpga

play07:34

so what if we can cut away all of the

play07:37

extra overhead latency of of pci express

play07:42

and our software application and cpu and

play07:45

memory

play07:46

problems and we can do that by trying to

play07:48

place

play07:49

all the logic we need to take a trading

play07:51

decision

play07:52

as close as possible to the network and

play07:54

that would be here in the fpga

play07:57

so if we if we dive deeper into the fpga

play08:00

we can actually start to see

play08:02

what what the skeleton of a of an fpga

play08:04

based trading system looks like

play08:06

internally um so what do we have here on

play08:10

the

play08:10

on the top left so we'll have our market

play08:13

data so this will be

play08:14

10 gig ethernet um some you know

play08:18

network traffic being broadcast

play08:19

indicating the state of the market

play08:21

somebody wants to buy somebody wants to

play08:23

sell that kind of thing

play08:25

what do we have to do in order to take a

play08:27

trading decision on this information

play08:29

well one of the first things that we're

play08:30

going to need to do is kind of unpack

play08:33

the lowest level ethernet protocols here

play08:36

there's some physical layer things we

play08:37

have to take into account

play08:38

and we use a mac to help us with this

play08:41

but the output of this

play08:42

uh here will be effectively a a packet

play08:45

stream

play08:46

a network stream

play08:46

[Music]

play08:49

because this is ethernet there's going

play08:52

to be a number of

play08:53

networking headers on top of the actual

play08:56

payload that we

play08:57

care about so there's going to be

play08:59

ethernet header an ip

play09:01

v4 header a udp header and these kinds

play09:04

of things

play09:05

and these what we can implement here in

play09:07

this first block

play09:08

in the market data handling block is a

play09:10

filter

play09:11

um on these networks there's going to be

play09:14

other networking equipment

play09:15

blabbing stuff we don't really care

play09:17

about maybe like routing protocol

play09:20

updates and that kind of thing and we

play09:22

don't want any of that to hit our

play09:24

trading logic that's stuff we want to

play09:26

ignore so we can filter out

play09:27

based off of you know higher level

play09:30

header information

play09:31

secondly we can use these filters to

play09:33

only select the instruments that we

play09:35

really

play09:36

care about generally exchanges will

play09:38

partition the

play09:39

the the stocks uh that that are being

play09:41

traded on these exchange by multicast

play09:43

group

play09:44

so if you know you only care about a

play09:45

subset of instruments you can also apply

play09:47

a filter

play09:48

at this stage to only you know get

play09:50

access to the instruments you really

play09:51

care about

play09:54

so once you've gotten rid of your your

play09:55

networking headers uh you're left with

play09:57

just your payload

play09:58

and this is where you know the meat

play10:02

of the um of the order message really is

play10:04

that

play10:05

somebody wants to buy five shares at 10

play10:07

euros

play10:08

now every exchange tends to encode all

play10:11

this information

play10:12

and their own native format so you'll

play10:14

have to consult you know the relevant

play10:16

exchange

play10:17

but generally what the way this will

play10:19

work is that uh the first couple bytes

play10:21

of the payload will have some kind of

play10:24

information which tells you

play10:26

what the structure of the rest of the

play10:27

packet looks like

play10:29

so that could be that this is a trade or

play10:32

that this is the

play10:33

end of the trading day and it's time to

play10:34

turn off or it could be

play10:36

that somebody wants to buy five shares

play10:38

of apple for 10 euros

play10:40

so once you once you know what kind of

play10:42

message this is

play10:43

then you're gonna start to unpack the

play10:45

stuff behind it

play10:47

and that could be like the next four

play10:49

bytes could indicate

play10:50

uh the instrument id so is it is this

play10:53

apple is this google is it tesla is it

play10:56

whatever um and then you behind next

play10:59

four bytes behind that could be

play11:01

this is the price that somebody's

play11:02

willing to pay represented in

play11:04

as a as a single precision float for

play11:07

example

play11:08

and then the following four bytes could

play11:10

be the volume how many shares do someone

play11:12

want to buy

play11:14

so once you've unpacked all that

play11:15

information in the market data handling

play11:17

block

play11:18

you can then pass it downstream to your

play11:19

to your trading logic

play11:21

i won't go into much detail about what

play11:23

we do at optiver here

play11:25

um but you could implement you know your

play11:27

very simple strategy which could just be

play11:29

if anyone is ever willing to buy apple

play11:32

for more than

play11:33

eight euros then sell sell your share of

play11:36

apple to that person

play11:38

so yeah up to you to implement what you

play11:40

actually want in this blog

play11:42

um leaving this block you'll have your

play11:45

order you need to send something back to

play11:46

the exchange to cause a trade to happen

play11:49

um and what one of the problems that

play11:53

you'll face

play11:54

is that just like the market data is

play11:56

encoded in a market specific

play11:58

format so are the orders so we need to

play12:01

make sure that all the information is

play12:03

placed in the right

play12:04

spot and encoded in the right way and

play12:07

that can also be quite a challenge

play12:08

because sometimes exchanges

play12:10

like really um unfortunate encodings

play12:13

like ascii sometimes we need to convert

play12:15

like binary numbers to ascii

play12:17

representation

play12:18

to make them accepted by the exchange

play12:19

which is not that fun

play12:22

um so that's that's what kind of logic

play12:24

you would actually need to

play12:25

uh build inside your order formatting

play12:27

block

play12:28

say additionally another critical thing

play12:30

to maybe put in this block would be some

play12:32

kind of limit checking

play12:34

logic so because we designed these

play12:37

systems to be as fast as possible

play12:39

um any any mistakes or any bugs or any

play12:42

you know um behavior you don't expect

play12:45

can be very costly so it would be a good

play12:48

idea to put a limit checking

play12:51

uh code in here to make sure you don't

play12:52

send for example

play12:54

too many orders per second or

play12:57

that's the prices that you're sending to

play12:59

the exchange fallen

play13:00

within some bound uh that you that you

play13:04

control

play13:06

uh once you've gone through all that

play13:07

then we're gonna get to our tcp

play13:09

uh stack and yeah well one of the

play13:12

one of the um things that i think almost

play13:14

every exchange expects

play13:16

is that the orders that you place going

play13:17

back to the exchange are encoded in

play13:20

tcp um which is a bit of a headache

play13:23

for us doing tcp means a lot of state

play13:27

tracking there's sequence numbers to

play13:29

worry about that kind of thing

play13:32

and as well there are features of tcp in

play13:34

fact the hallmark of tcp is that it's

play13:36

reliable

play13:37

um which means you know your your

play13:38

message your segment always gets the

play13:40

other side

play13:41

but when you really zoom in on what that

play13:42

means it means that if you send

play13:45

a segment and it's never acknowledged by

play13:47

the remote

play13:48

that you need to send it again and that

play13:50

means a lot of

play13:52

state management buffering and that kind

play13:54

of thing um which is a bit of a headache

play13:58

finally uh we'll get to the to the

play14:00

output mac here where we can again

play14:02

re-encode this information

play14:03

at the lowest level to be re-transmitted

play14:06

back to the exchange

play14:07

to try and make our our trade happen

play14:10

now we call this this loop we call this

play14:13

our our critical path

play14:15

or a hub path right i mean this is we

play14:17

try to make this path as fast as

play14:19

possible

play14:20

um as of a couple of years ago uh we

play14:22

were

play14:23

well below 200 nanoseconds um

play14:26

that would be from from the time that

play14:28

the you know the first bit

play14:30

hits the pin of the network connector uh

play14:32

on the fpga card

play14:34

until the first bit hits the pin on the

play14:36

on the egress side that went back to the

play14:38

exchange

play14:40

um one one element i didn't touch on on

play14:43

this is pci express

play14:45

we still do make use of this um and it

play14:47

we

play14:48

make use of this for um status and

play14:51

control

play14:52

so we will have a software application

play14:53

still running on a traditional

play14:55

server above us um but that's used

play14:58

for a couple of reasons one is yeah

play15:00

making sure that whatever logic

play15:02

you're doing here and the kind of orders

play15:04

that you're sending back to the exchange

play15:05

look sane and sensible and make sense

play15:09

but as well control so when when does it

play15:11

make sense to turn this thing off

play15:13

um do we want to do more configuration

play15:16

on

play15:16

what kind of multicast groups do we let

play15:18

through

play15:20

that kind of thing

play15:25

uh so finally last slide uh how do we

play15:28

approach

play15:28

design differently um at optiver

play15:30

compared to other companies which make

play15:32

use of fpgas

play15:34

first one is the technical side um

play15:36

there's a couple things here one is

play15:37

pipelining

play15:38

um yeah i think for most you know

play15:41

digital designers

play15:42

you know they're taught to pipeline

play15:43

their design right that's just how you

play15:45

how you do it you put a register on this

play15:47

on the input and register on the output

play15:48

and that's just what you do

play15:50

for us that's that's almost never what

play15:52

we want to do um

play15:54

because every register in our hot path

play15:56

is

play15:57

latency and we just can't afford to do

play16:00

that

play16:01

um an interesting flip side of this

play16:05

point is that for time and closure um

play16:08

you know i think for most engineers when

play16:10

when time enclosure is easy that's great

play16:13

that means you don't need to worry about

play16:14

it and you can go on to think about the

play16:16

next thing

play16:17

um for us that's not really the case if

play16:19

time enclosure is too easy

play16:22

then that probably means there's a clock

play16:24

cycle somewhere

play16:25

we can save so we'll deliberately try

play16:27

and make timing harder

play16:29

uh well as a result that we can save a

play16:31

clock somewhere uh yeah we can

play16:33

we can make it faster so the net result

play16:36

is that

play16:37

kind of perpetually in a state where

play16:39

timing is irritatingly hard to meet

play16:42

but not impossible um which i don't

play16:45

think is very normal elsewhere in the

play16:47

world

play16:49

um second point here is as best devices

play16:52

yeah

play16:52

you know we're we're really on the on

play16:53

the cutting edge here we will

play16:55

invest in buying whatever we need to do

play16:57

to make sure that we are on the on the

play17:00

on the best possible footing we can be

play17:03

um

play17:04

the last point here is around custom

play17:06

designed ip a lot of

play17:07

fpga vendors of course provide a lot of

play17:10

um

play17:11

you know pre-built pre-packaged ip for

play17:13

you to use like fifos and that kind of

play17:15

thing

play17:16

we don't generally make use of this

play17:18

stuff at least not on our hotpath

play17:20

uh most most of this um ip it's robust

play17:23

and tested

play17:24

but it's not designed for low latency

play17:27

purposes so we tend to hand design

play17:31

everything in our hotpath

play17:36

uh the second class of things on how we

play17:38

approach things differently

play17:39

is more environmental one is

play17:43

we have a real tight feedback loop on

play17:45

our work um

play17:46

i think whereas in some companies you

play17:48

might not see the result of your work

play17:50

for

play17:50

for weeks or months or maybe even years

play17:54

in our environment you could be making a

play17:56

change to the code base

play17:57

you know test benching it going through

play18:00

a regression suite

play18:01

but seeing you know a release out the

play18:02

next day and having it deployed and

play18:04

running

play18:05

and you can see the results of your work

play18:07

literally the next day

play18:09

um not only seeing it running but also

play18:11

seeing its impact

play18:12

on let's say the success of a trading

play18:14

strategy

play18:15

we are so dependent on on latency for

play18:18

some of these things that you can really

play18:19

see the impact

play18:20

um directly the second point

play18:24

is around being a high-paced environment

play18:26

trading is really dynamic um

play18:28

there's things are changing all the time

play18:29

around the world exchanges change the

play18:31

way that they

play18:32

operate um opportunities we perceive uh

play18:36

um come quickly so there's

play18:39

very rarely a dull day i think in

play18:42

october there's always something new

play18:43

uh coming i think the last point

play18:47

um i wanted to highlight was on yeah

play18:49

innovative solutions

play18:50

um yeah tying into the fact that we

play18:53

don't pipeline for example

play18:55

means that in order to solve some of the

play18:57

problems that we have

play18:58

without introducing latency you have to

play19:00

get real out-of-the-box thinking

play19:02

and that could be thinking about novel

play19:04

ways to

play19:05

achieve a comparison or to accomplish

play19:08

some

play19:09

logic um which yeah it's nice to always

play19:12

have that kind of fresh challenge

play19:13

uh present on on a daily basis

play19:16

um so yeah i think that was that was my

play19:20

presentation

play19:20

um thanks for joining me for this

play19:24

i don't think i have much time to take

play19:25

questions now

play19:27

however i will be present in the in the

play19:30

optiver

play19:31

uh virtual booth i believe at 7 15

play19:35

european time so happy to take any more

play19:37

questions or any

play19:38

any other things you want to know happy

play19:40

to talk in that venue

play19:48

[Music]

play19:50

thanks

Rate This
β˜…
β˜…
β˜…
β˜…
β˜…

5.0 / 5 (0 votes)

Related Tags
FPGAsLow LatencyTrading SystemsOptiverMarket MakingElectronic TradingNetwork ProtocolsOrder ExecutionLatency ReductionFinancial MarketsTechnical Innovation