FPGAs and low latency trading - Williston Hayes - Optiver - FPL2020

Summary

This presentation explores the intersection of FPGAs and low-latency trading, explaining how FPGAs can significantly reduce the time it takes to process trading decisions. The speaker from Optiver, a global market-making firm, discusses the architecture of an FPGA-based trading system and the unique design approach they use, focusing on minimizing latency and maximizing performance. The talk also covers the real-time impact of FPGA design on trading strategies and the dynamic, innovative solutions required in this high-paced financial environment.

Takeaways

• 😀 The speaker was initially unaware of the intersection between FPGAs and low-latency trading until a job offer from Optiver sparked interest.
• 🏦 Optiver is a global market-making firm founded in 1986, trading various financial instruments with their own capital and focusing on improving market liquidity.
• 🌐 The transition from traditional trading pits to electronic trading in data centers has significantly impacted how trades are executed, with latency becoming a critical factor.
• 🚀 The importance of low latency in trading is highlighted by the 'first come, first serve' mechanism used by exchanges to handle simultaneous orders.
• 🔍 Limitations of software-based trading systems include latency penalties from traversing the NIC, PCI Express, CPU, and memory, which FPGAs can help mitigate.
• 🛠️ FPGAs can reduce latency by handling trading logic close to the network, eliminating the need for data to pass through multiple system layers.
• 💡 The internal structure of an FPGA-based trading system includes market data handling, trading logic, order formatting, TCP stack management, and output MAC encoding.
• 🔧 Designing for FPGAs at Optiver involves avoiding pipelining to reduce latency, focusing on timing closure to save clock cycles, and custom designing IP for the hot path.
• 🔄 The use of PCI Express in FPGA trading systems is for status and control, not for the critical trading logic, which remains within the FPGA for speed.
• 🔑 Optiver's approach to FPGA design is distinguished by a tight feedback loop, allowing for rapid testing and deployment of changes, with immediate visibility of their impact.
• 💡 The trading environment at Optiver is characterized by a high pace, dynamic changes, and the need for innovative solutions to overcome challenges without introducing latency.

Q & A

What is the primary function of Optiver's trading systems?

Optiver's trading systems primarily function to improve the market by providing liquidity. They do this by buying and selling various financial instruments such as options, futures, ETFs, and stocks at prices they publish to the market.

How does Optiver's approach to using FPGAs in trading differ from other companies?

Optiver's approach to using FPGAs in trading is unique in that they focus on minimizing latency. They design their systems to be as fast as possible, avoiding pipeline registers in their critical path to reduce latency and ensure the first-come, first-serve mechanism in trading.

What is the significance of low latency in trading systems?

Low latency is crucial in trading systems because it allows for faster response times in the market. This is particularly important when multiple parties are competing to execute trades at the same time, as the first order to reach the exchange will be the one that is executed.

How does the shift from traditional trading pits to electronic trading impact the role of latency?

The shift to electronic trading has made latency a more critical factor. In electronic trading, orders are communicated through network packets, and the speed at which these packets are processed and sent to the exchange can determine the success of a trade.

What are the limitations of a software-based trading system in terms of latency?

A software-based trading system incurs latency due to the need to traverse the network interface card, PCI Express, CPU, and memory. Each of these components adds a delay, which can be significant in high-speed trading environments.

How does an FPGA-based trading system address the limitations of software-based systems?

An FPGA-based trading system addresses these limitations by placing the logic needed to make trading decisions as close as possible to the network, effectively reducing the overhead latency associated with PCI Express, CPU, and memory.

What is the role of the FPGA in the architecture of an FPGA-based trading system?

In an FPGA-based trading system, the FPGA handles tasks such as unpacking Ethernet protocols, filtering network traffic, decoding market data, and formatting orders. This allows for faster processing and decision-making compared to a software-based system.

Why is it important to filter out irrelevant network traffic in an FPGA-based trading system?

Filtering out irrelevant network traffic is important to ensure that only relevant market data reaches the trading logic. This prevents unnecessary processing of data that does not contribute to trading decisions, thereby saving processing time and reducing latency.

How does Optiver ensure that their trading systems are robust and error-free?

Optiver ensures robustness and error-free operation by implementing limit checking logic in their trading systems. This helps to prevent issues such as sending too many orders per second or sending orders with prices outside of a controlled range.

What are some of the challenges in designing an FPGA-based trading system?

Some challenges in designing an FPGA-based trading system include managing the encoding and decoding of market data and orders in market-specific formats, handling TCP/IP stack complexities, and ensuring that the system operates within tight timing constraints to minimize latency.

How does Optiver's approach to design and innovation in FPGA-based trading systems contribute to their competitive advantage?

Optiver's approach to design and innovation, such as avoiding pipeline registers, investing in the best devices, and hand-designing IP for the hotpath, contributes to their competitive advantage by allowing them to achieve lower latencies and more efficient trading systems.

Outlines

00:00

🤔 Introduction to FPGAs in Trading

The speaker begins by questioning the audience's familiarity with the relationship between FPGAs (Field-Programmable Gate Arrays) and low-latency trading. He shares his personal experience of being intrigued by an offer from Optiver, a trading firm, to work on an FPGA engineering role. The speaker outlines the agenda for the talk, which includes explaining trading, discussing the architecture of an FPGA-based trading system, and detailing the unique approach Optiver takes in designing FPGAs. Optiver is introduced as a global market-making firm that trades various financial instruments using its own capital and aims to improve market liquidity. The speaker also provides a brief introduction to himself, having been with Optiver for 11 years and having co-written the company's first FPGA-based trading system.

05:01

🚀 The Role of Latency in Electronic Trading

This paragraph delves into the significance of latency in electronic trading, especially when multiple sellers compete to trade with a single buyer. The speaker explains how exchanges handle such situations using a first-come, first-serve mechanism, emphasizing the importance of being the first to send an order back to the exchange. The limitations of a software-based trading system are also discussed, including the latency penalties incurred when data traverses through the NIC (Network Interface Card), PCI Express, CPU, and memory. The speaker then introduces the concept of using an FPGA to reduce this overhead latency by placing trading logic closer to the network, providing a more efficient path for trading decisions.

10:02

🛠️ FPGA-Based Trading System Architecture

The speaker provides an in-depth look at the internal workings of an FPGA-based trading system, starting with the reception of market data via 10 Gigabit Ethernet. The data is then processed through various stages, including unpacking Ethernet protocols, filtering irrelevant network traffic, and selecting specific instruments of interest. The payload, which contains the core order information, is extracted and passed to the trading logic block, where strategies are implemented. The speaker also discusses the complexities of encoding orders in a market-specific format and the challenges of working with TCP for order transmission, highlighting the importance of maintaining a fast and reliable critical path in the system.

15:03

🔧 Optiver's Unique FPGA Design Approach

In the final paragraph, the speaker contrasts Optiver's FPGA design approach with that of other companies. He discusses the avoidance of pipelining to reduce latency, the pursuit of challenging timing closures to save clock cycles, and the preference for custom-designed IP over vendor-provided solutions to ensure low latency. The speaker also touches on the environmental aspects of working at Optiver, including the rapid feedback loop, the high-paced and dynamic nature of trading, and the need for innovative solutions to overcome design challenges without introducing latency. The presentation concludes with an invitation for further questions at the Optiver virtual booth.

Keywords

💡FPGAs

FPGAs, or Field-Programmable Gate Arrays, are integrated circuits that can be programmed to perform various tasks. In the context of the video, FPGAs are used for their high-speed processing capabilities in low-latency trading systems, allowing for faster decision-making and order execution in financial markets. The script mentions that the speaker was initially puzzled by the use of FPGAs in a trading firm, but the video aims to shed light on their significance in this domain.

💡Low Latency Trading

Low Latency Trading refers to the practice of minimizing the time delay, or 'latency', in trading systems to gain a competitive edge in the market. The lower the latency, the quicker the trading decisions can be made and executed. The video discusses how FPGAs can be utilized to achieve this by reducing the time it takes for market data to be processed and orders to be placed.

💡Optiver

Optiver is a global market-making firm founded in 1986, based in Amsterdam. The company is involved in trading options, futures, ETFs, and stocks using their own capital. In the video, the speaker explains Optiver's role in the market, which includes providing liquidity by buying and selling financial instruments at published prices, and how they leverage FPGAs for their trading systems.

💡Market Liquidity

Market Liquidity refers to the ease with which assets can be bought or sold in the market without affecting the asset's price. In the video, the speaker mentions that Optiver's goal is to improve the market by showing liquidity, meaning they are ready to buy or sell assets at the prices they publish, thus facilitating smooth trading.

💡Trading System Architecture

The architecture of a trading system outlines the components and their interactions within the system. The script discusses the architecture of an FPGA-based trading system, emphasizing the placement of logic close to the network to reduce latency. This includes handling market data, making trading decisions, and formatting orders, all within the FPGA to achieve high-speed trading.

💡Latency

Latency in the context of trading refers to the time it takes for a trade order to be executed after it has been placed. The video highlights the importance of reducing latency for competitive advantage. For instance, in situations where multiple sellers respond to a single buyer, the first order to reach the exchange has a higher chance of being executed, making latency a critical factor.

💡Network Interface Card (NIC)

A Network Interface Card (NIC) is a hardware component that connects a computer to a network. In the video, the NIC is part of the traditional trading system's latency chain, where market data must pass through the NIC before reaching the CPU for processing. The use of FPGAs can bypass the NIC, reducing latency by processing data directly on the FPGA.

💡10 Gigabit Ethernet

10 Gigabit Ethernet is a networking technology that provides high-speed connectivity, capable of transferring data at 10 billion bits per second. The script mentions that data centers provide 10 Gigabit network connectivity for trading systems to communicate with exchanges, emphasizing the importance of high-speed networking for low-latency trading.

💡TCP/IP

TCP/IP, or Transmission Control Protocol/Internet Protocol, is a set of communication protocols used for transmitting data over networks. In the video, the speaker discusses the challenges of using TCP for trading systems due to its stateful nature, which includes managing sequence numbers and ensuring reliable data transfer, all of which can introduce latency.

💡Custom Designed IP

Custom Designed IP refers to the practice of creating specialized intellectual property cores for FPGAs, tailored to specific needs rather than using pre-built, generic IP provided by vendors. The video explains that Optiver avoids using standard IP on their hotpath to ensure the lowest possible latency, opting for hand-designed solutions that are optimized for speed.

💡High-Paced Environment

A high-paced environment is characterized by rapid changes and the need for quick responses. The video describes the trading environment as dynamic and fast-paced, with constant changes in market conditions and opportunities. This context requires trading systems to be adaptable and responsive, which is where the low-latency capabilities of FPGAs become crucial.

Highlights

Introduction to the intersection of FPGAs and low-latency trading, with a personal anecdote about joining Optiver.

Explanation of what trading is and how FPGAs fit into this domain.

Overview of Optiver as a global market-making firm and its role in improving the market.

Introduction of the speaker's background and experience with FPGA-based trading systems.

Historical context of trading from trading pits to electronic trading in data centers.

Importance of latency in trading and how it affects the trading process.

Limitations of software-based trading systems and the latency penalties involved.

Advantages of using FPGAs to reduce latency in trading systems.

Architecture of an FPGA-based trading system and its internal components.

Details on how market data is handled and processed in an FPGA-based system.

The role of filters in selecting relevant market data and ignoring unnecessary information.

Unpacking and understanding the structure of market data for trading decisions.

Implementation of trading logic within the FPGA and strategies for order execution.

Challenges in encoding orders in market-specific formats for exchange acceptance.

Importance of limit checking in trading systems to avoid costly mistakes.

TCP stack considerations and the challenges of state management in trading systems.

Optiver's unique approach to FPGA design, emphasizing speed and custom solutions.

Environmental factors that differentiate Optiver's approach to FPGA design and trading.

Innovative solutions and out-of-the-box thinking required for latency reduction in trading systems.

Invitation to the Optiver virtual booth for further questions and discussion.

Transcripts

[Music]

right well good evening

um let me start with a question uh who

have you out there have thought about

the

intersection between fpgas and low

latency trading

um i know i hadn't uh thought about it

much

i got a phone call about 11 years ago

from optiver

asking me if i was interested in an fpga

engineering role

and i remember thinking to myself what

on earth is this company doing

with with fpga they're a trading firm

well

in the next 15 minutes i hope to give

you a little bit of insight into

why and how we use fpgas in this domain

a little agenda for this talk first i'll

explain a little bit about what

trading really is and that can give me a

basis for explaining how fpgas actually

fit in

to this domain um i'll then talk a

little bit about

the architecture of an fpga-based

trading system

and finally i'll discuss a little bit

about how the

how we approach design for fpgas at

optiver and how it's a bit different

than

how other companies which makes use of

fpgas approach them

a little introduction maybe first on

optifer and myself

optiver itself was founded in 1986 in

amsterdam

it's a global market-making firm it

means we're a training firm

we trade all kinds of things but

predominantly options

futures etfs stocks and that kind of

thing

and we do all of this with our own

capital we don't have any clients

this is all done internally

and what's our goal here in the

marketplace it's to improve

the market and we do that by showing

liquidity and what does that mean

that means if if you want to buy

something we'll sell it to you

if you want to sell something we'll buy

it from you but always at prices that we

are

publishing uh yeah to the to the market

at large

um we are a global company

so we've got offices around the world

and from those offices we can cover

exchanges around the world um amsterdam

of course which is well where i'm based

but we also have offices in london

chicago sydney and shanghai about me

personally i've been with optiver for

about 11 years now which maybe tells you

something about how my phone call with

them went 11 years ago

um and in that time i've i co-wrote

the first fpga-based trading system that

we had

and went on to design and implement and

uh

break uh numerous other systems in the

last

11 years to bring to one add new

features and two to to bring

these systems to new exchanges

so what's trading really um yeah we see

we have a photo here this is a trading

pit this is the way that trading was

done

uh well and still is done to some degree

for uh 400 years

it's been done like this people yelling

at each other and

a cluster um and what's actually

happening here is that

one of these people will be saying

something like i want to buy

five shares of of apple for 10 euros

and anyone around this person or at

least with an earshot

uh can can hear that and think oh is

that a good deal or not

and if it is then they can yell back i

want to sell five shares to you

and that's how a trade is done now

in the last 25 years um this type of

trading has largely been replaced with

electronic trading

and that what does that look like it

looks like looks like this

uh these days so this is a data center

and

what happens now is um an exchange

a market can rent physical space um

in in this data center where they can

install their own servers their own

network equipment and this kind of thing

but so can market participants can also

um

rent space uh in in this in this data

center and install

their own equipment but how do we

actually

communicate here and that's that's one

of the keys and

and the old style it was by yelling in

this new style electronic trading it's

by

sending ethernet packets so these data

centers will provide

10 gigabit network connectivity

generally

which you can use to communicate with

the exchange

so now when somebody wants to buy five

shares of apple for

10 euros they have to encode that

information into an

into a network packet and send it to the

exchange's

central server at that point the

exchange can

rebroadcast that information to all

market participants now

where does the latency angle come in

where did yeah where did the lazy come

in here

so imagine that more than one person

wants to sell to the single buyer

imagine a thousand people

want to sell to the single buyer yeah

how do we solve that problem because

from the exchanges point of view there's

going to be a thousand

orders all coming in at the same time

here at the same time

um and one of the ways that an exchange

can solve that problem

is by implementing a kind of first come

first serve

style mechanism and then you can start

to see

why the latency angle starts to matter

which is yeah the first one to get their

order

back to the exchange will be the one who

can affect the trade

um so knowing that latency is

important um what are the limitations

of a software-based trading system um

here we can see uh on the on the left

we have our exchange who's going to be

broadcasting information

and then it's going to hit our server

and on the server we're going to have a

software application which

you know needs to understand what's

happening in order to take a trading

decision

but in order for the software

application to see this we're going to

have to first traverse

the nic so this is a network interface

card we're then going to traverse

pci express and then we get into our you

know our cpu and memory

um where we're going to have you know

our application running

now in this chain um we're gonna be

incurring a latency penalty

crossing this guy crossing pci express

but of course even software applications

are gonna have their own

variance and latency with respect to

getting access to an

actual cpu core imagine the colonel

wants to start doing something on the

processor you're running on and you get

booted off

that's going to incur a latency hit as

well on the memory side

imagine that the memory you're trying to

access gets dropped out of cash

and you're gonna have to go you know

further away to get the information you

actually need

it's also gonna introduce a lot more

variance in your in your latency

because ultimately what we're trying to

do here is get market data

into your application take a trading

decision and get back out again

and there's a lot to accomplish there so

how can fpgas

help us in this next slide

it doesn't look that much different but

there is a key difference which is

the nick has been replaced with an fpga

so what if we can cut away all of the

extra overhead latency of of pci express

and our software application and cpu and

memory

problems and we can do that by trying to

place

all the logic we need to take a trading

decision

as close as possible to the network and

that would be here in the fpga

so if we if we dive deeper into the fpga

we can actually start to see

what what the skeleton of a of an fpga

based trading system looks like

internally um so what do we have here on

the

on the top left so we'll have our market

data so this will be

10 gig ethernet um some you know

network traffic being broadcast

indicating the state of the market

somebody wants to buy somebody wants to

sell that kind of thing

what do we have to do in order to take a

trading decision on this information

well one of the first things that we're

going to need to do is kind of unpack

the lowest level ethernet protocols here

there's some physical layer things we

have to take into account

and we use a mac to help us with this

but the output of this

uh here will be effectively a a packet

stream

a network stream

[Music]

because this is ethernet there's going

to be a number of

networking headers on top of the actual

payload that we

care about so there's going to be

ethernet header an ip

v4 header a udp header and these kinds

of things

and these what we can implement here in

this first block

in the market data handling block is a

filter

um on these networks there's going to be

other networking equipment

blabbing stuff we don't really care

about maybe like routing protocol

updates and that kind of thing and we

don't want any of that to hit our

trading logic that's stuff we want to

ignore so we can filter out

based off of you know higher level

header information

secondly we can use these filters to

only select the instruments that we

really

care about generally exchanges will

partition the

the the stocks uh that that are being

traded on these exchange by multicast

group

so if you know you only care about a

subset of instruments you can also apply

a filter

at this stage to only you know get

access to the instruments you really

care about

so once you've gotten rid of your your

networking headers uh you're left with

just your payload

and this is where you know the meat

of the um of the order message really is

that

somebody wants to buy five shares at 10

euros

now every exchange tends to encode all

this information

and their own native format so you'll

have to consult you know the relevant

exchange

but generally what the way this will

work is that uh the first couple bytes

of the payload will have some kind of

information which tells you

what the structure of the rest of the

packet looks like

so that could be that this is a trade or

that this is the

end of the trading day and it's time to

turn off or it could be

that somebody wants to buy five shares

of apple for 10 euros

so once you once you know what kind of

message this is

then you're gonna start to unpack the

stuff behind it

and that could be like the next four

bytes could indicate

uh the instrument id so is it is this

apple is this google is it tesla is it

whatever um and then you behind next

four bytes behind that could be

this is the price that somebody's

willing to pay represented in

as a as a single precision float for

example

and then the following four bytes could

be the volume how many shares do someone

want to buy

so once you've unpacked all that

information in the market data handling

block

you can then pass it downstream to your

to your trading logic

i won't go into much detail about what

we do at optiver here

um but you could implement you know your

very simple strategy which could just be

if anyone is ever willing to buy apple

for more than

eight euros then sell sell your share of

apple to that person

so yeah up to you to implement what you

actually want in this blog

um leaving this block you'll have your

order you need to send something back to

the exchange to cause a trade to happen

um and what one of the problems that

you'll face

is that just like the market data is

encoded in a market specific

format so are the orders so we need to

make sure that all the information is

placed in the right

spot and encoded in the right way and

that can also be quite a challenge

because sometimes exchanges

like really um unfortunate encodings

like ascii sometimes we need to convert

like binary numbers to ascii

representation

to make them accepted by the exchange

which is not that fun

um so that's that's what kind of logic

you would actually need to

uh build inside your order formatting

block

say additionally another critical thing

to maybe put in this block would be some

kind of limit checking

logic so because we designed these

systems to be as fast as possible

um any any mistakes or any bugs or any

you know um behavior you don't expect

can be very costly so it would be a good

idea to put a limit checking

uh code in here to make sure you don't

send for example

too many orders per second or

that's the prices that you're sending to

the exchange fallen

within some bound uh that you that you

control

uh once you've gone through all that

then we're gonna get to our tcp

uh stack and yeah well one of the

one of the um things that i think almost

every exchange expects

is that the orders that you place going

back to the exchange are encoded in

tcp um which is a bit of a headache

for us doing tcp means a lot of state

tracking there's sequence numbers to

worry about that kind of thing

and as well there are features of tcp in

fact the hallmark of tcp is that it's

reliable

um which means you know your your

message your segment always gets the

other side

but when you really zoom in on what that

means it means that if you send

a segment and it's never acknowledged by

the remote

that you need to send it again and that

means a lot of

state management buffering and that kind

of thing um which is a bit of a headache

finally uh we'll get to the to the

output mac here where we can again

re-encode this information

at the lowest level to be re-transmitted

back to the exchange

to try and make our our trade happen

now we call this this loop we call this

our our critical path

or a hub path right i mean this is we

try to make this path as fast as

possible

um as of a couple of years ago uh we

were

well below 200 nanoseconds um

that would be from from the time that

the you know the first bit

hits the pin of the network connector uh

on the fpga card

until the first bit hits the pin on the

on the egress side that went back to the

exchange

um one one element i didn't touch on on

this is pci express

we still do make use of this um and it

we

make use of this for um status and

control

so we will have a software application

still running on a traditional

server above us um but that's used

for a couple of reasons one is yeah

making sure that whatever logic

you're doing here and the kind of orders

that you're sending back to the exchange

look sane and sensible and make sense

but as well control so when when does it

make sense to turn this thing off

um do we want to do more configuration

on

what kind of multicast groups do we let

through

that kind of thing

uh so finally last slide uh how do we

approach

design differently um at optiver

compared to other companies which make

use of fpgas

first one is the technical side um

there's a couple things here one is

pipelining

um yeah i think for most you know

digital designers

you know they're taught to pipeline

their design right that's just how you

how you do it you put a register on this

on the input and register on the output

and that's just what you do

for us that's that's almost never what

we want to do um

because every register in our hot path

is

latency and we just can't afford to do

that

um an interesting flip side of this

point is that for time and closure um

you know i think for most engineers when

when time enclosure is easy that's great

that means you don't need to worry about

it and you can go on to think about the

next thing

um for us that's not really the case if

time enclosure is too easy

then that probably means there's a clock

cycle somewhere

we can save so we'll deliberately try

and make timing harder

uh well as a result that we can save a

clock somewhere uh yeah we can

we can make it faster so the net result

is that

kind of perpetually in a state where

timing is irritatingly hard to meet

but not impossible um which i don't

think is very normal elsewhere in the

world

um second point here is as best devices

yeah

you know we're we're really on the on

the cutting edge here we will

invest in buying whatever we need to do

to make sure that we are on the on the

on the best possible footing we can be

um

the last point here is around custom

designed ip a lot of

fpga vendors of course provide a lot of

um

you know pre-built pre-packaged ip for

you to use like fifos and that kind of

thing

we don't generally make use of this

stuff at least not on our hotpath

uh most most of this um ip it's robust

and tested

but it's not designed for low latency

purposes so we tend to hand design

everything in our hotpath

uh the second class of things on how we

approach things differently

is more environmental one is

we have a real tight feedback loop on

our work um

i think whereas in some companies you

might not see the result of your work

for

for weeks or months or maybe even years

in our environment you could be making a

change to the code base

you know test benching it going through

a regression suite

but seeing you know a release out the

next day and having it deployed and

running

and you can see the results of your work

literally the next day

um not only seeing it running but also

seeing its impact

on let's say the success of a trading

strategy

we are so dependent on on latency for

some of these things that you can really

see the impact

um directly the second point

is around being a high-paced environment

trading is really dynamic um

there's things are changing all the time

around the world exchanges change the

way that they

operate um opportunities we perceive uh

um come quickly so there's

very rarely a dull day i think in

october there's always something new

uh coming i think the last point

um i wanted to highlight was on yeah

innovative solutions

um yeah tying into the fact that we

don't pipeline for example

means that in order to solve some of the

problems that we have

without introducing latency you have to

get real out-of-the-box thinking

and that could be thinking about novel

ways to

achieve a comparison or to accomplish

some

logic um which yeah it's nice to always

have that kind of fresh challenge

uh present on on a daily basis

um so yeah i think that was that was my

presentation

um thanks for joining me for this

i don't think i have much time to take

questions now

however i will be present in the in the

optiver

uh virtual booth i believe at 7 15

european time so happy to take any more

questions or any

any other things you want to know happy

to talk in that venue

[Music]

thanks