SERVING THE QUANTITATIVE FINANCE COMMUNITY

 
User avatar
JT77
Topic Author
Posts: 38
Joined: November 27th, 2011, 7:07 pm

FPGAs vs GPUs for high frequency?

January 28th, 2012, 11:17 pm

I am planning my dissertation which will be to showcase my skills for high frequency/low latency trading. I am planning to (attempt to) connect together two computers, one sends simulated data to the other and either:-hack out a lot of the linux kernel, re-write the network stack (write a new one and base it on the FIX protocol) and put a couple of GPUs in. Run monte carlo simulations on the available prices and assess whether the current price is too little/high. Use CUDA, C and C++-Write the monte carlo simulation into VHDL and put onto an FPGA board with a 10Gbit ethernet connection.Which would you say is most likely to provide me with the better skills for a career in this field?Im a little worried the FPGA will be much quicker, but more difficult to calculate the timings for (unless I time them from another hacked-out Linux distro on the price-simulating machine).On the other hand, i'm worried doing this project in GPUs will not help because perhaps the HF houses are moving towards FPGAs? Any advice would be most appreciated
 
User avatar
Cuchulainn
Posts: 62881
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

FPGAs vs GPUs for high frequency?

January 29th, 2012, 8:02 am

Do you have a particular FPGA board you are going to use? I assume you know VHDL?
Last edited by Cuchulainn on January 28th, 2012, 11:00 pm, edited 1 time in total.
Step over the gap, not into it. Watch the space between platform and train.
http://www.datasimfinancial.com
http://www.datasim.nl
 
User avatar
JT77
Topic Author
Posts: 38
Joined: November 27th, 2011, 7:07 pm

FPGAs vs GPUs for high frequency?

January 29th, 2012, 1:53 pm

QuoteOriginally posted by: CuchulainnDo you have a particular FPGA board you are going to use? I assume you know VHDL?IF I go the FPGA route, would most likely be one of these: http://www.advancedio.com/products/func ... olutions/I don't know VHDL but I would teach myself it.
 
User avatar
JT77
Topic Author
Posts: 38
Joined: November 27th, 2011, 7:07 pm

FPGAs vs GPUs for high frequency?

January 29th, 2012, 1:53 pm

QuoteOriginally posted by: CuchulainnDo you have a particular FPGA board you are going to use? I assume you know VHDL?Do you have any advice as to FPGA vs GPU?
 
User avatar
Cuchulainn
Posts: 62881
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

FPGAs vs GPUs for high frequency?

January 29th, 2012, 5:13 pm

QuoteOriginally posted by: JT77QuoteOriginally posted by: CuchulainnDo you have a particular FPGA board you are going to use? I assume you know VHDL?Do you have any advice as to FPGA vs GPU? Just impressions...FPGA seems to have some features like integrated memory and integration with standard h/w platforms and C/C++ compilers. I had a look at Convey once and its seemed useful.It seems you have to write a 'personality' layer in VHDL for each application. VHDL is a bit niche? How long does it take to become an expert in VHDL??Maybe this is usefulhttp://insidehpc.com/2011/07/13/video-accelerators-at-jp-morgan-chase/
Last edited by Cuchulainn on January 28th, 2012, 11:00 pm, edited 1 time in total.
Step over the gap, not into it. Watch the space between platform and train.
http://www.datasimfinancial.com
http://www.datasim.nl
 
User avatar
Polter
Posts: 2526
Joined: April 29th, 2008, 4:55 pm

FPGAs vs GPUs for high frequency?

January 29th, 2012, 5:51 pm

While Verilog and VHDL are indeed the "most widely-used and well-supported HDL", there are alternatives: http://en.wikipedia.org/wiki/Hardware_d ... _designYou may find THDL++ and ParC of interest:http://visualhdl.sysprogs.org/thdlpp.ph ... x1.htmlYou might also want to take a look at Kesturt, S. , Davis, J.D. , Williams, O. BLAS comparison on FPGA, CPU and GPU, Proceedings -- IEEE Annual Symposium on VLSI, ISVLSI 2010:http://gpuscience.com/cs/blas-compariso ... pu/Perhaps you can aim for a similar (i.e., comparison) paper for your HFT application so as to demonstrate your competence in these areas?
Last edited by Polter on January 28th, 2012, 11:00 pm, edited 1 time in total.
 
User avatar
farmer
Posts: 13478
Joined: December 16th, 2002, 7:09 am

FPGAs vs GPUs for high frequency?

January 29th, 2012, 6:14 pm

QuoteOriginally posted by: JT77IF I go the FPGA route, would most likely be one of these: http://www.advancedio.com/products/func ... olutions/I don't know VHDL but I would teach myself it.It sounds like your project is 90% common knowledge - like monte carlo or whatever - and only 10% FPGA. You should instead implement something simple on every hardware alternative offered there, and only then decide which one you would use for what and why. Your project doesn't start when you have chosen the board, it is finished when you have chosen. Or when you have designed a new one suited to a specific application.
 
User avatar
farmer
Posts: 13478
Joined: December 16th, 2002, 7:09 am

FPGAs vs GPUs for high frequency?

January 29th, 2012, 6:39 pm

You could do like a robot wars. Make a linux server rigged up with some crazy ethernet array that sends out a rising and falling number at random. Like 0 0 0 0 0 1 2 3 4 5 4 3 2 1 0 0 0 0 0 0 0 0 0 0 0. Then make the FPGA's play chicken. The idea is to be the first to send a buy when it is rising, and the first to send a sell when it is falling. Plus, the broadcast signal is the sum of the original signal and the sum of received sells and buys. So one FPGA can send out a buy, which will trigger all the other FPGA's, and then sell when they buy without the core even originating the spike. This will require a simple game-theory program on each board, with one board possibly having multiple hands in each game.core:0 0 1 2 3 4 5 4 3 2 1 0sent:0 0 1 2 4 6 7 6 4 2 1 0fpg1:0 0 0 1 0 0 0-1 0 0 0 0fpg2:0 0 0 0 1 0 0 0-1 0 0 0Then see how fast you can get it to cycle. See if you can get up to like 7,000 random spikes a second.
 
User avatar
AlexandreB
Posts: 34
Joined: December 27th, 2008, 2:53 pm

FPGAs vs GPUs for high frequency?

January 29th, 2012, 10:31 pm

QuoteOriginally posted by: farmerYou could do like a robot wars. Make a linux server rigged up with some crazy ethernet array that sends out a rising and falling number at random. Like 0 0 0 0 0 1 2 3 4 5 4 3 2 1 0 0 0 0 0 0 0 0 0 0 0. Then make the FPGA's play chicken. The idea is to be the first to send a buy when it is rising, and the first to send a sell when it is falling. Plus, the broadcast signal is the sum of the original signal and the sum of received sells and buys. So one FPGA can send out a buy, which will trigger all the other FPGA's, and then sell when they buy without the core even originating the spike. This will require a simple game-theory program on each board, with one board possibly having multiple hands in each game.core:0 0 1 2 3 4 5 4 3 2 1 0sent:0 0 1 2 4 6 7 6 4 2 1 0fpg1:0 0 0 1 0 0 0-1 0 0 0 0fpg2:0 0 0 0 1 0 0 0-1 0 0 0Then see how fast you can get it to cycle. See if you can get up to like 7,000 random spikes a second.very interesting, makes me wish i could go back in time and redo my thesis
 
User avatar
JT77
Topic Author
Posts: 38
Joined: November 27th, 2011, 7:07 pm

FPGAs vs GPUs for high frequency?

January 29th, 2012, 11:40 pm

QuoteOriginally posted by: farmerQuoteOriginally posted by: JT77IF I go the FPGA route, would most likely be one of these: http://www.advancedio.com/products/func ... olutions/I don't know VHDL but I would teach myself it.It sounds like your project is 90% common knowledge - like monte carlo or whatever - and only 10% FPGA. You should instead implement something simple on every hardware alternative offered there, and only then decide which one you would use for what and why. Your project doesn't start when you have chosen the board, it is finished when you have chosen. Or when you have designed a new one suited to a specific application.The FPGA option took away most of what I was going to do on my project- namely hacking out the Linux kernel and re-writing my own network stack based on FIX. FPGA makes this all redundant because the data will go directly into the FPGA and won't even touch the CPU..... So are you suggesting do a paper based upon comparing, maybe something like a barrier level (if price is less than X, buy) and compare the latencies for CPU, GPU and FPGA? If so, that would get me out of spending months learning VHDL to write monte-carlo in VHDL and it would make my testing section more interesting, to compare......?
 
User avatar
farmer
Posts: 13478
Joined: December 16th, 2002, 7:09 am

FPGAs vs GPUs for high frequency?

January 30th, 2012, 10:50 am

I don't know how people subscribe to multiple stock tickers. But there might be an opportunity there to use one of the layouts with multiple ethernet jacks, or to design your own parallel layout.Like if you are pairs trading, you could have a separate FPGA, or pair of them for every pair. Then you could have a smart computer update the FPGA's 10 times a second, with simplified trading rules that are only valid in the immediate price range for the next 1/10th second.For this type of idea, you would be designing maybe some kind of custom network demultiplexer, and a way to use multiple components in parallel.
 
User avatar
DrFPGA
Posts: 2
Joined: January 15th, 2012, 4:27 pm

FPGAs vs GPUs for high frequency?

January 30th, 2012, 11:57 pm

Hi JT77,My $0.02 worth:1. Focus and SkillsHow much time do you have to devote to this project? What finance, programming, networking and FPGA / circuit design skills you have already?Does your advisor/supervisor have skills in any of these areas or have you got someone else who can help to guide you?HFT takes many different forms and both FPGA and GPU skills are useful, but so are skills in low-level multi-core multi-threaded CPU programming and optimisation, real-time OS kernel tuning, net driver performance optimisation, FIX, TCP/IP, UDP, PCIe, etc. protocols, low-latency messaging, quirky fast databases, etc. I have no idea what you know already, but you will have to be clever in deciding what you want so that you can actually build something reasonably impressive in your timescale while also learning useful new skills.2. Comparing FPGAs versus GPUs?There?s been already a lot of work in this area and there certainly is more to be done. However, to do a good job of comparing any two technologies you have to be able to design and implement performance benchmarks while trying to push each technology to its limits. Then you have to invent some way of comparing the technologies ?fairly? given that some will be a better match for your benchmark than others. Not easy to do at all, many comparisons out there suffer from flaws in this department. And that?s just the beginning... this is a lot of work! It would be easier to choose just one benchmark with already published results (say a specific MC simulation) and see whether you can improve on it using another specific technology.3. FPGAs, VHDL/VerilogIf you are aiming to optimise performance in an FPGA you need to understand the details of your specific FPGA architecture, its timing characteristics, its various on-chip resources, soft/hard CPU cores, FPGA-vendor specfic design tools, your FPGA platform, you need to be able to know how to optimise your circuits for a specific performance goal, etc., there is lots and lots to learn here. This ultimately means learning VHDL or Verilog to a very high standard (not just the language itself but how to write it for circuit synthesis, how to optimise circuits for FPGAs using the VHDL/Verilog tools + FPGA vendor specific low-level tools). This may take you years, not months.Yes, they are numerous C/Java/C++/etc. based approaches for FPGA ?compilation? and these make the process easier in particular for larger projects (improved productivity, simpler system integration, optimise only what needs to be, often provide out-of-the-box support for specific boards, etc.). Beware that these approaches work through imposing their own computing models on top of a generic FPGA architecture. Unless the computing model matches the application you are implementing well, you will find yourself trying to fit a square peg into a round hole. If your objective is to maximise performance, stick with VHDL/Verilog. One good thing about these languages is that you can combine low-level optimised code (i.e. gates/Boolean expression connected with wires) with high-level sequential/parallel processes triggered by events; there are multiple levels of ?abstraction? hence VHDL/Verilog is really not like writing assembly. If you don?t mind trading some loss of performance for the benefit of productivity, then there are more choices, go with the tool vendor which has good board support as this will save you a lot of time.The above is true about optimising for any architecture (Intel CPUs, NVIDIA GPUs, etc.), age old argument about C++ vs. assembly, but things are more difficult with FPGAs as there are so many new concepts, ?tricks of the trade? and know-how to master.Compare this with GPUs - assuming you know C, you can learn everything you need to know about GPUs in a couple of weeks ? study the architecture, learn CUDA/OpenCL, look at all demos, case studies, examples, read all CUDA finance-related papers, 2-3 books and you are pretty-much set to go. GPUs are popular because learning GPU programming is relatively easy, very similar to learning to program for a different albeit parallel processor. FPGAs are another ball game altogether.I am not trying to put you off, just trying to get you thinking about what you want and can do in your timescale. For example you might not want to be risking spending all that time only to deliver a mediocre, half-working FPGA demo, just because you had to spend a month struggling with the tools or writing your own high-throughput memory/PCIe/PHY interface because your FPGA vendor did not provide one for you (or you thought you could write a better one). Having said all that, FPGA skills are good for many HFT houses.4. Projects - some suggestions- Something to do with low-level network processing in an FPGA ? at TCP/IP/UDP level this has been largely tackled by various TOE (TCP/IP Offload Engine) vendors, there is also some interest in getting FIX implemented onto FPGAs. Looking at how to implement a FIX engine on an FPGA (this is non-trivial, many tradeoffs to consider) might be useful to your career prospects. Ideally, though, you should aim for something simpler, for example start with getting the QuickFIX running on a CPU core inside an FPGA and then try to accelerate only a small part of the FIX protocol. If you can get your hands on a good FPGA TCP/IP module, it would make your life much easier. Or just stick with UDP and ITCH/OUCH protocols. This type of project is probably the most difficult and risky route you can take. If you cannot get the FPGA implementation finished in time, you will not have much to show for your dissertation.- Something with accelerating Monte Carlo simulations on FPGAs and/or GPUs ? a lot of work on this has been done already by several academic and industrial groups. You could look at how the simulation speed changes with complexity of MC models and computational accuracy, how to do large multi-dimensional simulations well and/or show how to do them quickly at sufficient accuracy. GPUs are an obvious choice for MC simulations, but FPGAs offer more flexibility (more opportunities for making the simulation run faster and/or playing with speed/accuracy tradeoffs) and also consume much less power.- Something like the farmer?s idea (nice) ? a game/application incorporating trading-like behaviour and technology challenge. Make it fun, but keep it relevant (e.g. you could read-up on standard HFT strategies and choose to model some of them in a small network). Incorporate FPGAs if you wish, but that might well take your focus away from developing your HFT algorithmic skills.- Something like a model of an optimised trading platform running on a standard multi-core CPU target ? forget FPGAs/GPUs altogether and see how far can you push a tuned Linux kernel with off-the-shelf 10G cards, write a custom network driver to speed-up processing of streamed network data, play with the RT kernel scheduler, low-level assembly optimisation, find a way of measuring communication and processing latency/throughput, etc. Your setup can be as simple as two servers pinging each other, or scale it up to having an actual exchange protocol connecting the two. This might not be as fancy as playing with FPGAs and GPUs, but there are not enough people knowing how to do this sort of stuff well! This is probably the least risky route as you will have ?something working? at each stage as you progress through the project. You will have useful results no matter if something unexpected goes wrong in the process.-I hope this helps. For the sake of readability I have not included references in the above, but feel free to ask for specific pointers if you cannot find more on the web.Best of luck,M.
 
User avatar
ktang
Posts: 122
Joined: January 15th, 2010, 7:16 pm

FPGAs vs GPUs for high frequency?

January 31st, 2012, 10:07 am

 
User avatar
zeta
Posts: 1973
Joined: September 27th, 2005, 3:25 pm
Location: Houston, TX
Contact:

FPGAs vs GPUs for high frequency?

January 31st, 2012, 11:40 pm

I think DrFPGA makes some excellent points. A cheap way to get some experience with fpga at roughly one tenth the price of a virtex board is http://www.fpga4fun.com/. Also checkout ISE webpack, it's relatively easy to get some simple bit files together, there are various projects at fpga4fun
ABOUT WILMOTT

PW by JB

Wilmott.com has been "Serving the Quantitative Finance Community" since 2001. Continued...


Twitter LinkedIn Instagram

JOBS BOARD

JOBS BOARD

Looking for a quant job, risk, algo trading,...? Browse jobs here...


GZIP: On