Serving the Quantitative Finance Community

 
User avatar
JinhuaColin
Topic Author
Posts: 3
Joined: May 30th, 2013, 4:33 pm

how is HFT possible

September 21st, 2017, 11:38 pm

Hi guys, i was working on a small automated trading project myself and this crazy question popped up in my head. How do people achieve a millisecond trading strategy?  A C++ program takes 0.2s even i only program it to initialize a class object and run a 100-cycle for loop. i guess most trading strategies have a optimization process leaving alone those more complex machine learning strategies; also some strategies need to handle hundreds of stocks. Is it really true a program can finish all this within milliseconds, or is "millisecond" just a saying?  i understand there are more advanced tools/techniques out there like GUP computing, parallel computing and distributed computing etc which can greatly reduce the computation time. But it is still difficult for me to believe this millisecond run time. Appreciate your help!
 
User avatar
Alan
Posts: 2958
Joined: December 19th, 2001, 4:01 am
Location: California
Contact:

Re: how is HFT possible

September 22nd, 2017, 5:33 am

Not my area, but it is hardware-based acceleration, pushing the trading logic to the network card with FPGA's -- for example see here for a circa 2012 discussion, probably with quite outdated timings by now. An excerpt:
For these products, precise latency figures are application dependent and difficult to obtain. However, Accelize claims their smart NICs can turn around financial trades on the FPGA in under 2µs. This is significantly lower than the minimum latency achievable with software, as discussed earlier.
 
User avatar
Billy7
Posts: 262
Joined: March 30th, 2016, 2:12 pm

Re: how is HFT possible

September 22nd, 2017, 10:57 am

Apart from the specialized hardware technologies/architectures such as the one Alan mentioned (which I was not aware of), even on the software side the initial premise for your disbelief (" A C++ program takes 0.2s even i only program it to initialize a class object and run a 100-cycle for loop") is false. Maybe when something starts and has to load a few things into memory it can take 0.2 secs, or maybe you're just starting the debugger? But as soon as the application is up and running, what you mention can be done in less than a microsecond (though you don't specify what every cycle loop does, but I'm assuming you're not talking about simulations of the universe!) 
 
User avatar
LocalVolatility
Posts: 13
Joined: May 27th, 2009, 10:07 am
Location: Amsterdam
Contact:

Re: how is HFT possible

October 9th, 2017, 9:53 am

 
User avatar
ISayMoo
Posts: 2332
Joined: September 30th, 2015, 8:30 pm

Re: how is HFT possible

October 28th, 2017, 6:28 pm

You can get microsecond latencies on Intel CPUs, no need to go for FPGA (which are a must for sub-microsecond latencies).

Principles:
  1. keep it simple (no fancy stuff in the main loop)
  2. pin it to the core (no core switching)
  3. keep it single-threaded
  4. overclock
  5. write your own drivers
  6. buy expensive, fast network cards
 
User avatar
snufkin
Posts: 64
Joined: January 25th, 2017, 9:05 am
Location: Cambridge

Re: how is HFT possible

November 3rd, 2017, 8:33 am

"millisecond" latency is easy even if you relax and program in Python:
Python 2.7.13 (default, Apr  4 2017, 08:47:57) 
[GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.38)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> def cycle():
...     for x in range(100):
...         print x
... 
>>> import time
>>> def measure():
...     t1 = time.time()
...     cycle()
...     t2 = time.time()
...     return t2 - t1
... 
>>> print measure()
0
1
# ... skipped ...
98
99
0.000275135040283
>>> 

Here you go, sub-milisecond timing in Python with an awful output. Just don't run your Monte Carlo, and you're likely to be fine for a draft project.
 
User avatar
outrun
Posts: 4573
Joined: January 1st, 1970, 12:00 am

Re: how is HFT possible

November 3rd, 2017, 8:43 am

Your cycle() time is roughly e*10E-5.

That would mean that the code below runs in 148.4123 seconds?
def measure():
  t1 = time.time()
  for _ in range(10):
    for _ in range(10):
      for _ in range(10):
        for _ in range(10):
          for _ in range(10):
            cycle()
  t2 = time.time()
  return t2 - t1
 
User avatar
snufkin
Posts: 64
Joined: January 25th, 2017, 9:05 am
Location: Cambridge

Re: how is HFT possible

November 3rd, 2017, 10:50 am

Your cycle() time is roughly e*10E-5.

That would mean that the code below runs in 148.4123 seconds?
32.5 second on my laptop, why? (Using the highly scientific approach of only taking one measure).
 
User avatar
outrun
Posts: 4573
Joined: January 1st, 1970, 12:00 am

Re: how is HFT possible

November 3rd, 2017, 11:06 am

Your cycle() time is roughly e*10E-5.

That would mean that the code below runs in 148.4123 seconds?
32.5484418869 on my laptop, why?
There is an obsession in the brainteasersforum discussing weird method of computing [$]e^5[$]. I though this was a new method but it's not very accurate!
 
User avatar
snufkin
Posts: 64
Joined: January 25th, 2017, 9:05 am
Location: Cambridge

Re: how is HFT possible

November 3rd, 2017, 11:18 am

Your cycle() time is roughly e*10E-5.

That would mean that the code below runs in 148.4123 seconds?
32.5484418869 on my laptop, why?
There is an obsession in the brainteasersforum discussing weird method of computing [$]e^5[$]. I though this was a new method but it's not very accurate!
That's a JIT optimised e! When you do HFT, you need to tune your constants, too.
 
User avatar
outrun
Posts: 4573
Joined: January 1st, 1970, 12:00 am

Re: how is HFT possible

November 3rd, 2017, 11:35 am

32.5484418869 on my laptop, why?
There is an obsession in the brainteasersforum discussing weird method of computing [$]e^5[$]. I though this was a new method but it's not very accurate!
That's a JIT optimised e! When you do HFT, you need to tune your constants, too.
Ah, yes, I forgot the HFT JIT correction factor of [$]\sqrt{21}[$]!
 
User avatar
ISayMoo
Posts: 2332
Joined: September 30th, 2015, 8:30 pm

Re: how is HFT possible

November 3rd, 2017, 8:19 pm

People were running low-latency strategies from Perl or KDB, nothing new here.
 
User avatar
outrun
Posts: 4573
Joined: January 1st, 1970, 12:00 am

Re: how is HFT possible

November 3rd, 2017, 9:05 pm

People were running low-latency strategies from Perl or KDB, nothing new here.
What can possibly go wrong!..
Image
 
User avatar
ISayMoo
Posts: 2332
Joined: September 30th, 2015, 8:30 pm

Re: how is HFT possible

November 4th, 2017, 9:46 am

They made money, which is #1 criteria of success in finance.
 
bingobangobongo
Posts: 3
Joined: January 25th, 2017, 4:00 pm

Re: how is HFT possible

January 30th, 2018, 12:13 am

most of it is bypassing the kernel for network IO, keeping the CPU cache lines full, pinning threads to specific cpus and minimising/eliminating context switches. Any language can be used to achieve this, C++ simply gives you a little bit more control.