SERVING THE QUANTITATIVE FINANCE COMMUNITY

 
User avatar
katastrofa
Topic Author
Posts: 8494
Joined: August 16th, 2007, 5:36 am
Location: Alpha Centauri

Re: Open binary file format

April 10th, 2019, 8:07 pm

Thank you, Farid. I'm only asking for your opinion if you have spare time and are keen on cracking others' CraPPy codes :-)
Re seconds_t, it's just a redefinition of the double type.
 
User avatar
rmax
Posts: 6080
Joined: December 8th, 2005, 9:31 am

Re: Open binary file format

April 24th, 2019, 12:18 pm

Depends on the use case. KDB? not a binary format but perhaps you need more than that...
 
User avatar
Cuchulainn
Posts: 60470
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Re: Open binary file format

May 2nd, 2019, 2:04 pm

Can you recommend an open binary format to store large simulation output consisting of columns of numbers and strings? (I can get rid of strings if necessary.)
Maybe this?

Is xarray to be preferred over ndarray?
http://www.datasimfinancial.com
http://www.datasim.nl

Approach your problem from the right end and begin with the answers. Then one day, perhaps you will find the final question..
R. van Gulik
 
User avatar
Cuchulainn
Posts: 60470
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Re: Open binary file format

October 7th, 2019, 6:56 pm

Hey, let's say you wanted to solve the 3d heat equation numerically. Could HDF5 be used

1. In 'real-time' on a machine (awful slow?)
2. Off line training and then store the HDF5 data for later generalisation?

xarray + NetCDF looks better.
http://www.datasimfinancial.com
http://www.datasim.nl

Approach your problem from the right end and begin with the answers. Then one day, perhaps you will find the final question..
R. van Gulik
 
User avatar
FaridMoussaoui
Posts: 453
Joined: June 20th, 2008, 10:05 am
Location: Genève, Genf, Ginevra, Geneva

Re: Open binary file format

October 7th, 2019, 10:58 pm

Until you do the experiments, can't conclude.

Bethel and al, analysed the 2010 Flash Crash using HDF5 as their internal data format with GB of data on HPC systems.

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1939522

As usual, you can get the document throught sci-hub.
ABOUT WILMOTT

PW by JB

Wilmott.com has been "Serving the Quantitative Finance Community" since 2001. Continued...


Twitter LinkedIn Instagram

JOBS BOARD

JOBS BOARD

Looking for a quant job, risk, algo trading,...? Browse jobs here...


GZIP: On