Page 2 of 2

Re: Open binary file format

Posted: April 10th, 2019, 8:07 pm
by katastrofa
Thank you, Farid. I'm only asking for your opinion if you have spare time and are keen on cracking others' CraPPy codes :-)
Re seconds_t, it's just a redefinition of the double type.

Re: Open binary file format

Posted: April 24th, 2019, 12:18 pm
by rmax
Depends on the use case. KDB? not a binary format but perhaps you need more than that...

Re: Open binary file format

Posted: May 2nd, 2019, 2:04 pm
by Cuchulainn
Can you recommend an open binary format to store large simulation output consisting of columns of numbers and strings? (I can get rid of strings if necessary.)
Maybe this?

Is xarray to be preferred over ndarray?

Re: Open binary file format

Posted: October 7th, 2019, 6:56 pm
by Cuchulainn
Hey, let's say you wanted to solve the 3d heat equation numerically. Could HDF5 be used

1. In 'real-time' on a machine (awful slow?)
2. Off line training and then store the HDF5 data for later generalisation?

xarray + NetCDF looks better.

Re: Open binary file format

Posted: October 7th, 2019, 10:58 pm
by FaridMoussaoui
Until you do the experiments, can't conclude.

Bethel and al, analysed the 2010 Flash Crash using HDF5 as their internal data format with GB of data on HPC systems.

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1939522

As usual, you can get the document throught sci-hub.