SERVING THE QUANTITATIVE FINANCE COMMUNITY

 
User avatar
ISayMoo
Posts: 2336
Joined: September 30th, 2015, 8:30 pm

Re: Python tricks

June 15th, 2020, 9:30 pm

Any tips how to write a simple `setup.py` for the binary .pyd file?
 
User avatar
Cuchulainn
Posts: 62608
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Re: Python tricks

June 16th, 2020, 8:38 am

Any tips how to write a simple `setup.py` for the binary .pyd file?
We did some initial tests a few months back but I need to revisit it again as we shall  wrap (rough) Heston code  in Python.

The setup used was
import os, sys

from distutils.core import setup, Extension
from distutils import sysconfig

cpp_args = ['-std=c++11', '-stdlib=libc++', '-mmacosx-version-min=10.7']

sfc_module = Extension(
    'PybindDLL', sources = ['module.cpp'],
    include_dirs=['pybind11/include'],
    language='c++',
    extra_compile_args = cpp_args,
    )

setup(
    name = 'PybindDLL',
    version = '1.0',
    description = 'Python package with superfastcode2 C++ extension (PyBind11)',
    ext_modules = [sfc_module],
)
Step over the gap, not into it. Watch the space between platform and train.
http://www.datasimfinancial.com
http://www.datasim.nl
 
User avatar
ISayMoo
Posts: 2336
Joined: September 30th, 2015, 8:30 pm

Re: Python tricks

June 16th, 2020, 10:12 pm

I'm building the .pyd file in Visual Studio.
 
User avatar
ISayMoo
Posts: 2336
Joined: September 30th, 2015, 8:30 pm

Re: Python tricks

June 17th, 2020, 9:46 pm

OK, solved it.
from setuptools import setup, find_packages

setup(
    name="PyML",
    version="0.1",
    packages=find_packages(),
    include_package_data=True,
    zip_safe=False,
    package_data={"PyML": ["PyML.pyd", "ML.dll"]}
 
User avatar
Cuchulainn
Posts: 62608
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Re: Python tricks

June 18th, 2020, 8:54 am

Cool. Is this public domain and can we use it for our ML project? 

I tried a number of binders already, by hand. The biggest pain was making that dll 'visible' to the ciient code.
Step over the gap, not into it. Watch the space between platform and train.
http://www.datasimfinancial.com
http://www.datasim.nl
 
User avatar
Cuchulainn
Posts: 62608
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Re: Python tricks

June 24th, 2020, 7:26 pm

I'm building the .pyd file in Visual Studio.
"Thanks a lot. It is helpful.

Another question arises, what if I have multiple sources file?

All of the examples I can find online are for one source file. I am struggling to make it work for project that contains multiple .cpp files."
Step over the gap, not into it. Watch the space between platform and train.
http://www.datasimfinancial.com
http://www.datasim.nl
 
User avatar
ISayMoo
Posts: 2336
Joined: September 30th, 2015, 8:30 pm

Re: Python tricks

June 25th, 2020, 7:20 pm

I'll post the code some time soon. It will be Open Source.
 
User avatar
Cuchulainn
Posts: 62608
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Re: Python tricks

June 26th, 2020, 9:56 am

The goal of the project is a mix of C++ and Python, narrow (replaceable) interfaces and a defined/standardised data flow. C++11 futures and GPU if time permits. A bit like kanban boards.
This is why pybind11 will be useful.
And project risk mitigation (budget == 3 months).
Image
Step over the gap, not into it. Watch the space between platform and train.
http://www.datasimfinancial.com
http://www.datasim.nl
 
User avatar
ISayMoo
Posts: 2336
Joined: September 30th, 2015, 8:30 pm

Re: Python tricks

June 27th, 2020, 7:42 pm

Interesting.
 
User avatar
Cuchulainn
Posts: 62608
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Re: Python tricks

July 11th, 2020, 11:53 am

What's the best/easiest way to expose C++ std::vector and std::vector<std::vector> (matrix) as equivalent Python data structures?
Step over the gap, not into it. Watch the space between platform and train.
http://www.datasimfinancial.com
http://www.datasim.nl
 
User avatar
katastrofa
Posts: 9444
Joined: August 16th, 2007, 5:36 am
Location: Alpha Centauri

Re: Python tricks

July 11th, 2020, 12:22 pm

To what structure? List or array (converting vector of vectors to it)?

Or do you mean sharing access to the same memory? You're writing some ugli codez there... ?
 
User avatar
Cuchulainn
Posts: 62608
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Re: Python tricks

July 11th, 2020, 1:52 pm

From C++ vectors/"matrices" to lists/arrays of tuples would  be fine, for starters. I haven't the time for an odyssey into all the options as it is extremely time-consuming at this stage.
Step over the gap, not into it. Watch the space between platform and train.
http://www.datasimfinancial.com
http://www.datasim.nl
 
User avatar
katastrofa
Posts: 9444
Joined: August 16th, 2007, 5:36 am
Location: Alpha Centauri

Re: Python tricks

July 11th, 2020, 8:14 pm

 
User avatar
Cuchulainn
Posts: 62608
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Re: Python tricks

July 19th, 2020, 11:50 am

This terrein is like being the in the Empty Quarter!
Long story short .. It took one of us 2 weeks to get how it worked (eventually finding the details from a Chinese web site..)
pybind11 has marshalling interface to Eigen library. But for other matrix types

1. Write interface to Eigen, easy?
2.  Write interface to pybind11, time-consuming a.s.

For 1, speed is not a concern since the training is done offline anyways. It takes 2 hours to compute 5 million datapoints (Heston) and store them in MySQL for later processing.
Step over the gap, not into it. Watch the space between platform and train.
http://www.datasimfinancial.com
http://www.datasim.nl
 
User avatar
Cuchulainn
Posts: 62608
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Re: Python tricks

July 19th, 2020, 12:43 pm

I'll post the code some time soon. It will be Open Source.
How's it going here? we would like to test it.
Precise, step-by-step documentation is essential.
Step over the gap, not into it. Watch the space between platform and train.
http://www.datasimfinancial.com
http://www.datasim.nl
ABOUT WILMOTT

PW by JB

Wilmott.com has been "Serving the Quantitative Finance Community" since 2001. Continued...


Twitter LinkedIn Instagram

JOBS BOARD

JOBS BOARD

Looking for a quant job, risk, algo trading,...? Browse jobs here...


GZIP: On