SERVING THE QUANTITATIVE FINANCE COMMUNITY

ISayMoo
Posts: 2334
Joined: September 30th, 2015, 8:30 pm

### Re: Python tricks

Any tips how to write a simple setup.py for the binary .pyd file?

Cuchulainn
Posts: 62586
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Re: Python tricks

Any tips how to write a simple setup.py for the binary .pyd file?
We did some initial tests a few months back but I need to revisit it again as we shall  wrap (rough) Heston code  in Python.

The setup used was
import os, sys

from distutils.core import setup, Extension
from distutils import sysconfig

cpp_args = ['-std=c++11', '-stdlib=libc++', '-mmacosx-version-min=10.7']

sfc_module = Extension(
'PybindDLL', sources = ['module.cpp'],
include_dirs=['pybind11/include'],
language='c++',
extra_compile_args = cpp_args,
)

setup(
name = 'PybindDLL',
version = '1.0',
description = 'Python package with superfastcode2 C++ extension (PyBind11)',
ext_modules = [sfc_module],
)
Step over the gap, not into it. Watch the space between platform and train.
http://www.datasimfinancial.com
http://www.datasim.nl

ISayMoo
Posts: 2334
Joined: September 30th, 2015, 8:30 pm

### Re: Python tricks

I'm building the .pyd file in Visual Studio.

ISayMoo
Posts: 2334
Joined: September 30th, 2015, 8:30 pm

### Re: Python tricks

OK, solved it.
from setuptools import setup, find_packages

setup(
name="PyML",
version="0.1",
packages=find_packages(),
include_package_data=True,
zip_safe=False,
package_data={"PyML": ["PyML.pyd", "ML.dll"]}

Cuchulainn
Posts: 62586
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Re: Python tricks

Cool. Is this public domain and can we use it for our ML project?

I tried a number of binders already, by hand. The biggest pain was making that dll 'visible' to the ciient code.
Step over the gap, not into it. Watch the space between platform and train.
http://www.datasimfinancial.com
http://www.datasim.nl

Cuchulainn
Posts: 62586
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Re: Python tricks

I'm building the .pyd file in Visual Studio.
"Thanks a lot. It is helpful.

Another question arises, what if I have multiple sources file?

All of the examples I can find online are for one source file. I am struggling to make it work for project that contains multiple .cpp files."
Step over the gap, not into it. Watch the space between platform and train.
http://www.datasimfinancial.com
http://www.datasim.nl

ISayMoo
Posts: 2334
Joined: September 30th, 2015, 8:30 pm

### Re: Python tricks

I'll post the code some time soon. It will be Open Source.

Cuchulainn
Posts: 62586
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Re: Python tricks

The goal of the project is a mix of C++ and Python, narrow (replaceable) interfaces and a defined/standardised data flow. C++11 futures and GPU if time permits. A bit like kanban boards.
This is why pybind11 will be useful.
And project risk mitigation (budget == 3 months).
Step over the gap, not into it. Watch the space between platform and train.
http://www.datasimfinancial.com
http://www.datasim.nl

ISayMoo
Posts: 2334
Joined: September 30th, 2015, 8:30 pm

### Re: Python tricks

Interesting.

Cuchulainn
Posts: 62586
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Re: Python tricks

What's the best/easiest way to expose C++ std::vector and std::vector<std::vector> (matrix) as equivalent Python data structures?
Step over the gap, not into it. Watch the space between platform and train.
http://www.datasimfinancial.com
http://www.datasim.nl

katastrofa
Posts: 9436
Joined: August 16th, 2007, 5:36 am
Location: Alpha Centauri

### Re: Python tricks

To what structure? List or array (converting vector of vectors to it)?

Or do you mean sharing access to the same memory? You're writing some ugli codez there... ?

Cuchulainn
Posts: 62586
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Re: Python tricks

From C++ vectors/"matrices" to lists/arrays of tuples would  be fine, for starters. I haven't the time for an odyssey into all the options as it is extremely time-consuming at this stage.
Step over the gap, not into it. Watch the space between platform and train.
http://www.datasimfinancial.com
http://www.datasim.nl

katastrofa
Posts: 9436
Joined: August 16th, 2007, 5:36 am
Location: Alpha Centauri

### Re: Python tricks

Cuchulainn
Posts: 62586
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Re: Python tricks

This terrein is like being the in the Empty Quarter!
Long story short .. It took one of us 2 weeks to get how it worked (eventually finding the details from a Chinese web site..)
pybind11 has marshalling interface to Eigen library. But for other matrix types

1. Write interface to Eigen, easy?
2.  Write interface to pybind11, time-consuming a.s.

For 1, speed is not a concern since the training is done offline anyways. It takes 2 hours to compute 5 million datapoints (Heston) and store them in MySQL for later processing.
Step over the gap, not into it. Watch the space between platform and train.
http://www.datasimfinancial.com
http://www.datasim.nl

Cuchulainn
Posts: 62586
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Re: Python tricks

I'll post the code some time soon. It will be Open Source.
How's it going here? we would like to test it.
Precise, step-by-step documentation is essential.
Step over the gap, not into it. Watch the space between platform and train.
http://www.datasimfinancial.com
http://www.datasim.nl