Serving the Quantitative Finance Community

 
User avatar
ISayMoo
Posts: 2332
Joined: September 30th, 2015, 8:30 pm

Re: Python tricks

June 15th, 2020, 9:30 pm

Any tips how to write a simple `setup.py` for the binary .pyd file?
 
User avatar
Cuchulainn
Posts: 20255
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Python tricks

June 16th, 2020, 8:38 am

Any tips how to write a simple `setup.py` for the binary .pyd file?
We did some initial tests a few months back but I need to revisit it again as we shall  wrap (rough) Heston code  in Python.

The setup used was
import os, sys

from distutils.core import setup, Extension
from distutils import sysconfig

cpp_args = ['-std=c++11', '-stdlib=libc++', '-mmacosx-version-min=10.7']

sfc_module = Extension(
    'PybindDLL', sources = ['module.cpp'],
    include_dirs=['pybind11/include'],
    language='c++',
    extra_compile_args = cpp_args,
    )

setup(
    name = 'PybindDLL',
    version = '1.0',
    description = 'Python package with superfastcode2 C++ extension (PyBind11)',
    ext_modules = [sfc_module],
)
 
User avatar
ISayMoo
Posts: 2332
Joined: September 30th, 2015, 8:30 pm

Re: Python tricks

June 16th, 2020, 10:12 pm

I'm building the .pyd file in Visual Studio.
 
User avatar
ISayMoo
Posts: 2332
Joined: September 30th, 2015, 8:30 pm

Re: Python tricks

June 17th, 2020, 9:46 pm

OK, solved it.
from setuptools import setup, find_packages

setup(
    name="PyML",
    version="0.1",
    packages=find_packages(),
    include_package_data=True,
    zip_safe=False,
    package_data={"PyML": ["PyML.pyd", "ML.dll"]}
 
User avatar
Cuchulainn
Posts: 20255
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Python tricks

June 18th, 2020, 8:54 am

Cool. Is this public domain and can we use it for our ML project? 

I tried a number of binders already, by hand. The biggest pain was making that dll 'visible' to the ciient code.
 
User avatar
Cuchulainn
Posts: 20255
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Python tricks

June 24th, 2020, 7:26 pm

I'm building the .pyd file in Visual Studio.
"Thanks a lot. It is helpful.

Another question arises, what if I have multiple sources file?

All of the examples I can find online are for one source file. I am struggling to make it work for project that contains multiple .cpp files."
 
User avatar
ISayMoo
Posts: 2332
Joined: September 30th, 2015, 8:30 pm

Re: Python tricks

June 25th, 2020, 7:20 pm

I'll post the code some time soon. It will be Open Source.
 
User avatar
Cuchulainn
Posts: 20255
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Python tricks

June 26th, 2020, 9:56 am

The goal of the project is a mix of C++ and Python, narrow (replaceable) interfaces and a defined/standardised data flow. C++11 futures and GPU if time permits. A bit like kanban boards.
This is why pybind11 will be useful.
And project risk mitigation (budget == 3 months).
Image
 
User avatar
ISayMoo
Posts: 2332
Joined: September 30th, 2015, 8:30 pm

Re: Python tricks

June 27th, 2020, 7:42 pm

Interesting.
 
User avatar
Cuchulainn
Posts: 20255
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Python tricks

July 11th, 2020, 11:53 am

What's the best/easiest way to expose C++ std::vector and std::vector<std::vector> (matrix) as equivalent Python data structures?
 
User avatar
katastrofa
Posts: 7440
Joined: August 16th, 2007, 5:36 am
Location: Alpha Centauri

Re: Python tricks

July 11th, 2020, 12:22 pm

To what structure? List or array (converting vector of vectors to it)?

Or do you mean sharing access to the same memory? You're writing some ugli codez there... ?
 
User avatar
Cuchulainn
Posts: 20255
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Python tricks

July 11th, 2020, 1:52 pm

From C++ vectors/"matrices" to lists/arrays of tuples would  be fine, for starters. I haven't the time for an odyssey into all the options as it is extremely time-consuming at this stage.
 
User avatar
katastrofa
Posts: 7440
Joined: August 16th, 2007, 5:36 am
Location: Alpha Centauri

Re: Python tricks

July 11th, 2020, 8:14 pm

 
User avatar
Cuchulainn
Posts: 20255
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Python tricks

July 19th, 2020, 11:50 am

This terrein is like being the in the Empty Quarter!
Long story short .. It took one of us 2 weeks to get how it worked (eventually finding the details from a Chinese web site..)
pybind11 has marshalling interface to Eigen library. But for other matrix types

1. Write interface to Eigen, easy?
2.  Write interface to pybind11, time-consuming a.s.

For 1, speed is not a concern since the training is done offline anyways. It takes 2 hours to compute 5 million datapoints (Heston) and store them in MySQL for later processing.
 
User avatar
Cuchulainn
Posts: 20255
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Python tricks

July 19th, 2020, 12:43 pm

I'll post the code some time soon. It will be Open Source.
How's it going here? we would like to test it.
Precise, step-by-step documentation is essential.