Serving the Quantitative Finance Community

 
User avatar
Cuchulainn
Posts: 20253
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Python tricks

July 11th, 2019, 10:25 am

Each language faction lives in its own Swiss Army Knife world. For some jobs you need a Husqvarna or Stihl.
Image
 
User avatar
katastrofa
Posts: 7440
Joined: August 16th, 2007, 5:36 am
Location: Alpha Centauri

Re: Python tricks

July 11th, 2019, 12:39 pm

Always with me!
 
User avatar
Cuchulainn
Posts: 20253
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Python tricks

July 12th, 2019, 2:38 pm

Is Numba a magic wand that we can use to speed up the Python code for rbtrees?
How 'universal' is Numba?
 
User avatar
ISayMoo
Posts: 2332
Joined: September 30th, 2015, 8:30 pm

Re: Python tricks

July 14th, 2019, 9:30 am

I haven't seen it used very widely. Where I work, if the performance offered by Python with Numpy with Pandas with TensorFlow is not good enough, someone writes the core part in C++ and exposes it to Python via Swig.
 
User avatar
Cuchulainn
Posts: 20253
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Python tricks

July 15th, 2019, 10:49 am

I haven't seen it used very widely. Where I work, if the performance offered by Python with Numpy with Pandas with TensorFlow is not good enough, someone writes the core part in C++ and exposes it to Python via Swig.
I am not a compiler builder but I reckon not everything can be optimised by Numba (I suppose a but like C++ 'inline' is not a guarantee that code will be optimised).

SWIG sounds better because you get performance + data interop. I haven't tried it yet; BTW can we call Python from C++ using SWIG. I believe Boost Python can.

I am supervising several MSc students (who know C++ well)  in ML-PDE-risk projects and are realising that C++ libraries e.g. OpenCV does not have the needed functionality. They are suggesting a pure  Keras but a mixed C++/Python might be the best 'middle ground', especially if we can wrap Keras in  a C++ jacket.
 
User avatar
Cuchulainn
Posts: 20253
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Python tricks

July 15th, 2019, 11:42 am

#include <boost/python/module.hpp>
#include <boost/python/def.hpp>

char const* greet()
{
   return "hello, world";
}

BOOST_PYTHON_MODULE(hello_ext)
{
    using namespace boost::python;
    def("greet", greet);
This seems to be a common issue..
This code does not build because it cannot find pyconfig.h

Any ideas (Visual Studio 2017 and Python 3.6)?
 
User avatar
FaridMoussaoui
Posts: 327
Joined: June 20th, 2008, 10:05 am
Location: Genève, Genf, Ginevra, Geneva

Re: Python tricks

July 15th, 2019, 12:23 pm

Well, just install the python development tools. On linux, it is seamless (as usual)
sudo apt-get install python-dev
or it is called "python3-devel" instead of python-dev.
 
User avatar
tags
Topic Author
Posts: 3162
Joined: February 21st, 2010, 12:58 pm

Re: Python tricks

July 15th, 2019, 2:31 pm

Dumb question. are Python36/include/  and  Python36/libs/  (or the like)  in your Includes?
 
User avatar
ISayMoo
Posts: 2332
Joined: September 30th, 2015, 8:30 pm

Re: Python tricks

July 15th, 2019, 4:40 pm

I haven't seen it used very widely. Where I work, if the performance offered by Python with Numpy with Pandas with TensorFlow is not good enough, someone writes the core part in C++ and exposes it to Python via Swig.
I am not a compiler builder but I reckon not everything can be optimised by Numba (I suppose a but like C++ 'inline' is not a guarantee that code will be optimised).

SWIG sounds better because you get performance + data interop. I haven't tried it yet; BTW can we call Python from C++ using SWIG. I believe Boost Python can.

I am supervising several MSc students (who know C++ well)  in ML-PDE-risk projects and are realising that C++ libraries e.g. OpenCV does not have the needed functionality. They are suggesting a pure  Keras but a mixed C++/Python might be the best 'middle ground', especially if we can wrap Keras in  a C++ jacket.
IMHO if you can do something in 1 language, try doing so. Mixing different languages always introduces additional cognitive and development overhead and makes debugging harder.
 
User avatar
Cuchulainn
Posts: 20253
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Python tricks

July 15th, 2019, 8:38 pm

If you define good API interfaces it's worth it?

I don't mind since having become accustomed to using NAG libraries and Hollerith cards in the old days.

More recently:

VBA/Excel and C dlls
C# to C++ via C++/CLI

Trump<-> Putin translator (I don't see the former learning po-russki any time soon).
 
User avatar
ISayMoo
Posts: 2332
Joined: September 30th, 2015, 8:30 pm

Re: Python tricks

July 15th, 2019, 8:57 pm

And how was debugging? I understand if you don't want to talk about it.
 
User avatar
katastrofa
Posts: 7440
Joined: August 16th, 2007, 5:36 am
Location: Alpha Centauri

Re: Python tricks

July 15th, 2019, 9:23 pm

But you can learn a lot in the process! Who doesn't want to know every-f***-thing about internal representations of floating point numbers! ;-)
 
User avatar
ISayMoo
Posts: 2332
Joined: September 30th, 2015, 8:30 pm

Re: Python tricks

July 15th, 2019, 9:26 pm

That's the fun part!
 
User avatar
Cuchulainn
Posts: 20253
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Python tricks

July 15th, 2019, 9:46 pm

And how was debugging? I understand if you don't want to talk about it.
https://en.wikipedia.org/wiki/Design_by_contract

Hardware engineers solved this problem some time ago.

A random google 
  • Faulty requirements definition.
  • Client-developer communication failures.
  • Deliberate deviations from software requirements.
  • Logical design errors.
  • Coding errors.
  • Non-compliance with documentation and coding instructions.
  • Shortcomings of the testing process.
  • User interface and procedure errors.
 
User avatar
katastrofa
Posts: 7440
Joined: August 16th, 2007, 5:36 am
Location: Alpha Centauri

Re: Python tricks

July 16th, 2019, 9:27 am

The problem is that those errors are hard to find, e.g. when two well-tested parts of code don't work together and you're left with no clue why.