Page 8 of 13

Re: Python tricks

Posted: July 11th, 2019, 10:25 am
by Cuchulainn
Each language faction lives in its own Swiss Army Knife world. For some jobs you need a Husqvarna or Stihl.
Image

Re: Python tricks

Posted: July 11th, 2019, 12:39 pm
by katastrofa
Always with me!

Re: Python tricks

Posted: July 12th, 2019, 2:38 pm
by Cuchulainn
Is Numba a magic wand that we can use to speed up the Python code for rbtrees?
How 'universal' is Numba?

Re: Python tricks

Posted: July 14th, 2019, 9:30 am
by ISayMoo
I haven't seen it used very widely. Where I work, if the performance offered by Python with Numpy with Pandas with TensorFlow is not good enough, someone writes the core part in C++ and exposes it to Python via Swig.

Re: Python tricks

Posted: July 15th, 2019, 10:49 am
by Cuchulainn
I haven't seen it used very widely. Where I work, if the performance offered by Python with Numpy with Pandas with TensorFlow is not good enough, someone writes the core part in C++ and exposes it to Python via Swig.
I am not a compiler builder but I reckon not everything can be optimised by Numba (I suppose a but like C++ 'inline' is not a guarantee that code will be optimised).

SWIG sounds better because you get performance + data interop. I haven't tried it yet; BTW can we call Python from C++ using SWIG. I believe Boost Python can.

I am supervising several MSc students (who know C++ well)  in ML-PDE-risk projects and are realising that C++ libraries e.g. OpenCV does not have the needed functionality. They are suggesting a pure  Keras but a mixed C++/Python might be the best 'middle ground', especially if we can wrap Keras in  a C++ jacket.

Re: Python tricks

Posted: July 15th, 2019, 11:42 am
by Cuchulainn
#include <boost/python/module.hpp>
#include <boost/python/def.hpp>

char const* greet()
{
   return "hello, world";
}

BOOST_PYTHON_MODULE(hello_ext)
{
    using namespace boost::python;
    def("greet", greet);
This seems to be a common issue..
This code does not build because it cannot find pyconfig.h

Any ideas (Visual Studio 2017 and Python 3.6)?

Re: Python tricks

Posted: July 15th, 2019, 12:23 pm
by FaridMoussaoui
Well, just install the python development tools. On linux, it is seamless (as usual)
sudo apt-get install python-dev
or it is called "python3-devel" instead of python-dev.

Re: Python tricks

Posted: July 15th, 2019, 2:31 pm
by tagoma
Dumb question. are Python36/include/  and  Python36/libs/  (or the like)  in your Includes?

Re: Python tricks

Posted: July 15th, 2019, 4:40 pm
by ISayMoo
I haven't seen it used very widely. Where I work, if the performance offered by Python with Numpy with Pandas with TensorFlow is not good enough, someone writes the core part in C++ and exposes it to Python via Swig.
I am not a compiler builder but I reckon not everything can be optimised by Numba (I suppose a but like C++ 'inline' is not a guarantee that code will be optimised).

SWIG sounds better because you get performance + data interop. I haven't tried it yet; BTW can we call Python from C++ using SWIG. I believe Boost Python can.

I am supervising several MSc students (who know C++ well)  in ML-PDE-risk projects and are realising that C++ libraries e.g. OpenCV does not have the needed functionality. They are suggesting a pure  Keras but a mixed C++/Python might be the best 'middle ground', especially if we can wrap Keras in  a C++ jacket.
IMHO if you can do something in 1 language, try doing so. Mixing different languages always introduces additional cognitive and development overhead and makes debugging harder.

Re: Python tricks

Posted: July 15th, 2019, 8:38 pm
by Cuchulainn
If you define good API interfaces it's worth it?

I don't mind since having become accustomed to using NAG libraries and Hollerith cards in the old days.

More recently:

VBA/Excel and C dlls
C# to C++ via C++/CLI

Trump<-> Putin translator (I don't see the former learning po-russki any time soon).

Re: Python tricks

Posted: July 15th, 2019, 8:57 pm
by ISayMoo
And how was debugging? I understand if you don't want to talk about it.

Re: Python tricks

Posted: July 15th, 2019, 9:23 pm
by katastrofa
But you can learn a lot in the process! Who doesn't want to know every-f***-thing about internal representations of floating point numbers! ;-)

Re: Python tricks

Posted: July 15th, 2019, 9:26 pm
by ISayMoo
That's the fun part!

Re: Python tricks

Posted: July 15th, 2019, 9:46 pm
by Cuchulainn
And how was debugging? I understand if you don't want to talk about it.
https://en.wikipedia.org/wiki/Design_by_contract

Hardware engineers solved this problem some time ago.

A random google 
  • Faulty requirements definition.
  • Client-developer communication failures.
  • Deliberate deviations from software requirements.
  • Logical design errors.
  • Coding errors.
  • Non-compliance with documentation and coding instructions.
  • Shortcomings of the testing process.
  • User interface and procedure errors.

Re: Python tricks

Posted: July 16th, 2019, 9:27 am
by katastrofa
The problem is that those errors are hard to find, e.g. when two well-tested parts of code don't work together and you're left with no clue why.