Serving the Quantitative Finance Community

 
User avatar
tags
Topic Author
Posts: 3159
Joined: February 21st, 2010, 12:58 pm

Re: Python tricks

April 23rd, 2021, 12:44 pm

Is there clean way to compare two separate dataframes in long format (dimension1, dimension2, .... dimensionN, value) ?
I have a pile of csv files, one new published each month, containing accumulated values and I need to show what has been going during each month. Any comment on this much welcome!

EDIT:  maybe I pass all the dimensions as as many index levels (multiindex feature of pd.DataFrame) while values are left as pandas columns. it is then easy to make the diff between rows in different columns.
 
User avatar
katastrofa
Posts: 7431
Joined: August 16th, 2007, 5:36 am
Location: Alpha Centauri

Re: Python tricks

April 23rd, 2021, 7:42 pm

You didn't give too many details, but is the diff enough to compare such multidimensional datasets?
Can the components be correlated? Python has lots of nice clustering algorithms, multidimensional scaling, etc. (mostly sklearn) for that. Functional data analysis, baby! :-D

You can simply do diff on a multidimensional df, I think.
 
User avatar
tags
Topic Author
Posts: 3159
Joined: February 21st, 2010, 12:58 pm

Re: Python tricks

April 23rd, 2021, 8:03 pm

thanks for the suggestion kat. apologies i wasn't clear enough earlier. y dataset was simple fundamental market data. i went with the multi-indexing solution that popped in my mind (the one I quickly  commented in my previous post). i hadn't visited the sklearn website for agesm i have to say. the library seems to have grown substantially.
 
 
User avatar
tags
Topic Author
Posts: 3159
Joined: February 21st, 2010, 12:58 pm

Re: Python tricks

December 30th, 2022, 11:40 am

I saw sqlalchemy released a 2.0.0rc1 version.
Did you friends spot something new and worth to explore in it?
 
User avatar
Cuchulainn
Posts: 20205
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Python tricks

March 5th, 2023, 9:31 pm

Breaking News, Pure Python Mode, a game changer 

https://cython.readthedocs.io/en/latest ... /pure.html

With my co-author Harold Kasperink we will have chapters on 

Call C++ from Python and vice versa
Call C# from Python and vice versa
Call C# from native C++ and vice versa
(ménage à trois?)

Modern Multiparadigm Software Architectures and Design Patterns
with Examples and Applications in C++, C# and Python Volume I
Datasim Press 2023
 
Daniel J. Duffy and Harold Kasperink
 
User avatar
Cuchulainn
Posts: 20205
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Python tricks

March 7th, 2023, 2:37 pm

The ultimate Python trick: ChatGPT polymorphic malware.

https://www.cyberark.com/resources/thre ... ic-malware


Feels a bit like mobile intelligent agents a few years back. Exciting times ahead.
 
User avatar
Cuchulainn
Posts: 20205
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Python tricks

May 9th, 2023, 11:37 am

Python getting popular is one of the worst things that has happened to software development so far, it was good in the beginning as more people got interested in software development but now it is just a trap for beginner developers reinforcing their delusions that what they do is more than playing and this is not a proper way to build anything more complex than a calculator app.

But the junior devs are very vocal when you tell them they need to learn a few more things before they become a software developers.
 
User avatar
Cuchulainn
Posts: 20205
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Python tricks

October 26th, 2023, 4:32 pm

Here's a quiz for Python gurus

f1 = lambda x: x*2
f2 = lambda x: 3*x

fA = f1 + f2; doesn't compile
 
User avatar
katastrofa
Posts: 7431
Joined: August 16th, 2007, 5:36 am
Location: Alpha Centauri

Re: Python tricks

October 26th, 2023, 5:51 pm

fA = lambda x: f1(x) + f2(x) # <3
 
User avatar
Cuchulainn
Posts: 20205
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Python tricks

October 26th, 2023, 10:45 pm

fA = lambda x: f1(x) + f2(x) # <3
but I (client code) want fA =  f1 + f2 this is a higher-order function.
 
User avatar
Cuchulainn
Posts: 20205
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Python tricks

October 26th, 2023, 10:47 pm

This code looks a bit like it, although C++ does it much better.

class Func:
        def __init__(self, func):
            self.func = func

        def __add__(self, other):
            return lambda x: self.func(x) + other.func(x)

        def __sub__(self, other):
            return lambda x: self.func(x) - other.func(x)

f1 = Func(lambda x: x**2)
f2 = Func(lambda x: 3 * x)
fA = f1 + f2
print("fA, 2 ", fA(1.0)) # 4.0
 
User avatar
Cuchulainn
Posts: 20205
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Python tricks

October 27th, 2023, 7:10 am

double AckleyGlobalFunctionII(double x)
{ // Building the functions in a HOF assembly process

	double a = 20.0; double b = 0.2; double c = 2.0*3.14159;

	// Bit unwieldy; used to show function vector spaces.
	FType<double> f11 = [&](double x) {return -b*x; };
	FType<double> f1 = -a*exp(f11);

	FType<double> f22 = [&](double x) {return std::cos(c*x); };
	FType<double> f2 = -exp(f22);
//	FType<double> f2A = f2 << f1 << f11;
	FType<double> f3 = assign(a + std::exp(1.0));
	FType<double> Ackley = f1 + f2 + f3;

	return Ackley(x);
}
The issue is it would be nice to make code more readable and maintainable for scipy etc. ("function assembly") instead of a monolith.
 
User avatar
katastrofa
Posts: 7431
Joined: August 16th, 2007, 5:36 am
Location: Alpha Centauri

Re: Python tricks

October 27th, 2023, 9:00 am

You can achieve a similar effect with closure trick:
def summation_operation(*funcs):
    def combined_function(x):
        return sum(func(x) for func in funcs)
    return combined_function
…

combined_function = fA(f1, f2)
 
User avatar
Cuchulainn
Posts: 20205
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Python tricks

October 27th, 2023, 1:45 pm

That works ) what if I want sub, div as well? 2) nested arithmetic? e.g. -f1 + f2/f3 - f4.

Q. is reduce() good?
# function composition
def compose(*functions): # "variadic" list of functions

    # implement as a lambda function
    def composePair(f, g):
        return lambda x: f(g(x)) 

    # Apply function of two pairs composePair to iterable'functions'
    return functools.reduce(composePair, functions, lambda x: x)