Page 2 of 3

Re: We've developed (free) software to predict volatility + other vital tools...

Posted: December 5th, 2018, 5:17 pm
by Cuchulainn
I am using 3 methods to test if the data is random or not, as you can guess most sets of stock prices appear as most likely random according to all of the tests! 

When they aren't I am using the NN, Genetic algo and Levenberg-Marqrant to see what can be predicted and what not or better yet just put them through the volatility test tool to see how much % is the real change on month-to-month basis so that I have an idea if I can buy cheap put/call OTM. 

I am interested in the rationale and the (inner workings) for choosing these three methods. They are a bit of a motley crew. They use a mix of heuristics and have very different properties.

Output from them feels like a description rather an explanation.

1. What is NN in this case? the usual GD, SGD,stuff with BPN and learning rates? 
2. What's the advantage of GA compared to say Differential Evolution which IMO is more versatile? I thought GA was a bit passe, but maybe it's on its way back.
3. LevMar is a Opel Kadette .. it works on most days but it is not very robust.

1 and 2 only produce local minima at best. Can't remember if this is also true of GA..

Do you have reports on how these methods compare to each other based on a range of data.

Re: We've developed (free) software to predict volatility + other vital tools...

Posted: December 5th, 2018, 7:56 pm
by alpher
@Cuchulain: i'll get back to you asap or you can PM me if you really like to know :) but long story short:

1. Multy-layer NN, I think the classic: multi layer perceptron
2. Can't say there's a huge advantage other than GA's ability to "learn" quickly from known data (supervised) but it's not CPU friendly (my implmentation at least!) and its predictive power isn't that good compared to LevMar
3. it is yes, it seems better than NN for some cases though. 

I'll show more reports and screenshots soon as promised. 

Re: We've developed (free) software to predict volatility + other vital tools...

Posted: December 6th, 2018, 12:42 pm
by Cuchulainn
GAs are inherently parallel so I would expect parallel design patterns (master-slave etc.) to be useful. These days desktops with lots of processsors are common so this should improve speedup?

For pure optimisation one trick is to use GA/DE to get the initial guess close to the exact solution (first few iterations) and then let LevMar do its job.

I think tackling this problem from three different approaches can't be a bad idea.

Re: We've developed (free) software to predict volatility + other vital tools...

Posted: December 22nd, 2018, 7:38 pm
by alpher
^ I know, I was too busy focusing on the predicting part, rather than thinking of performance, especially since performance isn't a big issue - I could easily wait for some 5 minutes for prediction since I'm not in HF anyway. 

But about speedup, in C# I know I can use the great Parralel,Invoke or in C they had something like amp_task or called similar. My point is: I am using ways that improve based on free resources and NOT based on threads of CPUs. The cpu/thread method is laughably ineffective for plenty of cases, including this one. 

Re: We've developed (free) software to predict volatility + other vital tools...

Posted: December 28th, 2018, 9:12 am
by Cuchulainn
2. Can't say there's a huge advantage other than GA's ability to "learn" quickly from known data (supervised) but it's not CPU friendly (my implmentation at least!) and its predictive power isn't that good compared to LevMar

@alpher,
What is your implementation, precisely? I had a look at your documentation on GA Beyond seeding the initial generation and using roulette wheel (both not uptodate) there is not much to go on to convince a potential client. BTW you don't mention Elitist selection in the Manual but you can see it in the screen shot.

I built a simple GA program for fun myself just to understand and _then_ I installed Accord.NET for all things ML, levMar, NN, SVM.

And then the boombshell: all the screen shots in your 1.48 Manual are from Accord.Net but you do not mention this crucial fact!! 
I can reproduce your screen shot at will without forking out 3K.

What's going on? Is there something that I have missed?


// Page numbers in manual makes light work. meh

Re: We've developed (free) software to predict volatility + other vital tools...

Posted: December 28th, 2018, 9:56 am
by Cuchulainn
My 101 test case for Accord.Net GA from the yugely popular [$]e^5[$] thread.

Image

Re: We've developed (free) software to predict volatility + other vital tools...

Posted: December 28th, 2018, 11:37 am
by Cuchulainn
So you are THE person using "access" as a database.

I am a Linux user so your soft is useless for me.
C# seems to be the engine, so porting to linux/Mono is TBD??

Re: We've developed (free) software to predict volatility + other vital tools...

Posted: December 28th, 2018, 1:33 pm
by alpher
The genetic algo + LB are the only tools taken 99% from accord/afroge...and to be more specific it's not even accord it's aforge since the accord dev have taken then 100% from the aforge originals...but since the documentation is 42+ pages with 20+ modules I am not asking for $3k for them...just as I am not asking for the black-shcoles modules which is also included but BS is all over google: for free. I even had charts for options and swtocks but removed them since someone made the same comment as you that they're useless since you can find such all over the net for free...

The rest: 

backtesting strategies, 
improved nn,
kelly cr. visualisation simulation of capital evolution;
levy, normal, logn pdf/cdf export;
real time all nasdaq stocks dividend scan,
version in chinese/mandarin;
average % change mean between prices;
all volatility points extracted with single click
randomness tests. 

etc etc. - I actually find the GA to be super slow and inaccurate when tested vs say all trading points of MSFT since 1986, the error is so big it makes the prediction useless. It works though for predictable volatility.

Re: We've developed (free) software to predict volatility + other vital tools...

Posted: December 28th, 2018, 1:35 pm
by alpher
PS
It's mentioned in the 'about' in order to compply with LGPL licenses. 

Re: We've developed (free) software to predict volatility + other vital tools...

Posted: December 28th, 2018, 3:14 pm
by FaridMoussaoui
I stick to standard softwares, e-g written in a well known language (C/C++/Fortran) and portable (e.g OS independent)

Numerical programming in C# is a joke (as well the Q# thing in the Quantum programming thread)

Re: We've developed (free) software to predict volatility + other vital tools...

Posted: December 28th, 2018, 3:31 pm
by alpher
^ OK then "Microsoft are a joke" according to the Linux expert...I rest my case here.

Re: We've developed (free) software to predict volatility + other vital tools...

Posted: December 28th, 2018, 3:38 pm
by FaridMoussaoui
While Linux is my prefered working environment, I do programming on windows OS (C++/R). I also do programming for Excel sheets (through C++).

Re: We've developed (free) software to predict volatility + other vital tools...

Posted: December 28th, 2018, 4:04 pm
by Cuchulainn
Yeah, can't beat Fortran for numerics. C++ is OK too. 
For other things, C# is super.

Re: We've developed (free) software to predict volatility + other vital tools...

Posted: December 28th, 2018, 4:07 pm
by Cuchulainn
The genetic algo + LB are the only tools taken 99% from accord/afroge...and to be more specific it's not even accord it's aforge since the accord dev have taken then 100% from the aforge originals...but since the documentation is 42+ pages with 20+ modules I am not asking for $3k for them...just as I am not asking for the black-shcoles modules which is also included but BS is all over google: for free. I even had charts for options and swtocks but removed them since someone made the same comment as you that they're useless since you can find such all over the net for free...

The rest: 

backtesting strategies, 
improved nn,
kelly cr. visualisation simulation of capital evolution;
levy, normal, logn pdf/cdf export;
real time all nasdaq stocks dividend scan,
version in chinese/mandarin;
average % change mean between prices;
all volatility points extracted with single click
randomness tests. 

etc etc. - I actually find the GA to be super slow and inaccurate when tested vs say all trading points of MSFT since 1986, the error is so big it makes the prediction useless. It works though for predictable volatility.
OK

etc etc. - I actually find the GA to be super slow and inaccurate when tested vs say all trading points of MSFT since 1986, the error is so big it makes the prediction useless. It works though for predictable volatility.

1. Why is it super slow .. have you profiled it?
2. Why then are you offering it as part of your product? 

Re: We've developed (free) software to predict volatility + other vital tools...

Posted: December 28th, 2018, 4:33 pm
by Cuchulainn
^ OK then "Microsoft are a joke" according to the Linux expert...I rest my case here.
That's not what Farid is saying. Don'y give in. Tell us how fast your stuff is.