SERVING THE QUANTITATIVE FINANCE COMMUNITY

 
User avatar
ISayMoo
Topic Author
Posts: 1083
Joined: September 30th, 2015, 8:30 pm

Re: If you are bored with Deep Networks

July 5th, 2018, 9:10 pm

E is stochastic (because mini-batches). Does your argument still apply?
 
User avatar
katastrofa
Posts: 6399
Joined: August 16th, 2007, 5:36 am
Location: Alpha Centauri

Re: If you are bored with Deep Networks

July 5th, 2018, 10:59 pm

That's a fair reasoning, but after it clashes against the big data, you end up with SGD (as far as I can tell).
 
User avatar
frolloos
Posts: 1566
Joined: September 27th, 2007, 5:29 pm
Location: Netherlands

Re: If you are bored with Deep Networks

July 6th, 2018, 1:34 am

T4A, you are a great example that some animals are very bad at learning anything, including proper manners.
I was going through this thread as I know nothing about deep networks (only superficial ones) and you appear to be quite/very knowledgeable on this.

But then what caught my attention is how the two of you always manage to get into ego-arguments with others, are you building your deep network?

No wonder people are leaving.
 
User avatar
ISayMoo
Topic Author
Posts: 1083
Joined: September 30th, 2015, 8:30 pm

Re: If you are bored with Deep Networks

July 6th, 2018, 1:41 am

Would you mind leaving personal attacks to Off topic forum?
 
User avatar
frolloos
Posts: 1566
Joined: September 27th, 2007, 5:29 pm
Location: Netherlands

Re: If you are bored with Deep Networks

July 6th, 2018, 1:54 am

As you did with T4A, Outrun, in this thread and others in non OT (and OT) threads as well?
 
Sure matey, whatever you say. 
 
User avatar
katastrofa
Posts: 6399
Joined: August 16th, 2007, 5:36 am
Location: Alpha Centauri

Re: If you are bored with Deep Networks

July 6th, 2018, 2:26 am

You freaked out when I corrected you on something in the past, frolloos. You and T4A started to make fun of me in quite an unsophisticated way and you continue your attempts to attack me (when I criticised the Dutch burqa ban, you accused me of calling you a bigot, even if I didn't know your views at that point; you took an offence at my friendly joke because I metaphorically referred to you as "something" and not "someone", etc). It's surprising to see you taking the high moral ground here, and even more so accusing others of ego problems). I don't know if that's the case, but if someone leaves because they cannot spew more bullshit, it's not bad. Maybe next time they will make more effort.

As for your accusations, what has ISayMoo done to T4A, not to mention outrun and some mysterious others?
Last edited by katastrofa on July 6th, 2018, 9:26 am, edited 1 time in total.
 
User avatar
Cuchulainn
Posts: 57050
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Re: If you are bored with Deep Networks

July 6th, 2018, 7:23 am

T4A, you are a great example that some animals are very bad at learning anything, including proper manners.
I was going through this thread as I know nothing about deep networks (only superficial ones) and you appear to be quite/very knowledgeable on this.
I agree on both counts. Friendly advice; write up or shut up. Fight on OT, not here.

Kata's comments on the quality (or lack of) of posts are well-founded and reflect my own opinion And it's on record.

There has been a yuge amount of waffle lately. I feel it reached a peak in the last few weeks. It is maddening.

Maybe you could delete your posts. If you look you can see I'm in a useful conversation with ISM and I now have to wade through this muck you have posted.
 
User avatar
katastrofa
Posts: 6399
Joined: August 16th, 2007, 5:36 am
Location: Alpha Centauri

Re: If you are bored with Deep Networks

July 6th, 2018, 10:49 am

Sine using tensorflow RNN, with linear activation function + SGD:
linear_sgd.png
 
User avatar
katastrofa
Posts: 6399
Joined: August 16th, 2007, 5:36 am
Location: Alpha Centauri

Re: If you are bored with Deep Networks

July 6th, 2018, 10:51 am

and sigmoid + SGD :-/
sigmoid_sgd.png
 
User avatar
Cuchulainn
Posts: 57050
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Re: If you are bored with Deep Networks

July 6th, 2018, 11:05 am

linear == ReLU?

The x-axis is [0,5000]. Are you modelling sin(ax) for some a?

//
edit: can we the 101 case sin(x) on [0, 8*pi]?
 
User avatar
katastrofa
Posts: 6399
Joined: August 16th, 2007, 5:36 am
Location: Alpha Centauri

Re: If you are bored with Deep Networks

July 7th, 2018, 12:09 am

It's a simple linear activation function, not rectified. The x range is [0, 8*pi), I just forgot to set it.
 
User avatar
Cuchulainn
Posts: 57050
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Re: If you are bored with Deep Networks

July 7th, 2018, 11:16 am

It's a simple linear activation function, not rectified. The x range is [0, 8*pi), I just forgot to set it.
I couldn't find this function in the list.
https://en.wikipedia.org/wiki/Activation_function
 
User avatar
katastrofa
Posts: 6399
Joined: August 16th, 2007, 5:36 am
Location: Alpha Centauri

Re: If you are bored with Deep Networks

July 7th, 2018, 1:53 pm

It's identity:  http://tflearn.org/activations/#linear

It basically finds a linear difference equation for a sine in this case.
 
User avatar
Cuchulainn
Posts: 57050
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Re: If you are bored with Deep Networks

July 8th, 2018, 1:28 pm

and sigmoid + SGD :-/
sigmoid_sgd.png
Image
 
User avatar
Cuchulainn
Posts: 57050
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Re: If you are bored with Deep Networks

July 13th, 2018, 6:22 am

E is stochastic (because mini-batches). Does your argument still apply?
It would seem that mini-batch are mainly an optimisation and not a functional requirement ("what you lose on the swings you gain on the roundabouts" aka Pyrrhic victory). 
Going to MB entails a new dynamical system in an enlarged state space.  Then the ODE system will probably become a random ODE with white noise or even some kind of Langevin/OU SDE. Then you are in the area of numerical solutions use cases..

I understand SGD 'plumbing' but I don't understand the problem it purports to solve. It is yet to be mathematically specified IMO. I see SGD as an example of gradient descent in a specific context. Maybe I am missing some vital info.
ABOUT WILMOTT

PW by JB

Wilmott.com has been "Serving the Quantitative Finance Community" since 2001. Continued...


JOBS BOARD

JOBS BOARD

Looking for a quant job, risk, algo trading,...? Browse jobs here...


GZIP: On