Serving the Quantitative Finance Community

 
User avatar
outrun
Posts: 4573
Joined: January 1st, 1970, 12:00 am

Re: Machine learning for fun, to pay your rent or whatever

April 6th, 2018, 9:50 pm

 
User avatar
ISayMoo
Posts: 2332
Joined: September 30th, 2015, 8:30 pm

Re: Machine learning for fun, to pay your rent or whatever

April 6th, 2018, 10:36 pm

I'd suspect that the stochastic part of SGD lets it be quite robust to pathological gradients.
You might be right. But it does sound somewhat ad hoc.
One thing: there are many GDs to choose from (like a Swiss Army knife?)

https://arxiv.org/abs/1609.04747

Which one should be used?
Gauss-Newton, conjugate gradient, L-BFGS, Newton-Powell... Which one should be used?
 
User avatar
ISayMoo
Posts: 2332
Joined: September 30th, 2015, 8:30 pm

Re: Machine learning for fun, to pay your rent or whatever

April 6th, 2018, 10:36 pm

Does it?
I thought I was the one asking the questions.
That was, well, rude.
 
User avatar
katastrofa
Posts: 7440
Joined: August 16th, 2007, 5:36 am
Location: Alpha Centauri

Re: Machine learning for fun, to pay your rent or whatever

April 7th, 2018, 8:41 am

No one is rude here to you :-*
 
User avatar
Cuchulainn
Posts: 20254
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Machine learning for fun, to pay your rent or whatever

April 7th, 2018, 8:53 am

Does it?
I thought I was the one asking the questions.
That was, well, rude.
It was my Dutch/Irish directness. It's rude to answer a question with another question. Try it next time you give a talk to an audiemce.
Anyways, compare the train of thought to "run to completion" metaphor; finish a topic and them move on to the next one. A technique that fallen into disuse somewhat.

https://en.wikipedia.org/wiki/Run_to_co ... scheduling
Last edited by Cuchulainn on April 7th, 2018, 8:57 am, edited 2 times in total.
 
User avatar
Cuchulainn
Posts: 20254
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Machine learning for fun, to pay your rent or whatever

April 7th, 2018, 8:54 am

I'd suspect that the stochastic part of SGD lets it be quite robust to pathological gradients.
You might be right. But it does sound somewhat ad hoc.Image
One thing: there are many GDs to choose from (like a Swiss Army knife?)

https://arxiv.org/abs/1609.04747

Which one should be used?
Gauss-Newton, conjugate gradient, L-BFGS, Newton-Powell... Which one should be used?
All gradient-based. With all the usual caveats. In particular, the gradient of the activation function is an issue? e.g. take the Heaviside function as a counter-example.
 
User avatar
tw
Posts: 592
Joined: May 10th, 2002, 3:30 pm

Re: Machine learning for fun, to pay your rent or whatever

April 7th, 2018, 10:09 am

Does it?
I thought I was the one asking the questions.
That was, well, rude.
 
User avatar
Cuchulainn
Posts: 20254
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Machine learning for fun, to pay your rent or whatever

April 7th, 2018, 10:16 am

Image
 
User avatar
ISayMoo
Posts: 2332
Joined: September 30th, 2015, 8:30 pm

Re: Machine learning for fun, to pay your rent or whatever

April 7th, 2018, 10:18 am

I thought I was the one asking the questions.
That was, well, rude.
It was my Dutch/Irish directness. It's rude to answer a question with another question.
https://stuffjewishpeoplelike.wordpress ... -question/
 
User avatar
katastrofa
Posts: 7440
Joined: August 16th, 2007, 5:36 am
Location: Alpha Centauri

Re: Machine learning for fun, to pay your rent or whatever

April 7th, 2018, 10:49 am

I can certify that you're not Jewish.
 
User avatar
ISayMoo
Posts: 2332
Joined: September 30th, 2015, 8:30 pm

Re: Machine learning for fun, to pay your rent or whatever

April 7th, 2018, 10:50 am

But I like that aspect of their culture. Are Talmudic Jews rude?
 
User avatar
Cuchulainn
Posts: 20254
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Machine learning for fun, to pay your rent or whatever

April 7th, 2018, 6:09 pm

But I like that aspect of their culture. Are Talmudic Jews rude?
Ask katastrofa?
 
User avatar
Cuchulainn
Posts: 20254
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Machine learning for fun, to pay your rent or whatever

April 7th, 2018, 6:10 pm

Image
"When did you last see ML and DE together?"
 
User avatar
tags
Topic Author
Posts: 3162
Joined: February 21st, 2010, 12:58 pm

Re: Machine learning for fun, to pay your rent or whatever

April 11th, 2018, 7:29 pm

Introducing TensorFlow Probability

Probably Cuch wants to have a look at it.
 
User avatar
ppauper
Posts: 11729
Joined: November 15th, 2001, 1:29 pm

Re: Machine learning for fun, to pay your rent or whatever

April 11th, 2018, 8:18 pm

Image
"When did you last see ML and DE together?"
you think they're the same person?