Serving the Quantitative Finance Community

 
User avatar
Cuchulainn
Posts: 20254
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Hacking ML for fun & profit

February 21st, 2019, 4:54 pm

The central problem with the present ‘machine learning’ and ‘big data’ hype is that so many think that they can get away with analysing real-world phenomena without any (commitment to) theory. But — data never speaks for itself. Without a prior statistical set-up, there actually are no data at all to process. And — using a machine learning algorithm will only produce what you are looking for.
Machine learning algorithms always express a view of what constitutes a pattern or regularity. They are never theory-neutral.
Clever data-mining tricks are not enough to answer important scientific questions. Theory matters.

Image

Do you agree?
 
User avatar
ISayMoo
Topic Author
Posts: 2332
Joined: September 30th, 2015, 8:30 pm

Re: Hacking ML for fun & profit

February 21st, 2019, 6:58 pm

Sure!
 
User avatar
Cuchulainn
Posts: 20254
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Hacking ML for fun & profit

February 21st, 2019, 7:36 pm

Sure!
I am the messenger. I have no idea what they are claiming.
 
User avatar
ISayMoo
Topic Author
Posts: 2332
Joined: September 30th, 2015, 8:30 pm

Re: Hacking ML for fun & profit

February 22nd, 2019, 9:37 am

You need a good inductive bias for a learning system to generalise well.
 
User avatar
Cuchulainn
Posts: 20254
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Hacking ML for fun & profit

March 6th, 2019, 11:12 pm

About 40% of Europe’s “AI companies” don’t use any AI at all

https://www.technologyreview.com/the-do ... ai-at-all/?
 
User avatar
Cuchulainn
Posts: 20254
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Hacking ML for fun & profit

March 12th, 2019, 10:31 am

Random question: why are most NN problems initially formulated and solved in discrete time?
Why not address continuous time models (e.g. CTRNN) and take it from there? And ODEs have been around for > 100 years!
 
User avatar
ISayMoo
Topic Author
Posts: 2332
Joined: September 30th, 2015, 8:30 pm

Re: Hacking ML for fun & profit

March 12th, 2019, 11:44 am

Because computer programs usually work in discrete time, I think. 

And in non-RL setting (e.g. image classification) there often isn't any notion of "time" at all. Modelling sequences (language or voice data) has discrete time (either language tokens / characters or audio sampling frequency).
 
User avatar
Cuchulainn
Posts: 20254
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Hacking ML for fun & profit

March 12th, 2019, 11:59 am

1. Because computer programs usually work in discrete time, I think. 

2. And in non-RL setting (e.g. image classification) there often isn't any notion of "time" at all. Modelling sequences (language or voice data) has discrete time (either language tokens / characters or audio sampling frequency).
1. yes, but that is the solution to a  problem without stating what the problem is. ML is skipping a step, like presenting a FD scheme without saying what the PDE is. Incomplete. You want to look at qualitative properties before jumping into numerics??
Computer programs are created by computer programmers. Some do ODE, some are most discrete.
2. We can take artificial time (e.g. adding to elliptic PDE) and/or let dt go to zero? And this is the use case of interest.

So, no size fits all problems?
 
User avatar
katastrofa
Posts: 7440
Joined: August 16th, 2007, 5:36 am
Location: Alpha Centauri

Re: Hacking ML for fun & profit

March 12th, 2019, 12:45 pm

I thought it was because it was not the problem and model motivating the use of a technique (NNs), but the other way around - the technique being accidentally used to solve the problem. (I'm not criticising that approach, if someone has resources and enough domain knowledge not to kill anyone with such blind shots.) I've never understood why NNs are said to imitate a human brain, though - cognitive processes are continuous, AFAIK.

ISayMoo, data may come in ordinal-indexed sequences, but their (real-valued) timestamp domain is continuous. What am I not getting?
 
User avatar
Cuchulainn
Posts: 20254
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Hacking ML for fun & profit

March 12th, 2019, 2:27 pm

I thought it was because it was not the problem and model motivating the use of a technique (NNs), but the other way around - the technique being accidentally used to solve the problem. (I'm not criticising that approach, if someone has resources and enough domain knowledge not to kill anyone with such blind shots.) I've never understood why NNs are said to imitate a human brain, though - cognitive processes are continuous, AFAIK.

ISayMoo, data may come in ordinal-indexed sequences, but their (real-valued) timestamp domain is continuous. What am I not getting?
I agree. It seems this was the first article on GD.

https://web.stanford.edu/class/psych209 ... apter8.pdf

I cannot look into authors' minds to discover how they got this brainwave. It might be possible to read between the lines of the text.
 
User avatar
Cuchulainn
Posts: 20254
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Hacking ML for fun & profit

March 12th, 2019, 3:57 pm

 I've never understood why NNs are said to imitate a human brain

Maybe AI scientists and the media should stop pretending that NN technology is only a stones' throw away from how the brain functions.
 
User avatar
ISayMoo
Topic Author
Posts: 2332
Joined: September 30th, 2015, 8:30 pm

Re: Hacking ML for fun & profit

March 12th, 2019, 9:19 pm

1. Because computer programs usually work in discrete time, I think. 

2. And in non-RL setting (e.g. image classification) there often isn't any notion of "time" at all. Modelling sequences (language or voice data) has discrete time (either language tokens / characters or audio sampling frequency).
1. yes, but that is the solution to a  problem without stating what the problem is. ML is skipping a step, like presenting a FD scheme without saying what the PDE is. Incomplete. You want to look at qualitative properties before jumping into numerics??
The problem is minimising the model loss on the training set. It's called Empirical Risk Minimisation learning.
2. We can take artificial time (e.g. adding to elliptic PDE) and/or let dt go to zero?
What for? Why force a PDE into a problem in which it wasn't present in the first place? Why add complexity?
 
User avatar
ISayMoo
Topic Author
Posts: 2332
Joined: September 30th, 2015, 8:30 pm

Re: Hacking ML for fun & profit

March 12th, 2019, 9:23 pm

I thought it was because it was not the problem and model motivating the use of a technique (NNs), but the other way around - the technique being accidentally used to solve the problem. (I'm not criticising that approach, if someone has resources and enough domain knowledge not to kill anyone with such blind shots.) I've never understood why NNs are said to imitate a human brain, though - cognitive processes are continuous, AFAIK.
I don't think anyone sane is saying that NNs imitate a human brain. They were inspired by some aspects of how biological brains work. So was reinforcement learning. But that's not "imitation".
ISayMoo, data may come in ordinal-indexed sequences, but their (real-valued) timestamp domain is continuous. What am I not getting?
What real-valued timestamps do you have when feeding a sentence into Google Translate? It's just a string of letters.
When processing digital recordings, timestamps are regular. You can discretise them.
 
User avatar
Cuchulainn
Posts: 20254
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Hacking ML for fun & profit

March 12th, 2019, 9:28 pm

You need to be conversant with numerical analysis to understand 2.
2. Parabolic is sometimes easier than elliptic PDEs (e.g. no elliptic solver like SSOR, Jacobi available, or too slow (they suck)).

AFAIK @Alan does this in NDSolve. The steady-state solution (T yuge) _is_ solution of elliptic PDE.

Just like this

http://eqworld.ipmnet.ru/en/board/schie ... ts_CES.pdf
 
User avatar
ISayMoo
Topic Author
Posts: 2332
Joined: September 30th, 2015, 8:30 pm

Re: Hacking ML for fun & profit

March 12th, 2019, 9:59 pm

OK, maybe I'm too stupid to get this.