SERVING THE QUANTITATIVE FINANCE COMMUNITY

Posts: 23951
Joined: September 20th, 2002, 8:30 pm

### Re: If you are bored with Deep Networks

The fundamental working hypothesis of AI is that intelligent behavior can be precisely described as symbol manipulation and can be modeled with the symbol processing capabilities of the computer.

??
The 1980s called and they want their hypothesis back.

ISayMoo
Topic Author
Posts: 2294
Joined: September 30th, 2015, 8:30 pm

### Re: If you are bored with Deep Networks

It's still true. Only we need to build this computer first.

Cuchulainn
Posts: 62106
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Re: If you are bored with Deep Networks

Does Tensorflow support multi-processor CPUs and can it be customised?

Cuchulainn
Posts: 62106
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Re: If you are bored with Deep Networks

Does Tensorflow support multi-processor CPUs and can it be customised? e.g. if you don't have GPUs or Cloud?
What's the speedup for this application? What's the maximum number of cores before performance degrades? I suppose it uses a fine-grained thread model with locking and pre-emption?

tagoma
Posts: 18354
Joined: February 21st, 2010, 12:58 pm

### Re: If you are bored with Deep Networks

with tf.device("/cpu:4"):
# ...

with tf.device("/cpu:7"):
# ...

with tf.device("/cpu:0"):
# ...

config = tf.ConfigProto(device_count={"CPU": 8},
sess = tf.Session(config=config)

Cuchulainn
Posts: 62106
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Re: If you are bored with Deep Networks

import time
def wait_on_b():
time.sleep(5)
print(b.result())  # b will never complete because it is waiting on a.
return 5

def wait_on_a():
time.sleep(5)
print(a.result())  # a will never complete because it is waiting on b.
return 6

a = executor.submit(wait_on_b)
b = executor.submit(wait_on_a)
I don't use Python(yet) but the issues are language-independent. Threads scale badly and anecdotal evidence shows that speedup in TensorFlow is bad with > 32 processors. Too fine-grained.

Python supports asynchronous futures which offer an alternative (C++11 has futures). The Scheduler determines what is pinned to what, not the application developer (sub-optimal).

https://docs.python.org/3/library/concu ... tures.html

mkoerner
Posts: 4
Joined: April 8th, 2009, 12:17 pm

### Re: If you are bored with Deep Networks

Does Tensorflow support multi-processor CPUs and can it be customised? e.g. if you don't have GPUs or Cloud?
What's the speedup for this application? What's the maximum number of cores before performance degrades? I suppose it uses a fine-grained thread model with locking and pre-emption?
As far as I know it supports multi-core CPUs on a number of levels, on a low level by dispatching calls to the MKL, and on a high level by running multiple sessions in parallel (FAQ, Does the runtime parallelize parts of graph execution?). Similar in pytorch where on a high-level the input data can be distributed along a dimension using DataParallelThis post on "Python Global Interpreter Lock (GIL) + numerical data processing" could also be informative if you are interested in Python.

Cuchulainn
Posts: 62106
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Re: If you are bored with Deep Networks

History of the tools

Logistic regression — 1958
Hidden Markov Model — 1960
Support Vector Machine — 1963
k-nearest neighbors — 1967
Artificial Neural Networks — 1975
Expectation Maximization — 1977
Decision tree — 1986
Q-learning — 1989
Random forest — 1995

Cuchulainn
Posts: 62106
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Re: If you are bored with Deep Networks

I find trying to grasp all these articles a bit painful. Lots of theory/theorems etc. but where is the example? e.g. that you can check against.

Posts: 23951
Joined: September 20th, 2002, 8:30 pm

### Re: If you are bored with Deep Networks

I find trying to grasp all these articles a bit painful. Lots of theory/theorems etc. but where is the example? e.g. that you can check against.

This does not sound right to me. My (limited) experience of math has been the opposite. Examples are mostly for applied math.

I wonder what percentage of academic math papers have examples?

Cuchulainn
Posts: 62106
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Re: If you are bored with Deep Networks

Maybe this example helps.(sections 3 and 5)
https://en.wikipedia.org/wiki/Metric_space

'Concrete' can be at many 'levels' e.,g. objects and types.
Just think about metrics for NNs, input representation and the mappings (hopefurrry bijective) between metric spaces etc.

Cuchulainn
Posts: 62106
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Re: If you are bored with Deep Networks

Some open research issues(??)
https://pdfs.semanticscholar.org/a2cf/2 ... 9e43dc.pdf

and some guidelines ..

https://www.ucl.ac.uk/~ucfamus/papers/i ... ions17.pdf

It would be useful for the august panel to give their feedback.

ISayMoo
Topic Author
Posts: 2294
Joined: September 30th, 2015, 8:30 pm

### Re: If you are bored with Deep Networks

The 1st article is very old (from 2007). A lot of the stuff in it is out of date or applies to things people are no longer really interested in.

The 2nd article (guidelines) is kind of "Captain Obvious speaking" paper. The author's right about everything (or maybe almost everything, but I didn't have the time to go nitpicking), but it's rather well-known stuff.

Cuchulainn
Posts: 62106
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Re: If you are bored with Deep Networks

The 1st article is very old (from 2007). A lot of the stuff in it is out of date or applies to things people are no longer really interested in.

The 2nd article (guidelines) is kind of "Captain Obvious speaking" paper. The author's right about everything (or maybe almost everything, but I didn't have the time to go nitpicking), but it's rather well-known stuff.
Fair enough.So things have progressed in leaps and bounds in the period 2007-2017?
2007 is not very old. BP is at least 50 years old.

Cuchulainn
Posts: 62106
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Re: If you are bored with Deep Networks

The recent spate of articles are great on description and narrative (the what) but fall short on explanation (the how), as Wittgenstein might say.

Reverse engineering is well-nigh impossible. It's all black box anno 2007 or has that been resolved, ISayMoo?

Does university education not teach how to write unambiguous algorithms?