@outrun thanks.

I'm not sure where this thread is heading. Still,* there was that article J.P.Morgan’s massive guide to machine learning and big data jobs in finance some weeks ago with some interesting statements.

E.g. "You won’t need to be a machine learning expert, you will need to be an excellent quant and an excellent programmer"

* My first comma in a while

Whether "deep learning" is being used in business or sought by employers is an empirical question. An analysis of resumes on LinkedIn and job postings in various internet sites would provide an answer.

Linkedin would not be my first port of call as a source of objective data. 
4. An army of people will be needed to acquire, clean, and assess the data 

From NYT on AI

https://www.nytimes.com/2017/07/29/opin ... ght-region

The WSJ view

Is there an alternative to gradient descent method? It's not that great, at least not always..

What gradient descent algorithm are you talking about, and what problem are you applying it to? This reads to me like "a car isn't that great".

What gradient descent algorithm are you  talking about, and what problem are you applying it to? This reads to me like "a car isn't that great".

Any of them. I'm reading up the links posted here and there, NYT, WSJ and my own experience with numerical analysis. In other areas gradient descent is not the only kid on the block.

Here's a random sample from WSJ

Even the trendy technique of “deep learning,” which uses artificial neural networks to discern complex statistical correlations in huge amounts of data, often comes up short. Some of the best image-recognition systems, for example, can successfully distinguish dog breeds, yet remain capable of major blunders, like mistaking a simple pattern of yellow and black stripes for a school bus. Such systems can neither comprehend what is going on in complex visual scenes (“Who is chasing whom and why?”) nor follow simple instructions (“Read this story and summarize what it means”).

Anyhoo, one question above is: is there an alternative to gradient descent? The answer is very simple: it is Yes or No.

That pointed out problem had a different cause than gradient descent. People have that too: things like optical illusions, is the dress blue or gold?

For a lot of problems you can't even use gradient descent.

OpenAI is a new, research platform funded by some famous people (Musk, Thiel,..) to research important open problems in general AI. They wrote a blog about this topic not too long ago (probably where WJ picked it up)?

https://blog.openai.com/adversarial-example-research/

There are also quite a few papers on arxiv. A good search key is "adversarial examples deep learning"

..and the humans equivalent!

https://en.m.wikipedia.org/wiki/The_dress

Cool overview of things we can expect in the next couple of years:

What gradient descent algorithm are you  talking about, and what problem are you applying it to? This reads to me like "a car isn't that great".

Any of them. I'm reading up the links posted here and there, NYT, WSJ and my own experience with numerical analysis. In other areas gradient descent is not the only kid on the block.

Here's a random sample from WSJ

Even the trendy technique of “deep learning,” which uses artificial neural networks to discern complex statistical correlations in huge amounts of data, often comes up short. Some of the best image-recognition systems, for example, can successfully distinguish dog breeds, yet remain capable of major blunders, like mistaking a simple pattern of yellow and black stripes for a school bus. Such systems can neither comprehend what is going on in complex visual scenes (“Who is chasing whom and why?”) nor follow simple instructions (“Read this story and summarize what it means”).

Anyhoo, one question above is: is there an alternative to gradient descent? The answer is very simple: it is Yes or No.

Absolutely yes.

This link (https://stats.stackexchange.com/questio ... ropagation ) mentions some of the derivative-free optimization algorithms and cites a comparison article. Another article discussing alternatives of gradient descent specifically in a neural net context is http://courses.mai.liu.se/FU/MAI0083/Re ... ikanfs.pdf .

And as outrun mentioned the example failures mentioned in the article may have nothing to do with gradient descent and more to do with the total number of examples given to the deep learning system and the limits of those specific deep learning instances due to current CPU & RAM performance. How well can a chimpanzee perform on this test? Or a mouse? I'd wager a fair percentage of humans would fail the “Who is chasing whom and why?” test and standardized test data certainly prove that most people do poorly on the “Read this story and summarize what it means” test.

The reason that people pay attention to these adversarial examples is because of *security*. These example are like computer virus, specifically tuned and tailored through inspecting a specific neural network in order to fool it.

The examples are mostly on classification tasks: is this a stop sign or not? There also some remedies being researched but the main concern is a car that gets hacked into taking a turn, a drone that gets fooled into attacking the wrong people.  These adversarial examples won't pop up under normal conditions, but a nefarious person could potentially hack a neural network into illusions. 

E.g. suppose you succeed into training a NN to classify these images into either a chihuahua or a muffin, and it has 99% accuracy. What will it do when presented with a picture of a Kiwi? Whatever it should do, it will very likely do a very bad job. It has learned to look at subtle textures, number of black dots and their relative positions. But a Kiwi lives in a tangent dimension, al the way up there with the shrew. 

... I'm currently studying HF bots, see if I can manipulate them into 'positive' behaviour. Those bot's aren't very clever.

Does ML use TDA?

https://en.wikipedia.org/wiki/Topological_data_analysis