Time for capsules!
Technical papers:
https://arxiv.org/abs/1710.09829
https://openreview.net/pdf?id=HJWLfGWRb
What is really impressive is that the new architecture is much more robust to adversarial examples - a bane of deep learning.
But the point about adversarial examples in Deep Learning is that you can find an adversarial example from about any starting point, not just a few near a "border". There are algorithms which do it (two are referenced in the capsule paper). The reason is not that deep networks are classifiers, but that the probability distribution they learn is too concentrated on the examples they see.Classifiers segment space, there will always be point close to the border that need just a very small nudge pass the border. That's actually the algorithm behind finding adversarial cases.
Indeed. All these things are well-known in the community.
Ah, but does it matter? If the goal is science, then "yes". If the goal is practical solutions, then "no".tl&dr: we don't understand how deep networks learn
This is not wrong. But it does not address the core issue.Ah, but does it matter? If the goal is science, then "yes". If the goal is practical solutions, then "no".tl&dr: we don't understand how deep networks learn
Humans seem perfectly comfortable using their brains despite having no clue how they work.
I sincerely hope that the AI community is able to reach beyond the level of QF magazines.A NN + a loss function is a high dimensional function. Back propagation is a supervised calibration method. Replacing BP with something else doesn't change these issues. These model are indeed empirically assessed, but is that any different from a table in a QF magazine illustrating the error in a small set of option prices for some numerical method?
I matters a lot! If you don't understand the mathematical foundations, you're groping around in the dark and progress is very slow. People who want to push AI forward are very keen on understanding the mathematical foundations of NN learning, because only then will we be able to create systems which can "learn to learn", and create AGI (artificial general intelligence).Ah, but does it matter? If the goal is science, then "yes". If the goal is practical solutions, then "no".tl&dr: we don't understand how deep networks learn
We do have some clues, and I wouldn't say that we are "perfectly comfortable" with the current state of knowledge about how ours (and other animals') brains work - we don't know how to treat depression and other mental disorders, we don't know how to optimally teach and train people, etc.Humans seem perfectly comfortable using their brains despite having no clue how they work.