QuoteOriginally posted by: CuchulainnQuoteI think Machine learning is harder than real analysis or measure theory which are both introductory topics at an graduate levelMeasure and real analysis are typically what maths students learn in 1st/2nd year *undergraduate* level. And MT is not so useful in daily work but it is popular in academia. You need real analysis because you can then analyse problems mathematically. A prerequisite for any kind of exact work in my opinion, especially when you develop new models. What I notice is that people can understand the conceptual ideas relating to a problem but they tend to have difficulty applying these ideas because they have either not been trained in analytical process (step-by-step) or they have not used it in a while.If you are good at analysis and have become 'fit' in this technique, learning other areas (e.g. ML) is *relatively* easy. Personally, it feels like getting fit for doing sport. I mostly agree with you here, except I don't understand what you mean by "being trained in analytical process (step-by-step)". And who? The applied people (engineers)or certain mathematicians that tend to be too much theoretically oriented?Of course measure theory and analysis are only basic foundations in applied probability and math finance, but they're essential to a certain extent.E.g., how can one understand the fundamental theorem without MT? The concept of a martingale? But right, you don't need MT day by day...Obviously, my contact to machine learners was a bit frustrating, that's why I was asking here what the (mathematically) demanding part of ML should be.Of course, SVMs, MLPs, Bayesian statistics etc could all be useful in the right place, but I think these techniques are applied just a bit too boldly withoutlooking at the structure of the problem at hand. MLers tend to call stochastic calculus, SDEs, probability theory and "real" statistics superfluous, because theirsuper-ultra-flexible algorithm can adapt to all the complicated structure they don't understand -- just my humble impression.

I have an impression that machine learning is mostly statistically based, with another line of terms to describe something that has existed in statistics for decades. For example, they call regression "supervised learning". Am I mistaken?Where is ML used most in industry?

QuoteOriginally posted by: BrusselatorQuoteOriginally posted by: CuchulainnQuoteI think Machine learning is harder than real analysis or measure theory which are both introductory topics at an graduate levelMeasure and real analysis are typically what maths students learn in 1st/2nd year *undergraduate* level. And MT is not so useful in daily work but it is popular in academia. You need real analysis because you can then analyse problems mathematically. A prerequisite for any kind of exact work in my opinion, especially when you develop new models. What I notice is that people can understand the conceptual ideas relating to a problem but they tend to have difficulty applying these ideas because they have either not been trained in analytical process (step-by-step) or they have not used it in a while.If you are good at analysis and have become 'fit' in this technique, learning other areas (e.g. ML) is *relatively* easy. Personally, it feels like getting fit for doing sport. I mostly agree with you here, except I don't understand what you mean by "being trained in analytical process (step-by-step)". And who? The applied people (engineers)or certain mathematicians that tend to be too much theoretically oriented?Of course measure theory and analysis are only basic foundations in applied probability and math finance, but they're essential to a certain extent.E.g., how can one understand the fundamental theorem without MT? The concept of a martingale? But right, you don't need MT day by day...Obviously, my contact to machine learners was a bit frustrating, that's why I was asking here what the (mathematically) demanding part of ML should be.Of course, SVMs, MLPs, Bayesian statistics etc could all be useful in the right place, but I think these techniques are applied just a bit too boldly withoutlooking at the structure of the problem at hand. MLers tend to call stochastic calculus, SDEs, probability theory and "real" statistics superfluous, because theirsuper-ultra-flexible algorithm can adapt to all the complicated structure they don't understand -- just my humble impression.Yea Measure is not needed for daily tasks but it gives you an added dimension of understanding to all things in applied math. I remember change of measure is directly used in things like convexity and time adjustments in finance though. I'm just saying machine learning is hard. In your list of courses, ML stands out as an upper lvl course that needs prerequisites while most others don't so be careful if you're not stat savy.

Quote Measure [...] gives you an added dimension of understanding to all things in applied mathI am not sure about this. I find MT neither sufficient nor necessary. In later years in pure maths people study stuff like Haar and this is a generalisation of the usual notion of measure. But it is not computational?QuoteI mostly agree with you here, except I don't understand what you mean by "being trained in analytical process (step-by-step)". And who? The applied people (engineers)or certain mathematicians that tend to be too much theoretically oriented?All my viewpoints are very subjective and tend to be based on very limited and focused knowledge. But in general I notice that many problem-solving activities reduce to mapping concepts to code. This can be learned. To take an exampple, defining a model -> solve in a (P)DE --> numerics --> algorithms --> code is what I mean, and the ability to work out all these steps.Typical quant profile? HTH

Please excuse my ignorance but could you give me some examples of where differential geometry comes up in machine learning? The reason I ask is because I know some differential geometry and am interested in machine learning but don't know much of the latter so I'd like to see where differential geometry is used.Thanks

QuoteOriginally posted by: snthPlease excuse my ignorance but could you give me some examples of where differential geometry comes up in machine learning? The reason I ask is because I know some differential geometry and am interested in machine learning but don't know much of the latter so I'd like to see where differential geometry is used.ThanksIt depends on the dept. and the focus, cs nlp oriented course won't use much geometry but a stat course will touch upon basic things like manifolds, natural gradients, banach/hilbert spaces. These are mostly applied to reinforcement learning and data mining.

QuoteOriginally posted by: BrusselatorQuoteOriginally posted by: twofish One reason why ML and FEM are not that often used is because they are very hard mathematically, but if your goal is to improve your math skills then that might be worth taking. If you do non-trivial things in ML or FEM, people may think to themselves, "we'll if he can handle FEM code, then the monte carlo solver we want him to babysit should be easy."What parts of ML are you thinking of that are hard mathematically? I had to deal with some machine learners in the past, and my impression is they come rather from a computer science background and often don't really understand the models they implement. Instead, they follow a rather crude try-and-error approach. They have little to no knowledge of statistical priciples and can deal with basic linear algebra and some very basic probabilistic topics. Nevertheless, I think that ML techniques could be useful now and then due to their flexible algorithmic component, but the mathematics involved is quite weak. Am I missing any point?Tons of EE people study machine learning, and at least at UIUC (my grad school), they're quite adept at mathematics. Specifically, it's quite common to find people doing machine learning who have gone through a measure theoretic probability sequence, and possibly beyond. I cannot speak for computer science types, but at least of the EEs doing machine learning that I know, they're no slouches when it comes to formal mathematics.

Also, measure theory is not an undergraduate topic, nor is it an undergraduate topic in the first year. There is often an advanced calculus course required of undergraduate math students which requires students to learn the usual epsilon-delta method of essentially deriving one-dimensional calculus, but rarely does it go into measure theory. Berkeley is a rare example, in which the course does go into Lebesgue (but not abstract/general) measure theory. It is the second course of an upper level (usually 4th year) sequence in real analysis. And from those I've talked to, the treatment isn't nearly as rigorous as that of the graduate sequence.At any rate, as others have mentioned, measure theory isn't useful for 98% of the topics that one comes across in a probabilistic setting AFAIK. It has more to do with mathematical maturity than anything else.

QuoteOriginally posted by: skibum1981Also, measure theory is not an undergraduate topic, nor is it an undergraduate topic in the first year. There is often an advanced calculus course required of undergraduate math students which requires students to learn the usual epsilon-delta method of essentially deriving one-dimensional calculus, but rarely does it go into measure theory. Berkeley is a rare example, in which the course does go into Lebesgue (but not abstract/general) measure theory. It is the second course of an upper level (usually 4th year) sequence in real analysis. And from those I've talked to, the treatment isn't nearly as rigorous as that of the graduate sequence.At any rate, as others have mentioned, measure theory isn't useful for 98% of the topics that one comes across in a probabilistic setting AFAIK. It has more to do with mathematical maturity than anything else.Well when you become a grad/phd student you have to relearn essentially everything by learning mt first, its always a required course for prob concentrations. It's impossible to state what a random variable is without going into mt and it eliminates the need to seperate discrete from continuous settings. Most good finance programs iv seen always have a class on stochastics and mt is a useful if not necessary prerequisite to work with sdes. But yea it's never in undergrad or maybe even in grad program. I could be wrong since I am still a student.There were actually 2 berkeley grads who's in my dept. and their understanding of general analysis techniques do seem to be a bit stronger.

QuoteOriginally posted by: nov1ceQuoteOriginally posted by: skibum1981Also, measure theory is not an undergraduate topic, nor is it an undergraduate topic in the first year. There is often an advanced calculus course required of undergraduate math students which requires students to learn the usual epsilon-delta method of essentially deriving one-dimensional calculus, but rarely does it go into measure theory. Berkeley is a rare example, in which the course does go into Lebesgue (but not abstract/general) measure theory. It is the second course of an upper level (usually 4th year) sequence in real analysis. And from those I've talked to, the treatment isn't nearly as rigorous as that of the graduate sequence.At any rate, as others have mentioned, measure theory isn't useful for 98% of the topics that one comes across in a probabilistic setting AFAIK. It has more to do with mathematical maturity than anything else.Well when you become a grad/phd student you have to relearn essentially everything by learning mt first, its always a required course for prob concentrations. It's impossible to state what a random variable is without going into mt and it eliminates the need to seperate discrete from continuous settings. Most good finance programs iv seen always have a class on stochastics and mt is a useful if not necessary prerequisite to work with sdes. But yea it's never in undergrad or maybe even in grad program. I could be wrong since I am still a student.There were actually 2 berkeley grads who's in my dept. and their understanding of general analysis techniques do seem to be a bit stronger.A lot of the developments in applied stochastic processes actually came from EE or physics (thinking Kalman and particle filters here), and such developments didn't require measure theory. I agree, at least for completeness sake, it's important stuff to know, and especially is very interesting, but it's not very useful from an applications standpoint. The first random processes course I took (designed by Bruce Hajek) was not measure theoretic, though it did make you aware of the issues of measure that exist. Proving dominated convergence theorems, etc., however.... that's another issue. I cannot speak for the extent it's necessary for stochastic SDEs. I do know that Wilmott's own courses don't touch measure theory with a ten foot pole, at least from what he said to me.As far as stating what a random variable is, this can be done after basic calculus. Out of curiosity, which program are you in? Do you enjoy it? What are you studying? Me: I did signal processing MS at UIUC and now looking for PhD programs, though I might stay at UIUC. Fantastic school, but boring location. Such is life.

QuoteOriginally posted by: skibum1981Also, measure theory is not an undergraduate topic, nor is it an undergraduate topic in the first year. There is often an advanced calculus course required of undergraduate math students which requires students to learn the usual epsilon-delta method of essentially deriving one-dimensional calculus, but rarely does it go into measure theory. Berkeley is a rare example, in which the course does go into Lebesgue (but not abstract/general) measure theory. It is the second course of an upper level (usually 4th year) sequence in real analysis. And from those I've talked to, the treatment isn't nearly as rigorous as that of the graduate sequence.At any rate, as others have mentioned, measure theory isn't useful for 98% of the topics that one comes across in a probabilistic setting AFAIK. It has more to do with mathematical maturity than anything else.TextTextSounds like Berkeley is not very demanding. I learnt Lebesgue measure theory in the first term of my second undergrad year in Hong Kong.

QuoteOriginally posted by: londonerSounds like Berkeley is not very demanding. I learnt Lebesgue measure theory in the first term of my second undergrad year in Hong Kong.As far as I understand, undergrad maths in US is pretty useless (e.g. the pure brain damage that is the Calculus sequence), all undergrad students who have a decent background take graduate classes instead.

Berkeley not demanding? You guys can't be serious...

I would REALLY love to get in berkeley for a phd in math.

Anyone would love to get into Berkeley for just about any of their graduate programs. It's just an outstanding institution, pure and simple.