Any suggestions for books, papers, ppt slides, etc... on Stochastic Dynamic Control? Something to get my feet wet with, high-undergrad / low Ph.D level. God bless you for your kind heart,Hiboumalin

D. Bertsekas has great bookscheck out his web-site

Here you find internet address of Dynamic Programming and Stochastic Control of Stanford University and related stuff:http://www.stanford.edu/class/msande351/handouts.htm

The answer breaks down into whether you're considering linear systems or not:If so, simplest book is Stengel.http://www.princeton.edu/~stengel/OptConEst.htmlYou might also consider the recent book on estimation by Kailath which contains some parts on control -- a dual problem.For nonlinear systems, I'd recommend any of the books by Harold Kushner.For a good overview of the field -- a five minute read! -- seehttp://www.ams.org/bull/pre-1996-data/199431-1/davis.pdfHEY! Please lemme know if this helps! And if you can mention it, why you are interested -- as a former controls guy I am always on the lookout for connections to quant-fin.

I think you'll really like the 2 book series by Bertsekas, Dynamic Programmingand Optimal Control. It's ideal for a first year grad student and for the remainder of us who can't remember what we did in grad school.

Most importantly, it's being reviewd in Wilmott magazine in the near future

I read portions of Bertsekas' book, and did not find it to be anything great. There are much better treatments...

Handler,I also really like the books by Harold Kushner especially his treatment of the old Bellman equation usingBM, but they're likely too advanced and too specific (non-linear PDEs) for Hiboumalin.I believe he should get the broad treatment first.-newton

For the benefit of all (hopefully), lemme add this to my first post.If you're interested in controlling LINEAR differential eqs. (or difference eqs.) then the treatment need not use Ito calculus. etc. Instead basic EE-level material (matrix theory, differential eqs., delta-functions) suffices. This is easy stuff for any quant.If the system is nonlinear then we're already talking SDE-level treatments.- Handler PS to Newton, my impression was that Bertsekas covered discrete-time systems only. Am I wrong on this?

PS to Newton, my impression was that Bertsekas covered discrete-time systemsonly. Am I wrong on this? You are 100% correct. The BM assumption allows one (with some restrictions) tomodel continuous systems with non-linear PDEs. An understanding of StochasticAnalysis on Manifolds would be helpful but not necessary. If you're interestedin viscosity, Kushner is the man.

I'm also trying to learn some stochastic control theory, thanks everyone for these references.Do any of these books (or other papers/books) study nonlinear problems with states driven by Poisson noise as well as Brownian motion? I've found a few papers on the net, but nothing systematic.

Do any of these books (or other papers/books) study nonlinear problems with states driven by Poisson noise as well as Brownian motion? I've found a few papers on the net, but nothing systematic.This early tutorial is probably hard to obtain outside of an acedemic library but is really good. It does cover Poisson noise as well as brownian motion.Random Differential Equations In Control Theory, W.M. Wonham; In Probablisitic MEthods in Applied Mathematics, Ed. Bharucha-Reid, vol II, 1971 (!)Hey, can any of you people say -- even hinting or paper-linking is cool -- just how this is relevant to pricing/trading? Excuse the elementary-ness of the question; I see how the tools of stochastic control help therein, but not theory per se.

I see how the tools of stochastic control help therein, but not theory per se.I don't know in general, but I use this theory on control problems that require principal subspacetracking.

QuoteHey, can any of you people say -- even hinting or paper-linking is cool -- just <u>how</u> this is relevant to pricing/trading? Excuse the elementary-ness of the question; I see how the <u>tools</u> of stochastic control help therein, but not theory per se.If you see how to use the tools of stochastic control, then the theory seems to follow. In order (at least for me) to be very comfortable applying something non-trivial like stoch. control with a Poisson or more general jump process, I want to have a basic understanding of the conceptual and theoretical underpinnings. This can end up yielding a more reliable implementation of the ideas, since I have a better understanding of the limitations, plus might lead to a slicker and more general implementation of the tools.For my particular problem, I'm looking at some sort of portfolio/risk management problem where I'm sensitive to transaction costs, and the portfolio positions are getting perturbed randomly (by the Poisson process) that I can't directly control.Thanks for the reference, I'm going to go look it up.

Thanks all for your recommendations. Handler, I'm right now doing research on optimal dynamic portfolio allocation with Bernoulli-type (1 or 0) utility functions. Research is doing good so far, but I was feeling like reiventing the wheel each step of the way. I went to see my old and wise stoch calc professor and he told me that my problem was a dynamic stochastic control problem, and that I might benefit hugely from learning what it is and using already preexistant results instead of having to re-prove everything, hence my sudden interest in the topic. As for the relation to asset pricing, I haven't all the proofs ready yet, but it seems that in presence of 1/0-type utility function, the optimal investment strategy is to do just the opposite of what CAPM recommends. I don't want to talk too much about this right now since:- I'm probably wrong (no Popperian pun intended)- My proofs aren't complete- You don't give a sh^t anyways Thanks again,Hiboumalin.