Serving the Quantitative Finance Community

  • 1
  • 2
  • 3
  • 4
  • 5
  • 9
 
User avatar
Paul
Topic Author
Posts: 6604
Joined: July 20th, 2001, 3:28 pm

Financial Modelers' Manifesto

January 7th, 2009, 12:52 pm

The Financial Modelers' Manifestoby Emanuel Derman and Paul WilmottPrefaceA spectre is haunting Markets – the spectre of illiquidity, frozen credit, and the failure of financial models. Beginning with the 2007 collapse in subprime mortgages, financial markets have shifted to new regimes characterized by violent movements, epidemics of contagion from market to market, and almost unimaginable anomalies (who would have ever thought that swap spreads to Treasuries could go negative?). Familiar valuation models have become increasingly unreliable. Where is the risk manager that has not ascribed his losses to a once-in-a-century tsunami? To this end, we have assembled in New York City and written the following manifesto. ManifestoIn finance we study how to manage funds – from simple securities like dollars and yen, stocks and bonds to complex ones like futures and options, subprime CDOs and credit default swaps. We build financial models to estimate the fair value of securities, to estimate their risks and to show how those risks can be controlled. How can a model tell you the value of a security? And how did these models fail so badly in the case of the subprime CDO market? Physics, because of its astonishing success at predicting the future behavior of material objects from their present state, has inspired most financial modeling. Physicists study the world by repeating the same experiments over and over again to discover forces and their almost magical mathematical laws. Galileo dropped balls off the leaning tower, giant teams in Geneva collide protons on protons, over and over again. If a law is proposed and its predictions contradict experiments, it's back to the drawing board. The method works. The laws of atomic physics are accurate to more than ten decimal places. It's a different story with finance and economics, which are concerned with the mental world of monetary value. Financial theory has tried hard to emulate the style and elegance of physics in order to discover its own laws. But markets are made of people, who are influenced by events, by their ephemeral feelings about events and by their expectations of other people's feelings. The truth is that there are no fundamental laws in finance. And even if there were, there is no way to run repeatable experiments to verify them. You can hardly find a better example of confusedly elegant modeling than models of CDOs. The CDO research papers apply abstract probability theory to the price co-movements of thousands of mortgages. The relationships between so many mortgages can be vastly complex. The modelers, having built up their fantastical theory, need to make it useable; they resort to sweeping under the model's rug all unknown dynamics; with the dirt ignored, all that's left is a single number, called the default correlation. From the sublime to the elegantly ridiculous: all uncertainty is reduced to a single parameter that, when entered into the model by a trader, produces a CDO value. This over-reliance on probability and statistics is a severe limitation. Statistics is shallow description, quite unlike the deeper cause and effect of physics, and can’t easily capture the complex dynamics of default. Models are at bottom tools for approximate thinking; they serve to transform your intuition about the future into a price for a security today. It’s easier to think intuitively about future housing prices, default rates and default correlations than it is about CDO prices. CDO models turn your guess about future housing prices, mortgage default rates and a simplistic default correlation into the model’s output: a current CDO price. Our experience in the financial arena has taught us to be very humble in applying mathematics to markets, and to be extremely wary of ambitious theories, which are in the end trying to model human behavior. We like simplicity, but we like to remember that it is our models that are simple, not the world. Unfortunately, the teachers of finance haven’t learned these lessons. You have only to glance at business school textbooks on finance to discover stilts of mathematical axioms supporting a house of numbered theorems, lemmas and results. Who would think that the textbook is at bottom dealing with people and money? It should be obvious to anyone with common sense that every financial axiom is wrong, and that finance can never in its wildest dreams be Euclid. Different endeavors, as Aristotle wrote, require different degrees of precision. Finance is not one of the natural sciences, and its invisible worm is its dark secret love of mathematical elegance and too much exactitude. We do need models and mathematics – you cannot think about finance and economics without them – but one must never forget that models are not the world. Whenever we make a model of something involving human beings, we are trying to force the ugly stepsister’s foot into Cinderella’s pretty glass slipper. It doesn't fit without cutting off some essential parts. And in cutting off parts for the sake of beauty and precision, models inevitably mask the true risk rather than exposing it. The most important question about any financial model is how wrong it is likely to be, and how useful it is despite its assumptions. You must start with models and then overlay them with common sense and experience. Many academics imagine that one beautiful day we will find the ‘right’ model. But there is no right model, because the world changes in response to the ones we use. Progress in financial modeling is fleeting and temporary. Markets change and newer models become necessary. Simple clear models with explicit assumptions about small numbers of variables are therefore the best way to leverage your intuition without deluding yourself. All models sweep dirt under the rug. A good model makes the absence of the dirt visible. In this regard, we believe that the Black-Scholes model of options valuation, now often unjustly maligned, is a model for models; it is clear and robust. Clear, because it is based on true engineering; it tells you how to manufacture an option out of stocks and bonds and what that will cost you, under ideal dirt-free circumstances that it defines. Its method of valuation is analogous to figuring out the price of a can of fruit salad from the cost of fruit, sugar, labor and transportation. The world of markets doesn’t exactly match the ideal circumstances Black-Scholes requires, but the model is robust because it allows an intelligent trader to qualitatively adjust for those mismatches. You know what you are assuming when you use the model, and you know exactly what has been swept out of view. Building financial models is challenging and worthwhile: you need to combine the qualitative and the quantitative, imagination and observation, art and science, all in the service of finding approximate patterns in the behavior of markets and securities. The greatest danger is the age-old sin of idolatry. Financial markets are alive but a model, however beautiful, is an artifice. No matter how hard you try, you will not be able to breathe life into it. To confuse the model with the world is to embrace a future disaster driven by the belief that humans obey mathematical rules. MODELERS OF ALL MARKETS, UNITE! You have nothing to lose but your illusions.The Modelers' Hippocratic Oath~ I will remember that I didn't make the world, and it doesn't satisfy my equations.~ Though I will use models boldly to estimate value, I will not be overly impressed by mathematics.~ I will never sacrifice reality for elegance without explaining why I have done so.~ Nor will I give the people who use my model false comfort about its accuracy. Instead, I will make explicit its assumptions and oversights.~ I understand that my work may have enormous effects on society and the economy, many of them beyond my comprehension. Emanuel Derman and Paul WilmottJanuary 7 2009
 
User avatar
Cuchulainn
Posts: 20256
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Financial Modelers' Manifesto

January 7th, 2009, 12:59 pm

The part about mathematics is interesting; if the maths does not give the correct results, it's the fault of the 'upstream' models. Maths is always correct (by definition), but the models may not fit into it. I would add:don't use a model just because everyone else is using and has wriiten about it (check it out first). Example: Crank Nicolson.
Last edited by Cuchulainn on January 6th, 2009, 11:00 pm, edited 1 time in total.
 
User avatar
daveangel
Posts: 5
Joined: October 20th, 2003, 4:05 pm

Financial Modelers' Manifesto

January 7th, 2009, 1:16 pm

QuoteYou can hardly find a better example of confusedly elegant modeling than models of CDOs. The CDO research papers apply abstract probability theory to the price co-movements of thousands of mortgages. The relationships between so many mortgages can be vastly complex. The modelers, having built up their fantastical theory, need to make it useable; they resort to sweeping under the model's rug all unknown dynamics; with the dirt ignored, all that's left is a single number, called the default correlation. From the sublime to the elegantly ridiculous: all uncertainty is reduced to a single parameter that, when entered into the model by a trader, produces a CDO value. This over-reliance on probability and statistics is a severe limitation. Statistics is shallow description, quite unlike the deeper cause and effect of physics, and can’t easily capture the complex dynamics of default. I appreciate that the article may not have been aimed at a technical audience and also I am not a CDO expert but I always thought the price or value of a CDO is relatively easy to determine as its the assets held by the CDO and that correlation and the Gaussian copula only came in when one starts tranching the CDO ...
knowledge comes, wisdom lingers
 
User avatar
trackstar
Posts: 3420
Joined: January 1st, 1970, 12:00 am

Financial Modelers' Manifesto

January 7th, 2009, 2:34 pm

Outstanding! For those in doubt, check the original model.1848 Manifesto
Last edited by trackstar on January 6th, 2009, 11:00 pm, edited 1 time in total.
 
User avatar
Traden4Alpha
Posts: 3300
Joined: September 20th, 2002, 8:30 pm

Financial Modelers' Manifesto

January 7th, 2009, 3:38 pm

This should also be part of the "Oath of Office" for central bankers.
 
User avatar
Fermion
Posts: 2
Joined: November 14th, 2002, 8:50 pm

Financial Modelers' Manifesto

January 7th, 2009, 5:35 pm

QuoteOriginally posted by: PaulMany academics imagine that one beautiful day we will find the ‘right’ model. But there is no right model, because the world changes in response to the ones we use. Progress in financial modeling is fleeting and temporary. Markets change and newer models become necessary.I think you have made an unwarranted logical jump here. Just because progress is "fleeting and temporary" that does not mean that successive steps cannot converge onto a "right" model -- at least for market macrosopics. Changing "regimes" can, in principle, evolve from a single overall model.I'm not saying that such steps will necessarily converge, only that you have made an assumption of no convergence without making it clear that it is an assumption.QuoteSimple clear models with explicit assumptions about small numbers of variables are therefore the best way to leverage your intuition without deluding yourself.Amen to that! (Although, again, the "therefore" bit is inappropriate as there is no logical connection between this assertion -- which I agree with -- and the previous sentences.) However, my agreement does have a caveat. A convergent model may consist of a combination of several such simple models. The key is to break the problem down into the "right" pieces.
 
User avatar
piterdias
Posts: 0
Joined: July 14th, 2002, 3:00 am

Financial Modelers' Manifesto

January 7th, 2009, 6:13 pm

QuoteIt's a different story with finance and economics, which are concerned with the mental world of monetary value. Financial theory has tried hard to emulate the style and elegance of physics in order to discover its own laws. But markets are made of people, who are influenced by events, by their ephemeral feelings about events and by their expectations of other people's feelings. The truth is that there are no fundamental laws in finance. And even if there were, there is no way to run repeatable experiments to verify them. From my Engineering experience the main problem of financial model use is people doesn't understand that lack of experiment means lack of problem understanding.I can make an Engineering or Physics experiment where just some variables are changed, and improve the experiment to understand who two or more variables moving together change the final result.In Finance we have everything changing at the same (or delayed times) so we are not able to test some of hypothesis. Econometrics tries to help on this, but at the price of losing power of tests.It is an exciting job whose down side is being “guilty” by any unexpected financial loss.
 
User avatar
Fermion
Posts: 2
Joined: November 14th, 2002, 8:50 pm

Financial Modelers' Manifesto

January 7th, 2009, 6:28 pm

QuoteOriginally posted by: PaulThe truth is that there are no fundamental laws in finance. And even if there were, there is no way to run repeatable experiments to verify them. At risk of seeming pedantic. I'd question this. Government makes laws all the time. For example, if a company cannot meet its debts it must declare insolvency. Markets make rules too. Are you are asserting that we can't detect rules that govern behaviour that haven't been explicitly declared? That seems to me to defy logic. Those declared rules have consequences. Uncovering those consequences is what we are trying to do. You seem to be saying they don't exist. I doubt that is true.As regards repeatable experiments, I just do not understand why you say this. The key is to specify quantities that are expected not to change. Then every instance can be a repeated experiment. There is exactly the same problem in physics. The only reason we can repeat experiments on atoms is because we conjecture that the concept of an atom does not change with time. As long as our experiments hold up, we continue with the same conjecture. If there are no stationary concepts or quantities in finance then your statement would be true. But I doubt that this is the case. It is merely an assertion, stemming from our collective ignorance of what concepts and quantities are stationary --not a statement of fact.
Last edited by Fermion on January 6th, 2009, 11:00 pm, edited 1 time in total.
 
User avatar
Fermion
Posts: 2
Joined: November 14th, 2002, 8:50 pm

Financial Modelers' Manifesto

January 7th, 2009, 6:37 pm

QuoteOriginally posted by: piterdiasIn Finance we have everything changing at the same (or delayed times) so we are not able to test some of hypothesis. We can test any hypothesis that is assumed to be stationary.
 
User avatar
Fermion
Posts: 2
Joined: November 14th, 2002, 8:50 pm

Financial Modelers' Manifesto

January 7th, 2009, 6:57 pm

QuoteOriginally posted by: mackbarOutstanding! For those in doubt, check the original model.1848 ManifestoI am tempted to say I prefer the original. It does seem to discuss some fundamental laws that have passed the test of repeated experiments over 160 years....
 
User avatar
Traden4Alpha
Posts: 3300
Joined: September 20th, 2002, 8:30 pm

Financial Modelers' Manifesto

January 7th, 2009, 8:37 pm

QuoteOriginally posted by: FermionQuoteOriginally posted by: PaulMany academics imagine that one beautiful day we will find the ‘right’ model. But there is no right model, because the world changes in response to the ones we use. Progress in financial modeling is fleeting and temporary. Markets change and newer models become necessary.I think you have made an unwarranted logical jump here. Just because progress is "fleeting and temporary" that does not mean that successive steps cannot converge onto a "right" model -- at least for market macroscopics. Changing "regimes" can, in principle, evolve from a single overall model.I'm not saying that such steps will necessarily converge, only that you have made an assumption of no convergence without making it clear that it is an assumption.Paul is right on this one. There is a very good body of theory, evolutionary theory, that proves the transience of all convergences in systems such as the financial system. Saying that someday we will have (or approach) a ‘right’ model is like a biologist thinking that someday evolution will produce (or approach) the ‘right’ genetic sequence for an organism.At best, we might experience the low-volatility portion of a punctuated equilibrium (some would say that's what we had in recent years). But we can be sure that no matter how "right " a model may seem, there are actions, reactions, and unintended consequences to that model that will inevitably invalidate it (often catastrophically as we are seeing right now). In fact, the more right the model looks, the greater the danger.Even multiple-regime models will fail (assuming one can even create timely detectors for the current regime). Regime models in finance will, undoubtly, induce period-doubling phenomena and lead to chaotic behavior.The only constant in finance is the unbounded potential for complexity in the instruments, combinatorial portfolios, latent text strings for regulations, and the astronomical number of counterparty network configurations. Thus, no sensible risk manager (would that we had these!) should ever assume that their model is right because the ability of the risk manager to deconvolve the complexity of the system is vastly exceeded by the system's ability to generate new, complex, pathological, modes of behavior.
Last edited by Traden4Alpha on January 6th, 2009, 11:00 pm, edited 1 time in total.
 
User avatar
rwinston
Posts: 0
Joined: February 15th, 2007, 5:48 pm

Financial Modelers' Manifesto

January 7th, 2009, 9:28 pm

The great statistician George Box is quoted as saying: "All models are wrong, some are useful". It's amazing how often this truism is forgotten!
 
User avatar
Yossarian22
Posts: 4
Joined: March 15th, 2007, 2:27 am

Financial Modelers' Manifesto

January 7th, 2009, 11:05 pm

Benjamin Disreali - - One of the hardest things in this world is to admit you are wrong. And nothing is more helpful in resolving a situation than its frank admission.He is also know for saying (paraphrase) - There are three kinds of lies: lies, damned lies, and statistics!
 
User avatar
sheepski
Posts: 0
Joined: March 25th, 2008, 1:21 am

Financial Modelers' Manifesto

January 7th, 2009, 11:52 pm

If you can get the users of the financial models to take the same oath, we may get somewhere..
 
User avatar
Fermion
Posts: 2
Joined: November 14th, 2002, 8:50 pm

Financial Modelers' Manifesto

January 8th, 2009, 12:36 am

QuoteOriginally posted by: Traden4AlphaQuoteOriginally posted by: FermionQuoteOriginally posted by: PaulMany academics imagine that one beautiful day we will find the ‘right’ model. But there is no right model, because the world changes in response to the ones we use. Progress in financial modeling is fleeting and temporary. Markets change and newer models become necessary.I think you have made an unwarranted logical jump here. Just because progress is "fleeting and temporary" that does not mean that successive steps cannot converge onto a "right" model -- at least for market macroscopics. Changing "regimes" can, in principle, evolve from a single overall model.I'm not saying that such steps will necessarily converge, only that you have made an assumption of no convergence without making it clear that it is an assumption.Paul is right on this one. There is a very good body of theory, evolutionary theory, that proves the transience of all convergences in systems such as the financial system. Saying that someday we will have (or approach) a ‘right’ model is like a biologist thinking that someday evolution will produce (or approach) the ‘right’ genetic sequence for an organism.At best, we might experience the low-volatility portion of a punctuated equilibrium (some would say that's what we had in recent years). But we can be sure that no matter how "right " a model may seem, there are actions, reactions, and unintended consequences to that model that will inevitably invalidate it (often catastrophically as we are seeing right now). In fact, the more right the model looks, the greater the danger.I doubt that you can prove your assertion of inevitable invalidation. Actually it's easy to prove the contrary in a trivial but artificial example -- one in which finance is banned. No finance, no complexity. Model is simplest possible (i.e. it is nothing).A slightly more realistic example would be a planned economy with no markets. If the state is able to enforce it then finance is a stable centrally-defined system. (Of course, there is still the possibility of political instability, but that is not finance.)More generally, the comparison with evolution is inappropriate. For instance, unlike evolution, we can define the financial system we want. If we want stability then a combination of legislation, enforcement and theoretically optimal behaviour could in principle create it. It may be very difficult in practice. You may even be right that it turns out to be impossible. I'm just asking you to prove your assertion. I don't think you can.Of course, we can only be talking about an isolated closed system here -- just like physics does(!). There is no "equation of the universe". So it is perfectly reasonable that a stable system and models might not be static but will change as human needs do. But this doesn't mean we can't still continue to model them. If all you are saying is that human society will continue to evolve in ways we can't predict (at least not yet), then yes, I'll agree with you. But is that all you are saying? It seems to me that you are saying something much stronger.QuoteEven multiple-regime models will fail (assuming one can even create timely detectors for the current regime). Regime models in finance will, undoubtly, induce period-doubling phenomena and lead to chaotic behavior.Multiple regime models cannot reasonably be modelled by regime-switching criteria since they essentially represent a logical discontinuity -- if not a mathematical one. The change in regime must be inherent in the model itself. (I.e. "regimes" are an artifical concept. Useful perhaps but only from a qualitative viewpoint.)QuoteThe only constant in finance is the unbounded potential for complexity in the instruments, combinatorial portfolios, latent text strings for regulations, and the astronomical number of counterparty network configurations. Thus, no sensible risk manager (would that we had these!) should ever assume that their model is right because the ability of the risk manager to deconvolve the complexity of the system is vastly exceeded by the system's ability to generate new, complex, pathological, modes of behavior.Of course, in practice, with current levels of knowledge and current systems, you are right. But again, you are simply documenting the diffculty not the impossibility, given human enterprise.