It is easy to write down a probability distribution with infinite variance.But practically speaking, what tests can you run on a finite amount of data that would indicate that the underlying distribution has finite variance?Intuitively, I guess a distribution with infinite variance would look sort of lognormal for awhile and then there would be a big outlier, or maybe the size of the outliers ends up being inversely proportional to their frequency?I am trying to get some intuition here - anyone have nice, simple ways of thinking about this?Are there many natural processes that act as if they have infinite variance? Or maybe simple financial examples like Ponzi schemes?

student t with nu=2 has infinite variance, i believe..

QuoteOriginally posted by: erstwhileIt is easy to write down a probability distribution with infinite variance.But practically speaking, what tests can you run on a finite amount of data that would indicate that the underlying distribution has finite variance?Intuitively, I guess a distribution with infinite variance would look sort of lognormal for awhile and then there would be a big outlier, or maybe the size of the outliers ends up being inversely proportional to their frequency?I am trying to get some intuition here - anyone have nice, simple ways of thinking about this?Are there many natural processes that act as if they have infinite variance? Or maybe simple financial examples like Ponzi schemes?I don't have any answers to offer, but I love speculating on wild ideas so certain fairly obvious questions come to mind:1. How could a pdf that is conditional on it being a delta-function (vanishing variance) at some time in the past evolve to have infinite variance in a finite time? It's easy to construct a volatility function that blows up at some future time t1 (e.g. vol = t/|t-t1|) but what implications does this have that are relevant to any meaningful process? It's as though the goddess came along and said "from now on no market player has any knowledge of the past" so that no one has any basis to predict either expected value or variance, or any other moment. Is it at all reasonable to suppose that any moment could be singular anywhere on the real positive axis (for price or time)? 2. Is it reasonable to suppose that the pdf was ever a delta-function at some time in the past? (I.e. did we ever know the relevant "price" for that moment in time that accurately)?

I don't think you'd get `plausible' processes with continuous sample paths that blow up in finite time, unless you employ a scheme such as Fermion's. You can get infinite variance though, if you allow for jumps. Levy processes, which have the additional `nice' feature of being i.i.d. are an example. Think of a jump diffusion, where the jump distribution has infinite variance, e.g. the appropriate t-distribution. Then all finite variances would be finite.An indication of infinite variance would be an `increasing' sample variance as the sample increases, as new observations arrive.You can statistically test the infinite variance by estimating the parameters of such a process by ML, if you wish to parametrize it. Or you can estimate the tail index, which measures with what speed the tails of the distribution go to zero. If the tail index is q thenEabs(X)^p=infinite for all p>=q, and the corresponding moments would not exist.In the attached I have three graphs:* HillNorm shows the estimator for simulated normal data: The tail index goes to zero, since the tails go to zero exponentially fast, and all moments exist.* HillT2 shows the estimator for simulated t(2) data: The tail index goes to 2, since no moments p>=2 exist.* HillSP shows the estimator for a SP500 log-series.For more details you can see these lecture notes or PM me for the file.Kyriakos.

Even though it lead to ridicule from my peers, this is one of my favorite topics.There are several non-parametric tests that you can perform to detect infinite variance behavior (...in contrast to Hill estimators and the like for specific distributional models). These tests are typically based upon identifying power-law behavior in the distribution tails.A reasonable introduction to this topic may be found in Sornette, Critical Phenomena in Natural Sciences or in Voit, The Statistical Mechanics of Financial Markets.Hope this is helpful.-CC-

Fermion: I don't follow, sorry! I am imagining a time series that has fat tails - where is the delta function coming from?Rez: thanks for that, but i think my stats knowledge isn't up to fully understanding - i need to learn about the Hill estimators. also when you say "You can statistically test the infinite variance by estimating the parameters of such a process by ML", what are you referring to? Maybe i should pull out my trusty old extreme value theory book?ClosetChartist: Ridicule from your peers? Is that after setting up a Sornette "market-quake seismograph" or something similar? Someone at Merrill did that and it appeared to always produce random noise plus a few fake-out mini-signals, I seem to recall. But he was too senior for anyone to ridicule him!

"Market-quake seismograph", what a perfect description of Sornette's project! But no, I haven't shared that bit of sophistry with my peers. I don't confuse a posteriori description with a priori prediction.I work with contracts that have payoffs that are computed on and triggered by unhedgeable risks. Furthermore, it is very difficult to estimate the model or parameters for those unhedgeable risks. I argue, and can show from the data, that our conditional forecast distribution for these risks is effectively infinite variance. (Once you roll up model specificiation risk, parameter estimation risk, and normal process variation you wind up with a "honkin lot" of volatility!) My peers don't want to believe this, it's bad career publicity.-CC-

Does this book has anything relavent here? (sorry I don't have it in hand, flipped through quickly years ago)http://www.amazon.com/exec/obidos/ASIN/ ... 36-9782550

Good book for extreme value theory (viz. parameteric models). However, the primary focus of the book is on static distributions rather than stochastic processes.A good book for infinite variance stochastic processes is Samorodnidtsky and Taqqu, Stable Non-Gaussian Random Processes. Although there are some very practical estimation techniques in this book, the material is most definitely not for the mathematically faint of heart.-CC-

QuoteOriginally posted by: erstwhileIt is easy to write down a probability distribution with infinite variance.Are there many natural processes that act as if they have infinite variance?Yes. Many phase transitions (water to steam, for example) have what are called critical pointsat certain combinations of temp/pressure. Generally, away from critical points, mostcorrelation-type functions have exponential decay, but these can become weakerpower laws decays at the critical points. Second moments are things like the specificheat, and these become infinite (in infinite systems) at the critical points.It's pretty common physical phenomena, also seen in magnets and lots of field theories generally.regards,alan p.s. some classic papers:http://hrst.mit.edu/hrs/renormalization ... o/#Fluidsa second moment (in a simulated, finite size system)

QuoteOriginally posted by: erstwhileFermion: I don't follow, sorry! I am imagining a time series that has fat tails - where is the delta function coming from?Pdfs aren't arbitrary at arbitrary times! We normally think of them as having developed from a previous state, fat tails and all. They are therefore (theoretically at least) conditional on a previous pdf. Every time we build a diffusion tree (say), starting with a known price at a known time, then we start with a delta-function. So, unless we invoke singularities along the way (or, more realistically, admit that we don't know the price at any time with exact precision) it's going to be difficult to go from a vanishing variance to an infinite variance in a finite time. You posit a pdf with an infinite variance, but without specifying any conditions on time evolution. An infinite variance at the current time implies that our knowledge of the current price is a fiction. Is that reasonable? Rez, has supplied an example where one can get jumps with an infinite variance -- which can take a finite variance to an infinite variance in one time step -- but is that a reasonable possibility?Am I missing the point?

Distributions of independent random variables always have finite variance (finite second moment).Otherwise variance is undefined (that is there is no second moment). You can think you're measuring or defining something with infinite variance, but you should think again. (What properties must your manifold have for variance to exist???)N

I am not putting any of my Kiwi cents on the stuff below; perhaps the Levy gurus can tell me if it is a load of rubbish.Levy processes (that are i.i.d. and might exhibit infinite variance) can be constructed by subordinating a Brownian motion.For example (a special case of) the inverse Gaussian subordinator is defined as the time when a Brownian motion B1 hits a lineary increasing barrier:T(t) = inf( s : B1(s) = a t)The subordinator is routinely translated as the `trading activity'. If a=sqrt(2)/2 I believe that this would give the Levy subordinator.Starting with a second Brownian motion B2, and using T(t) as the time change, we end up with a processX(t) = B2(T(t))We travel `faster' or `slower' through B2 according to the behaviour of B1.I believe that this process has infinite (undefined) second moment, and the process described above shows its construction (starting from a delta function).K.

AlanWhat does a phase transition look like in a market? What sort of market conditions (analogous to temperature/pressure) could bring it about?

I'm not a believer in that kind of finance. You'll have to ask the 'econophysics' guys regards,