Serving the Quantitative Finance Community

 
User avatar
dunrewpp
Topic Author
Posts: 0
Joined: April 22nd, 2009, 3:36 am

Rare Event

October 29th, 2010, 6:10 am

It takes on average a very large, but unknown, number of days -- say, N days -- for an extremely rare event to occur. Find the probability that the event occurs only after the first N days.
 
User avatar
Teodor
Posts: 0
Joined: September 29th, 2008, 8:01 pm

Rare Event

October 31st, 2010, 1:08 pm

Looks like it...Assume the probability of the event is p.Then E(days) = p + (1-p)(1+E(days)) => E(days) = 1/p.We have E(days) = N thus p = 1/N.P(Only after N days) = (1-p)^N (I assume that this is what means by "only after N days")Thus (1-1/N)^N -> 1/e with N->\infty.
Last edited by Teodor on October 30th, 2010, 10:00 pm, edited 1 time in total.
 
User avatar
Landscape
Posts: 0
Joined: July 2nd, 2010, 11:13 am

Rare Event

December 20th, 2010, 2:32 am

"Rare event occurs" = X ~ Bin(N,p), N*p = 1, P(X=0) = (1-1/N)^N ~ 1/e for large N
Last edited by Landscape on December 19th, 2010, 11:00 pm, edited 1 time in total.
 
User avatar
anilmag
Posts: 0
Joined: May 13th, 2010, 11:30 am

Rare Event

April 12th, 2011, 1:14 pm

Can be solved using exponential distribution as well.CDF of Exponential Distribution is P(x<X) = the probability asked in question is 1-P(x<N) with Therefore 1-P(x<N)=1-(1-1/e)=1/e edit: there was a typo
Last edited by anilmag on April 11th, 2011, 10:00 pm, edited 1 time in total.
 
User avatar
Traden4Alpha
Posts: 3300
Joined: September 20th, 2002, 8:30 pm

Rare Event

April 12th, 2011, 1:27 pm

This question makes assumptions about the underlying process. For IID processes, the 1/e answer is right. But other processes have other answers.For example, February 29th is a rare event but it occurs with almost 100% probability every 1461 days (if you include all the rules of leap years, the probability of occurrence by the average N would be near 0). On the other hand, extreme floods and other correlated volatility processes tend to show a bimodal distribution of event-to-event occurrences so the probability of an event in the first N days may be greater than 50%.
 
User avatar
dunrewpp
Topic Author
Posts: 0
Joined: April 22nd, 2009, 3:36 am

Rare Event

April 16th, 2011, 7:50 am

Suppose you fall into a very long and deep slumber and next time you wake up, you do so on some random day (of the year). When you are alert enough to understand a word, you are told that a rare day is awaiting you with 100% certainty. You are not told what that rare day is; only that when it does befall you, you will be notified so. Even in this scenario, the probability, as seen from your perspective, that the rare day will befall you after the average number of days of waiting for it is 1/e. The only issue here is the notion of rareness as applied to the number of days in our earthly year. I'd rather think of rareness in the order of 1,000,000 or more. Of course, rareness is all relative.QuoteOriginally posted by: Traden4AlphaThis question makes assumptions about the underlying process. For IID processes, the 1/e answer is right. But other processes have other answers.For example, February 29th is a rare event but it occurs with almost 100% probability every 1461 days (if you include all the rules of leap years, the probability of occurrence by the average N would be near 0). On the other hand, extreme floods and other correlated volatility processes tend to show a bimodal distribution of event-to-event occurrences so the probability of an event in the first N days may be greater than 50%.
 
User avatar
Traden4Alpha
Posts: 3300
Joined: September 20th, 2002, 8:30 pm

Rare Event

April 16th, 2011, 1:11 pm

QuoteOriginally posted by: dunrewppSuppose you fall into a very long and deep slumber and next time you wake up, you do so on some random day (of the year). When you are alert enough to understand a word, you are told that a rare day is awaiting you with 100% certainty. You are not told what that rare day is; only that when it does befall you, you will be notified so. Even in this scenario, the probability, as seen from your perspective, that the rare day will befall you after the average number of days of waiting for it is 1/e. The only issue here is the notion of rareness as applied to the number of days in our earthly year. I'd rather think of rareness in the order of 1,000,000 or more. Of course, rareness is all relative.QuoteOriginally posted by: Traden4AlphaThis question makes assumptions about the underlying process. For IID processes, the 1/e answer is right. But other processes have other answers.For example, February 29th is a rare event but it occurs with almost 100% probability every 1461 days (if you include all the rules of leap years, the probability of occurrence by the average N would be near 0). On the other hand, extreme floods and other correlated volatility processes tend to show a bimodal distribution of event-to-event occurrences so the probability of an event in the first N days may be greater than 50%.Maybe, maybe not. There remain at least two exceptions to the 1/e rule in the scenario you pose, one is highly significant and the other is more dismissible.First, suppose the "rare event" is a "large earthquake" in particular part of the world. Lets assume these occur about every 200 years because that's how long it takes for sufficient strain to build in the fault zone to create a large earthquake. At first glance we have N=200. In such a scenario, with a random phase of awakening, the probability of observing a "large earthquake" in the next 200 years is essentially 100%.But lets also suppose that this once-every-200-year megaquake spawns major aftershocks in the ensuing days after the big one and that one of these aftershocks is big enough to also count as a "large earthquake." Now, we have an average number of years between rare events of N=100 and a probability of observing one in the next randomly picked 100 year interval of only about 50%.Second, suppose the first major earthquake is what wakes you from your slumber. Now your chance of observing another "large earthquake" in the next 100 years is nearly 100% because you are almost certain to experience the ensuing large aftershock. You may rightfully quibble that the original scenario implied a random awakening and yet the case of correlated awakening must be considered because it happens in real life. One "rare event" triggers vigilance which makes one realize the rare event (e.g. a mini flash crash or threat of sovereign default) is not as rare as one thought.My points remain that: the process that generates "rare events" matters and the process that generates vigilance of the rare event matters, too. Only in the case of IID do we get the 1/e result.
 
User avatar
dunrewpp
Topic Author
Posts: 0
Joined: April 22nd, 2009, 3:36 am

Rare Event

April 16th, 2011, 7:20 pm

Your discussion expands my original problem. Of course the answer to a probability question depends on how much you know about a given situation. A particular perspective gives a corresponding probability answer for the same event. Different people will arrive at different probabilities for the same event depending on how much each knows about the situation probably leading to the event. In my original problem, there was little information given and that's all you've got to go by. With just that, what's your answer?In fact it's very easy to take your line of analysis and apply it to just any probability problem. All you need to do is just creatively spin words around possibilities that somehow might have been present in the probability problem but not accounted for.Here's an example of creatively spinning words around possibilities ....Suppose there is a box and inside it are only two balls of the same size, weight, and other physical characteristics, but one is red, the other blue. We shake the box wildly until one ball falls out. What is the probability that the red one falls out?We have had problems of this sort all the while. So, now let's spin words creatively: Well, we have not accounted for the possibility that the red ball might be glued to the bottom of the box and that no amount of violent shaking of the box would release it from the box. So, the answer to the question is 0, and not 1/2.QuoteOriginally posted by: Traden4AlphaQuoteOriginally posted by: dunrewppSuppose you fall into a very long and deep slumber and next time you wake up, you do so on some random day (of the year). When you are alert enough to understand a word, you are told that a rare day is awaiting you with 100% certainty. You are not told what that rare day is; only that when it does befall you, you will be notified so. Even in this scenario, the probability, as seen from your perspective, that the rare day will befall you after the average number of days of waiting for it is 1/e. The only issue here is the notion of rareness as applied to the number of days in our earthly year. I'd rather think of rareness in the order of 1,000,000 or more. Of course, rareness is all relative.QuoteOriginally posted by: Traden4AlphaThis question makes assumptions about the underlying process. For IID processes, the 1/e answer is right. But other processes have other answers.For example, February 29th is a rare event but it occurs with almost 100% probability every 1461 days (if you include all the rules of leap years, the probability of occurrence by the average N would be near 0). On the other hand, extreme floods and other correlated volatility processes tend to show a bimodal distribution of event-to-event occurrences so the probability of an event in the first N days may be greater than 50%.Maybe, maybe not. There remain at least two exceptions to the 1/e rule in the scenario you pose, one is highly significant and the other is more dismissible.First, suppose the "rare event" is a "large earthquake" in particular part of the world. Lets assume these occur about every 200 years because that's how long it takes for sufficient strain to build in the fault zone to create a large earthquake. At first glance we have N=200. In such a scenario, with a random phase of awakening, the probability of observing a "large earthquake" in the next 200 years is essentially 100%.But lets also suppose that this once-every-200-year megaquake spawns major aftershocks in the ensuing days after the big one and that one of these aftershocks is big enough to also count as a "large earthquake." Now, we have an average number of years between rare events of N=100 and a probability of observing one in the next randomly picked 100 year interval of only about 50%.Second, suppose the first major earthquake is what wakes you from your slumber. Now your chance of observing another "large earthquake" in the next 100 years is nearly 100% because you are almost certain to experience the ensuing large aftershock. You may rightfully quibble that the original scenario implied a random awakening and yet the case of correlated awakening must be considered because it happens in real life. One "rare event" triggers vigilance which makes one realize the rare event (e.g. a mini flash crash or threat of sovereign default) is not as rare as one thought.My points remain that: the process that generates "rare events" matters and the process that generates vigilance of the rare event matters, too. Only in the case of IID do we get the 1/e result.
 
User avatar
Traden4Alpha
Posts: 3300
Joined: September 20th, 2002, 8:30 pm

Rare Event

April 16th, 2011, 9:31 pm

QuoteOriginally posted by: dunrewppYour discussion expands my original problem. Of course the answer to a probability question depends on how much you know about a given situation. A particular perspective gives a corresponding probability answer for the same event. Different people will arrive at different probabilities for the same event depending on how much each knows about the situation probably leading to the event. In my original problem, there was little information given and that's all you've got to go by. With just that, what's your answer?In fact it's very easy to take your line of analysis and apply it to just any probability problem. All you need to do is just creatively spin words around possibilities that somehow might have been present in the probability problem but not accounted for.Here's an example of creatively spinning words around possibilities ....Suppose there is a box and inside it are only two balls of the same size, weight, and other physical characteristics, but one is red, the other blue. We shake the box wildly until one ball falls out. What is the probability that the red one falls out?We have had problems of this sort all the while. So, now let's spin words creatively: Well, we have not accounted for the possibility that the red ball might be glued to the bottom of the box and that no amount of violent shaking of the box would release it from the box. So, the answer to the question is 0, and not 1/2.QuoteOriginally posted by: Traden4AlphaThis question makes assumptions about the underlying process. For IID processes, the 1/e answer is right. But other processes have other answers................................My points remain that: the process that generates "rare events" matters and the process that generates vigilance of the rare event matters, too. Only in the case of IID do we get the 1/e result.I admit that I expanded the problem and yet any answer to a probability-style brainteaser must expand the problem by bringing in base-level knowledge of "rare events", notions of probability, and time-based event generator processes, etc.In the context of undergraduate probability problems, IID is the norm and 1/e is the answer. But should we consider IID to be the default if we are an advanced quant? Here I think we might consider at least two streams of thought in answering:First, what's interesting to me is that the history of games of chance is one of people working extremely hard to create the normally unnatural condition of IID. Gamblers, casinos, and casino regulators have worked hard to create a fair games in a world in which fair games are, in fact, not the norm (by accident or fraud). From an empirical standpoint most natural and economic rare events arise from a process that doesn't generate perfect IID over time. So if my default reasoning on the question assumes a natural and economic source of the event, the P≠1/e.Second, you are right that I've spun the words of the problem. And yet that is exactly the kind of behaviour one should expect from counter-parties, regulators, opposing lawyers, and politicians. If the rare event involves a pay-off or consequences to other stakeholders, then I must consider the potential for interference that hastens or retards that event or induces non-independent versions of that event (e.g., a rare bankruptcy leads to a spate of bank runs).If Wilmott's brainteaser forum is meant for lower-level quants being hired by less-thoughtful people, then the answerer should assume IID and say "1/e". But if the brainteaser is meant for creme-of-the-crop quants who are expected to innovate to find new ideas, respond to complex risks, and deal with a socio-legal-political environment found at the very top of the quant food chain, then they'd better have a better answer than "1/e." That answer might be as I've outlined (i.e., "it depends on the event generator and here some common alternatives") or it might be an empirical answer (i.e., "given positively autocorrelated volatility often observed in financial markets, then P<1/e") or it might be a game theoretic answer (i.e., "how much will I and others win/lose if I assume IID?") Perhaps the real brain teaser, for the prospective job candidate, is to judge the cleverness of the interviewer as well as the nature of the job and decide if they want the undergraduate probability solution or something much more sophisticated.