Serving the Quantitative Finance Community

 
User avatar
jgelfand
Topic Author
Posts: 6
Joined: December 30th, 2004, 8:18 pm

Kolmogorov-Smirnov Normality Test

February 13th, 2019, 9:59 pm

Hello,

I have the following question...please, help

I have two vectors with returns, and run two sample Kolmogorov-Smirnov test on them, that comes back with zero i.e. can't reject null hypothesis that both vectors come from the same distribution.

Then I start testing each vector separately for Normality with one-sample  Kolmogorov-Smirnov test, and one set comes back as Normal (more precise, can't reject normality hypothesis), while CAN REJECT for another.

In all tests use 5% confidence.

How is it possible ? 
 
User avatar
Alan
Posts: 2958
Joined: December 19th, 2001, 4:01 am
Location: California
Contact:

Re: Kolmogorov-Smirnov Normality Test

February 14th, 2019, 1:59 am

I don't see an issue. All statistical tests have errors -- indeed Type I and Type II errors. Take a normal distribution and generate two artificial vectors of returns. Apply your tests. Say the two-sample KS test can't reject the Null -- fine, it's correct. Now do the one-sample tests. One says normal and one rejects normal. OK, the one rejecting made a Type I error, which it is *supposed* to do in 5% of the samples.  Now there is probably some smaller, but strictly positive probability that you get an incorrect rejection *given* the other two results.

So, the pattern of the three test results was: {CORRECT, CORRECT, INCORRECT}. I suppose you could do Monte Carlos to see what percentage of the time you get this pattern.  I don't see any reason why all 8 possible patterns won't each occur with some positive probability, regardless of the (common) confidence level.   

Of course, the general rule with financial data is that, if you collect enough of it, any *particular* distribution will always be rejected an arbitrary confidence level. 
 
User avatar
jgelfand
Topic Author
Posts: 6
Joined: December 30th, 2004, 8:18 pm

Re: Kolmogorov-Smirnov Normality Test

February 14th, 2019, 1:36 pm

Thank you. The problem for me is - what result can I lean more ? Two-sample KS, and conclude that the two vectors come from the same distribution ? Or , the second one, that rejects the hypothesis that Vector_1 is normal, and can't reject that Vector_2 is not normal ? Two test results contradict , do they ?
 
User avatar
Alan
Posts: 2958
Joined: December 19th, 2001, 4:01 am
Location: California
Contact:

Re: Kolmogorov-Smirnov Normality Test

February 14th, 2019, 3:25 pm

Contradict is too strong: let's just say the tests have weak power and are not very informative in your case.
 
I suggest you move on to other tests and approaches: Q-Q plots, for example. 

To make any further sensible suggestions, one would need to know some details: what are the assets, data frequency, and sample sizes?  
 
User avatar
jgelfand
Topic Author
Posts: 6
Joined: December 30th, 2004, 8:18 pm

Re: Kolmogorov-Smirnov Normality Test

February 14th, 2019, 6:48 pm

In a nutshell: I have a model output of 5000 returns, and I would like to make sure that they are sensible i.e. that model generated returns and actual asset realized returns (lets say S&P 500) come from the same distribution. The generated returns are normal, actual observed fail normality test, but two-sample KS fails to reject NULL hypothesis that they come from the same distribution.
 
User avatar
Alan
Posts: 2958
Joined: December 19th, 2001, 4:01 am
Location: California
Contact:

Re: Kolmogorov-Smirnov Normality Test

February 14th, 2019, 8:03 pm

I see. Well in this case, we can be sure that the two-sample KS result is simply misleading -- i.e. wrong. That's because, at least at daily frequency, there is voluminous evidence that S&P500 returns are not normal: having skewness, excess kurtosis, and much heavier tails than a normal distribution. This is clear from both the sample moments, the mentioned Q-Q plots, and maximum likelihood fits to parameterized non-Gaussian distributions. 

In fact, one knows from basic GARCH modelling that daily SPX returns are not even conditionally normal, conditioning on the daily volatility estimates. Again, this is due to the relatively wide tails. 

Apparently, the problem with the K-S test is that it is not sensitive to the tails. A bit of googling a few minutes ago confirmed the notion. See the Abstract here:

It is well known that the Kolmogorov-Smirnov (K-Stest exhibits poor sensitivity to deviations from the hypothesized distribution that occur in the tails. A modified version of the K-S test is introduced that is more sensitive than the K-S test to deviations in the tails.
 

The bottom line is that K-S is apparently a very poor way to investigate deviations from normality for financial returns data. Your results are simply a case in point.
 
User avatar
jgelfand
Topic Author
Posts: 6
Joined: December 30th, 2004, 8:18 pm

Re: Kolmogorov-Smirnov Normality Test

February 14th, 2019, 9:05 pm

Thank you. A few more observations:
1. I use annual data (if it matters)
2. I did run the analysis for IG Fixed Income (US Agg), Large Cap and Mid Cap. In q-q plots for US Agg and Large Capp the deviations are visible in the tails, while the quantiles in the middle form almost perfect line. The two sample KS fails to reject NULL.
For MidCap (Russell Mid Cap), the deviations in Q-Q plots are visible now in the belly , and KS reject s NULL.
 
User avatar
Alan
Posts: 2958
Joined: December 19th, 2001, 4:01 am
Location: California
Contact:

Re: Kolmogorov-Smirnov Normality Test

February 14th, 2019, 11:40 pm

Thank you. A few more observations:
1. I use annual data (if it matters)
That makes a big difference. While daily data is strongly non-Gaussian, the deviations from normality decay with longer horizons. When you said you simulated 5000 draws, I naturally assumed it represented about 20 years of daily obs.  Of course, with annual observations, it's hard to get a decent size sample.
 
User avatar
ISayMoo
Posts: 2332
Joined: September 30th, 2015, 8:30 pm

Re: Kolmogorov-Smirnov Normality Test

February 19th, 2019, 10:48 pm

Are the returns overlapping, by any chance?