I am doing a little survey of the existing methods; below are the ones I found so far:Constructive methods : 1) Spectral decomposition (Rebonato-Jackel 1998) 2) "completion method" (Jackel-Kahl 2009)OPtimization methods: 1) Hypersphere decomposition (Rebonato-Jackel 1998, Dash 2004(?), Kercheval 2005) 2) Differential Evolution (Mishra 2007) 3) Quasi-Newton methods (Qi-Sun, Hingham 2006-2009)My questions are : are there any other methods ? Which are the benchmark ones ? my guess would be Rebonato-Jackel and Differential Evolution...What kind of errors (or norms) are commonly used ? Any other references (articles or C++ codes) are most welcome !Thank you

Norms are completely irrelevant. Unfortunately all the papers keep using norms, to make the results look more mathematical. What you care about are pricing errors. Say there is a true correlation matrix of your assets, R1. Due to the finite number of observations, you can estimate R2 which is different, but still positive definite. Due to missing data in some of the timeseries, you will calculate yet a different matrix, R3, which is not even positive definite. To this matrix you apply your nearest correlation matrix estimation, and let's say you produce R4. Now because you have several ways to do this, you have R4-Jackel, R4-Higham, etc. What you should do is price a test option using R1, R2 and the several versions of R4. Call the difference between the price obtained with R2 vs R1 the base "measurement uncertainty" noise. No matter what nearest correlation matrix procedure you use, you cannot hope to get below this noise level. Now benchmark your procedures agains this. If with one procedure you get a price difference roughly equal to 5 times this base noise, and with another procedure you get one 10 times, then the first procedure is better. Best,V.

Costeanu : being a mathematician and prone to use norms I totally agree with you on this one : here norms are useless. This is a subject that have been bothering me since the first article I read. The thing is norms are used here to measure the optimization error and the problem being finite dimensional ... all norms are equivalent.Now, one could try to define a norm that would somehow make sense economically or, and I think this is the best direction, understand what are the economical constraints behind a correlation matrix : considering only the mathematical ones (symmetry and poditive definite) is too simplistic.You approach does not apply directly the situation I am working on (loan portfolio risk) but I can easily derive one for it: thanks !Outrun : I also think the statistical approach is a step in the right direction : if we cant understand the economic constraints behind the correlation at least we can try to estimate them. Thank you for the references !Another question : if I understand you correctly, both your methods apply while in the process of estimating correlations. What about situation where you wanna measure the sensibility or stress your portfolio with respect to those correlations ?

QuoteOriginally posted by: outrunyou mean that you want to model the assumed effect that correlations can change in time? No, but that is a situation I will study in the futureIn the case of a loan portfolio, I want to understand the impact of correlation movements on the risk measures (stress testing for regulatory purpose and sensibility study for internal strategy). For the sake of completness and clarity below is a simple example :say some countries correlations are doubled what is the impact on VaR ? (doubling a coefficient will usually break the def pos property hence the need again to estimate a nearest correlation matrix)Note: I will comment on your video link later since I cant watch it atm

QuoteOriginally posted by: outrunQuoteOriginally posted by: Bourbadoubling a coefficient will usually break the def pos property hence the need again to estimate a nearest correlation matrixI would say that "doubling a coefficient" would be an invalid correlation matrix adjustment. Suppose you do like you said: 1) you adjust the correlation matrix manually2) you let some algorithm modify it furtherHow do you make certain that what happens in step 2 doesn't interfere with what you do in step 1)? ... Step 2) might -as a side effect of step 1)- adjust other correlations in such a way that in some situations VAR goes up, and in other cases VAR goes down... The overall change in VAR caused by the two steps combined may not be linked to your step 1 adjustment, so you can't say "doubling coefficient X has Y effect on the VAR"A more valid method I've come across is this:You create a set of correlation matrix samples based on historical periodsYou use the set of matrices to model the correlation between correlation coefficients.. some coefficient are highy correlated, other not...With that knowledge you can change certain coefficients, and let some EM algorithm figure out the most likely changes in the other coefficient to keep the matrix valid.First thank you ... because this is what I have been wanting to hear for quite some time (on this question the main feedback I have been getting comes from consultants with whom i am working: they have no problem doubling correlations coefficients )I could not agree more on your first paragraph : that is exactly my issue with these optimization algorithms. Sometimes they might not give to bad estimates, and that is only because we have a matrix with sufficiently SMALL negative eigenvalues, hence the distance to the nearest correlation matrix will be SMALL (not necessarly of the same order since the problem is highly nonlinear, but by continuity for sufficiently small it works). Now, in the case of my former example the change I am doing is nothing but small, and the impact on the eigenvalues will be in general that BIG negative eigenvalues appear : then like you said the nearest correlation obtained is economically meaningless...One way to temper this problem are the "new" methods that allow to fix some coefficients of the target matrix : the same problems remain but it is a tad better imho (see Kercheval, Si-Qu in my first post).I really like the method you present on your second paragraph.

Hi Bourba,One way to stress correlation matrices is to notice that they form a convex subset of all the n x n matrices. Among all correlation matrices there are two distinguished ones: the identity, and the one where all the entries are one, let's call them R0 and R1. If you want to stress your correlation matrix R you can form convex combinations with these: with R0 is you want to decorrelate and with R1 if you want to correlate. Obviously, with this method you can't stress the correlation of a specific pair. Another method is this (again can't stress a specific pair, but I think it's the next best thing): imagine your correlation matrix comes from a one-factor model. Meaning there is a hidden "market" variable M, and all your variables are of the form X_i = beta_i *M + idiosynchratic. This means that the pairwise correlations are rho(X_i, X_j) = beta_i * beta_j. With such a model you can stress the individual betas. Now if you have a more general matrix, just do a PCA on it and retain enough principal components so you are satisfied you start with a correlation matrix very close to the original one. Of course you will have many more betas to stress, but you can stress only the beta for the first principal component for example. I think this concept is pretty simple (all portfolio managers understand betas). Best of luck,V.