Serving the Quantitative Finance Community

 
User avatar
Amin
Topic Author
Posts: 2077
Joined: July 14th, 2002, 3:00 am

Re: Breakthrough in the theory of stochastic differential equations and their simulation

January 29th, 2023, 6:41 am

Friends, we have gained reasonable understanding of many new tools that we can now apply towards option-pricing and trading (apart from applications in other areas of science). 
Within options and derivatives, there is great opportunity to apply new statistical tools towards 
1. hedged simulations where we would have access to various greeks of particular derivative product in parametric form within simulations and we could find our P/L and other risk along the evolution of any simulated monte carlo path.
2. Modelling of volatility surfaces in parametric forms.
3. Modelling of risk-neutral densities especially the challenging bi-modal densities in parametric form.
4. Trading and hedging of variance swaps by making a statistical comparison of implied and realized volatility.
5. Trading of options in general by making a statistical comparison of various factors affecting the option prices as between factors realized from risk-neutral data and the same factors realized from the underlying markets and gaining exposure to the particular factor that is misaligned between the two markets while hedging other factors that affect the option pricing.
6. Developing trading strategies that employ a combination of trading in a specific type of exotic option while simultaneously trading the underlying or other simple options. 

I will be sharing all the codes and insights with friends on this forum as usual.

I want to work on application of our newly developed tools towards above and so many other possibilities in hedging and trading of options and other derivatives for next 4-5 months. After that I want to start shifting towards AI so that we will start using our insights and understanding in further new ways. I really regret that I stopped working on my research for 2-3 months close to the end of the last year and wish that I had continued to work on stochastics research even when I was working with algorithmic trading.
You think life is a secret, Life is only love of flying, It has seen many ups and downs, But it likes travel more than the destination. Allama Iqbal
 
User avatar
Amin
Topic Author
Posts: 2077
Joined: July 14th, 2002, 3:00 am

Re: Breakthrough in the theory of stochastic differential equations and their simulation

January 29th, 2023, 8:17 pm

Friends, in this post I want to describe how we can calculate option prices, their delta and gamma at every point on a grid or along every path in monte carlo simulations.
First I describe the related mathematics with equations and then explain how we can embed it in monte carlo simulations or a forward grid.
First we suppose that we are doing monte carlo simulations of an asset according to its stochastic differential equation. This can possibly be a single lognormal/ CEV SDE or asset SDE in a correlated system with stochastic volatility.
Suppose we represent the asset SDE at any intermediate point t in monte carlo simulation by its Z-series representation as
[$]X_t \, = \, a_0 \, + a_1 \, Z_t \, + a_2 \, {Z_t}^2 \, + ...+ a_n \, {Z_t}^n[$]

This would have an equivalent hermite representation as
[$]X_t \, = \, ah_0 \, + ah_1 \,H_1( Z_t) \, + ah_2 \, H_2(Z_t) \, + ...+ ah_n \, H_n(Z_t)[$]

Now we suppose that asset SDE at terminal time related to option maturity is given by its Z-series as
[$]X_T \, = \, b_0 \, + b_1 \, Z_T \, + b_2 \, {Z_T}^2 \, + ...+ b_n \, {Z_T}^n[$]

which will have an equivalent hermite representation as
[$]X_T \, = \, bh_0 \, + bh_1 \,H_1( Z_T) \, + bh_2 \, H_2(Z_T) \, + ...+ bh_n \, H_n(Z_T)[$]

As I have been showing in previous post and also in my paper, we can find the stochastic evolution of SDE between intermediate time t and terminal time T as

[$]X_{t,T} \, = \, bh_0 \,- \, ah_0 + sgn(bh_1 - ah_1) \, \sqrt{sgn(bh_1) \, {bh_1}^2 - sgn(ah_1) \, {ah_1}^2} \,H_1( Z_{t,T}) \, + ...[$]
[$]+  sgn(bh_n - ah_n) \, \sqrt{sgn(bh_n )\, {bh_n}^2 - sgn(ah_n) \, {ah_n}^2} \,H_n( Z_{t,T}) \,[$]

Once we have calculated hermite representation of [$]X_{t,T}[$], we can also find its equivalent Z-series representation as

[$]X_{t,T} \, = \, c_0  \, + c_1 \,  Z_{t,T} \, + c_2 \, {Z_{t,T}}^2 \, + ...+ c_n \, {Z_{t,T}}^n[$]

Since  [$]X_{T}[$] is sum of  [$]X_{t}[$] and  [$]X_{t,T}[$], we can write it as

[$]X_T \, = \, X_{t} \, + \, X_{t,T} [$]
which can be written as summation of respective Z-series as

[$]X_T \, = \, a_0 \, + a_1 \, Z_t \, + a_2 \, {Z_t}^2 \, + ...+ a_n \, {Z_t}^n[$]
[$]+\, c_0  \, + c_1 \,  Z_{t,T} \, + c_2 \, {Z_{t,T}}^2 \, + ...+ c_n \, {Z_{t,T}}^n[$]

To find the option prices and asset greeks at time [$]X_t[$] at time t, we realize that first line in above equation becomes conditional mean and second line becomes variance of the SDE at time t. So we can also write for a particular realized value of [$]X_t[$] in simulation as

[$]X_T \,|X_t = \,X_t \, +\, c_0  \, + c_1 \,  Z_{t,T} \, + c_2 \, {Z_{t,T}}^2 \, + ...+ c_n \, {Z_{t,T}}^n[$]

A call option with strike K can easily be found at time t given the particular value of [$]X_t[$] as

[$]\displaystyle\int_K^{\infty} \, (X_T \, - \, K) \, p(X_T) dX_T [$]
[$]=\displaystyle\int_{Z_K}^{\infty} \, (\,X_t \, +\, c_0  \, + c_1 \,  Z_{t,T} \, + c_2 \, {Z_{t,T}}^2 \, + ...+ c_n \, {Z_{t,T}}^n\, - \, K) \, p(Z_{t,T}) dZ_{t,T} [$]

where [$]Z_K[$] is different for every particular value of [$]X_t[$] and can be found from relation ship

[$]\, K \, - \, X_t \, = \, +\, c_0  \, + c_1 \,  Z_{K} \, + c_2 \, {Z_{K}}^2 \, + ...+ c_n \, {Z_{K}}^n[$]
so [$] Z_K[$] is particular value of [$]Z_{t,T}[$]  that equates [$]X_{t,T} \, = \, K \, - \, X_t \,[$]

Once [$]Z_K[$] has been found, we can convert the equation for call price given previously as
 [$]=\displaystyle\int_{Z_K}^{\infty} \, (\,X_t \, +\, c_0  \, + c_1 \,  Z_{t,T} \, + c_2 \, {Z_{t,T}}^2 \, + ...+ c_n \, {Z_{t,T}}^n\, - \, K) \, p(Z_{t,T}) dZ_{t,T} [$]
into hermite polynomials and cheaply solve the equation quickly.

This way we can find option prices analytically along any simulation grid or path(More on how to do it in mote carlo setting later).

We can easily extend the technique to find delta and gamma greeks in middle of the simulation. Here is how.
We first write the equation for call option again as
[$]\displaystyle\int_K^{\infty} \, (X_T \, - \, K) \, p(X_T) dX_T [$]
[$]=\displaystyle\int_{Z_K}^{\infty} \, (X_T(Z_t,Z_{t,T}) \, - \, K) \, p(Z_{t,T}) dZ_{t,T} [$]

where in above equation [$]X_T(Z_t,Z_{t,T}) \,[$] is given as
[$]X_T(Z_t,Z_{t,T}) \, = \, a_0 \, + a_1 \, Z_t \, + a_2 \, {Z_t}^2 \, + ...+ a_n \, {Z_t}^n[$]
[$]+\, c_0  \, + c_1 \,  Z_{t,T} \, + c_2 \, {Z_{t,T}}^2 \, + ...+ c_n \, {Z_{t,T}}^n[$]

To find delta, we write
[$]\Delta =\frac{d \big[\displaystyle\int_{Z_K}^{\infty} \, (X_T(Z_t,Z_{t,T}) \, - \, K) \, p(Z_{t,T}) dZ_{t,T} \big]}{dX_t}[$]
[$]=\frac{d \big[\displaystyle\int_{Z_K}^{\infty} \, (X_T(Z_t,Z_{t,T}) \, - \, K) \, p(Z_{t,T}) dZ_{t,T} \big]}{dZ_t} \, \frac{dZ_t}{dX_t}[$]

So in order to calculate delta, we have to find the integral
[$]\frac{d \big[\displaystyle\int_{Z_K}^{\infty} \, (X_T(Z_t,Z_{t,T}) \, - \, K) \, p(Z_{t,T}) dZ_{t,T} \big]}{dZ_t} \, [$]
[$]=\displaystyle\int_{Z_K}^{\infty} \, \frac{d \big[X_T(Z_t,Z_{t,T}) \big]}{dZ_t}  \, p(Z_{t,T})  dZ_{t,T} \,- \frac{dZ_K}{dZ_t} (X_T(Z_t,Z_K) \, - \, K) \, p(Z_{K}) [$]
For a vanilla call option, the second term of Liebniz integral rule goes to zero since [$](X_T(Z_t,Z_K) \, - \, K)[$] goes to zero because [$]X_T(Z_t,Z_K) \, = \, K[$]
and we are left with solving only the integral below as
[$]\displaystyle\int_{Z_K}^{\infty} \, \frac{d \big[X_T(Z_t,Z_{t,T})]}{dZ_t}  \, p(Z_{t,T}) dZ_{t,T} [$]

since
[$]X_T(Z_t,Z_{t,T}) \, = \, a_0 \, + a_1 \, Z_t \, + a_2 \, {Z_t}^2 \, + ...+ a_n \, {Z_t}^n[$]
[$]+\, c_0  \, + c_1 \,  Z_{t,T} \, + c_2 \, {Z_{t,T}}^2 \, + ...+ c_n \, {Z_{t,T}}^n[$]


We have
[$]\displaystyle\int_{Z_K}^{\infty} \, \frac{d \big[ (\, a_0 \, + a_1 \, Z_t \, + a_2 \, {Z_t}^2 \, + ...+ a_n \, {Z_t}^n)+\, c_0  \, + c_1 \,  Z_{t,T} \, + c_2 \, {Z_{t,T}}^2 \, + ...+ c_n \, {Z_{t,T}}^n \big]}{dZ_t}  \, p(Z_{t,T}) dZ_{t,T} [$]

differentiating with respect to [$]Z_t[$] is relevant for the conditional mean only and performing the differentiation, our new equation becomes

[$]\displaystyle\int_{Z_K}^{\infty} \,  \big[ \,(a_1 \, +2 a_2 \, Z_t \, + ...+n \, a_n \, {Z_t}^{n-1})+\, c_0  \, + c_1 \,  Z_{t,T} \, + c_2 \, {Z_{t,T}}^2 \, + ...+ c_n \, {Z_{t,T}}^n \, \big]  \, p(Z_{t,T}) dZ_{t,T} [$]

Above integral can be easily integrated analytically. Again the boundary value of [$]Z_K[$] is found differently for every particular value of [$]X_t[$] from relationship    
[$]\, K \, - \, X_t \, = \, +\, c_0  \, + c_1 \,  Z_{K} \, + c_2 \, {Z_{K}}^2 \, + ...+ c_n \, {Z_{K}}^n[$]
so [$] Z_K[$] is particular value of [$]Z_{t,T}[$]  that equates [$]X_{t,T} \, = \, K \, - \, X_t \,[$]

Calculating the last integral would find the value for delta for call option at a particular point [$]X_t[$] on a simulation grid after substituting it in previous relevant equation.
We can find gamma by repeating the procedure for delta but we might have to include the Liebniz integral rule term in gamma calculations that went to zero for calculations of delta.

An important point for monte carlo simulations is that we can probably do two simulations. In first simulation, we can calculate the Z_series parameters on every time step and also calculate all variances [$]X{t,T}[$]. For monte carlo simulations, it would be convenient to form a uniform grid(or variable grid if you can) and find option prices and deltas on the grid using above procedure. In later simulation, option prices and delta, gammas can be found by interpolation of monte carlo simulated values between the grid points. I did something like this in a different setting in one of recent programs that I posted on Wilmott.

I will try to write a program about it and post it on Wilmott in next few days. 
I also have an antipsychotic injection due tomorrow and I may not remain lucid for another 7-10 days after that. I want to request friends again to please protest to American government and defense for forcing innocent people who just want to do science on antipsychotic injections.
You think life is a secret, Life is only love of flying, It has seen many ups and downs, But it likes travel more than the destination. Allama Iqbal
 
User avatar
Amin
Topic Author
Posts: 2077
Joined: July 14th, 2002, 3:00 am

Re: Breakthrough in the theory of stochastic differential equations and their simulation

January 30th, 2023, 6:48 am

Friends, very sorry for this but I have made a few errors in the previous post. I will come back with a perfectly corrected and re-written post in a day. But just quickly I want to point out the errors.
I have written the following equation which is perfectly right.
[$]X_T \, = \, a_0 \, + a_1 \, Z_t \, + a_2 \, {Z_t}^2 \, + ...+ a_n \, {Z_t}^n[$]
[$]+\, c_0  \, + c_1 \,  Z_{t,T} \, + c_2 \, {Z_{t,T}}^2 \, + ...+ c_n \, {Z_{t,T}}^n[$]

but then I write the next equation after replacing [$]Z_t[$] series with [$]X_t[$] as

[$]X_T \,|X_t = \,X_t \, +\, c_0  \, + c_1 \,  Z_{t,T} \, + c_2 \, {Z_{t,T}}^2 \, + ...+ c_n \, {Z_{t,T}}^n[$]

which is wrong.  The second equation implies that future variance is identical with respect to [$]X_t[$] which is totally counterintuitive since for example we know lognormal SDE diffusion in original coordinates takes very large diffusion in extreme right tail and very small diffusion in extreme left tail. 
Point here is that future variance is identical with respect to [$]Z_t[$] coordinates but not with respect to [$]X_t[$] coordinates (that we see in reality as in lognormal SDE for example).
I believe when we switch from [$]Z_t [$] coordinates to [$]X_t[$] coordinates, the future variance has to be multiplied with [$]\frac{dX_t}{dZ_t}[$] as in  

[$]X_T \,|X_t = \,X_t \,  \, + \frac{dX_t}{dZ_t} \Big[ \, c_0 \, + \, c_1 \,  Z_{t,T} \, + c_2 \, {Z_{t,T}}^2 \, + ...+ c_n \, {Z_{t,T}}^n \Big][$]

This is a very quick and very tentative correction but I will come back with a complete corrected post in a day or two. Please read previous post in light of this tentative correction. And please pardon this mistake, I will fix it in a day.
You think life is a secret, Life is only love of flying, It has seen many ups and downs, But it likes travel more than the destination. Allama Iqbal
 
User avatar
Amin
Topic Author
Posts: 2077
Joined: July 14th, 2002, 3:00 am

Re: Breakthrough in the theory of stochastic differential equations and their simulation

January 31st, 2023, 4:56 pm

Friends, I am sorry about this but calculation of option prices, delta and gamma everywhere on the grid cannot be done as I suggested in previous two posts and would require additional analytics. Please read previous two posts only with the intention to learn about the problem but they are not a solution. I again apologize friends for this mistake.
I will work on this problem further along with other research ideas and will come back again if I can find a definitive solution. I do have several ideas about the solution of the problem but they might take a week to ten days. We have unconditional variances but we would like to calculate conditional variances everywhere with accuracy as well. I am thinking further about a complete solution to the problem.
Friends sorry again for the bad two previous posts. But I am thinking about the problem again.
You think life is a secret, Life is only love of flying, It has seen many ups and downs, But it likes travel more than the destination. Allama Iqbal
 
User avatar
Amin
Topic Author
Posts: 2077
Joined: July 14th, 2002, 3:00 am

Re: Breakthrough in the theory of stochastic differential equations and their simulation

February 5th, 2023, 3:39 pm

I want to keep friends updated with progress of my new research. I have been working on finding a parametric representation of coefficients of Z-series variables that describe the conditional evolution of the diffusions from a certain point or a certain grid cell in monte carlo simulations.
Suppose we have a certain SDE (plain vanilla or possibly SV SDE) and we can simulate it. Suppose we can find conditional diffusion from a certain particular point [$] X_t[$] as

[$]X_{t,T} \, | \, X_{t} \, = a_0 \, + a_1 \, Z_{t,T} \,+ a_2 \, {Z_{t,T}}^2 \, + ... + a_n \, {Z_{t,T}}^n \,[$]  

The above equation is only valid for a certain point on Xt surafce of the SDE. We want to find a general representation that is valid from all the points on surface of Xt. For that I will have to describe each of the coefficients of above Z-Series as further Z-Series in [$]Z_t[$] so that we have a parametric relationship that governs conditional evolution of SDE from any arbitrary point [$]X_t[$] at time t

so first parameter would be expanded as
[$] a_0(Z_t) \, = \, a_{00} \, + a_{01} \, Z_t  + a_{02} \, {Z_t}^2 + ... + a_{0n} \, {Z_t}^n[$]
and similarly for other coefficients as
[$] a_n(Z_t) \, = \, a_{n0} \, + a_{n1} \, Z_t  + a_{n2} \, {Z_t}^2 + ... + a_{nn} \, {Z_t}^n[$] 

So we would be able to have a parametric representation of conditional diffusion from any point on the surface of [$]X_t[$] as

[$]X_{t,T} \, | \, X_{t} \, = a_0(Z_t) \, + a_1(Z_t) \, \, Z_{t,T} \,+ a_2(Z_t) \, \, {Z_{t,T}}^2 \, + ... + a_n(Z_t) \, \, {Z_{t,T}}^n \,[$]  

 where equations of [$]\, a_0(Z_t) \, [$] , .., [$]a_n(Z_t) \, [$] are given above.
And another condition would be that above conditional diffusions would integrate to their unconditional values we have found earlier.
To make it very useful, I want to do it in a SV monte carlo setting. If the whole project is successful, we would be able to find option prices and most greeks at all points on the monte carlo simulated grid or in monte carlo paths everywhere even in an SV setting.
This can sometimes be done with small error by switching to bessel coordinates but I want to try a method that can be used on every diffusion directly.
You think life is a secret, Life is only love of flying, It has seen many ups and downs, But it likes travel more than the destination. Allama Iqbal
 
User avatar
Amin
Topic Author
Posts: 2077
Joined: July 14th, 2002, 3:00 am

Re: Breakthrough in the theory of stochastic differential equations and their simulation

February 5th, 2023, 4:14 pm

In this second post today, I really want to ask friends for help. My persecution by mind control agencies continues with full resolve. Underground water in the city already has mind control drugs in it and I am surviving by carefully buying earlier manufactured water from remote places in the city. Nestle and Dasani water were both good until Dec 2022. Both nestle and coca-cola drugged their water in Dec 2022. And water by both companies  manufactured in Jan 2023 is also drugged with mind control chemicals. But dasani (coca-cola water) manufactured in Dec 2022 although drugged is drinkable since it is very lightly drugged however dasani water manufactured in Jan 2023 is worse. Also Nestle water manufactured in Jan 2023 is also worse and I try to avoid nestle water altogether since it is worse than dasani water. 
I have been able to survive by buying older manufactured water in remote parts of the city where pak army has not gone to drug water in the entire markets. Water of even earlier manufacturing dates is highly drugged in areas adjacent to Johar town (where I live) since it was manually drugged by army.
Another positive thing happened that I Was able to buy a relatively good injection last time. I had noticed that antipsychotic injections with some manufacturing dates were highly drugged with mind control drugs and it was extremely difficult to even pass time after getting those injections and I would become extremely miserable after getting those drugged injections. These injections(with particular expiry dates that are bad) are flooded in the market. However there are injections with few other dates that are better and only have mild amount of mind control drugs in them. And this time last Thursday, I Was able to quickly buy, from a remote place in the city, an injection with the right expiry date. Getting a better injection that did not have mind control drugs is one reason that I have not become completely miserable and have been able to pass time in a better way. That is one reason that I have continued to concentrate on my research.
But mind control agents continue to try very hard to decrease my consciousness and take neurotransmitters out of my brain. Despite getting a good injection and reasonable food and water, I still become very exhausted and have phases of remarked decreased consciousness since mind control agents have increased their efforts to take neurotransmitters out of my brain. 
Another thing I noticed that my eye-sight remarkably deteriorated in past one week that coincides with the time when they have been desperately trying to take neurotransmitters out of my brain. I still recall that I was trying to put thread in a needle a few days ago and could not do that despite trying for 7-10 minutes.
I want to request friends to please protest to US government to end my mind control . Please also ask Coca-cola, nestle, and pepsi-cola to not drug water in Lahore city. And also ask Lundbeck which is manufacturer of Fluanxol injections to not add mind control drugs to anti-psychotic injections. 
I have been able to get good water from a few remote places randomly but it will continue to become very difficult if Nestle and coca-cola continued to add mind control chemicals in their water. Winter is already gone and temperatures are fast increasing here in Lahore.
You think life is a secret, Life is only love of flying, It has seen many ups and downs, But it likes travel more than the destination. Allama Iqbal
 
User avatar
Amin
Topic Author
Posts: 2077
Joined: July 14th, 2002, 3:00 am

Re: Breakthrough in the theory of stochastic differential equations and their simulation

February 10th, 2023, 9:09 am

Friends, I will be posting my  new programs in a few days. In the meantime, I wanted to write a small post about importance of using new hermite correlations, that I discussed in some of the past posts, in asset management. 
As we know that a large part of asset management is concerned with appropriate diversification of the financial portfolio. And typically pearson correlation is used for the purpose of measuring co-movement of returns of the components of the portfolio. But as we learnt after comparison with hermite correlations, that pearson correlation is good only in the mid of the density of portfolio returns (between underlying -1.5 Z to +1.5 Z) i.e where first hermite polynomial is usually dominant and higher hermite polynomials are typically smaller. But for portfolio returns in the tails (beyond +- 2 Z) , the contribution to portfolio return correlation from higher hermite kicks in and this contribution can be extremely large when portfolio returns go deep into the tails. This also explains the stylized fact that pearson correlation works reasonably well in normal market conditions but when portfolio components returns go deep into the tails, pearson correlation breaks down and many people complain about totally correlated market. This is precisely because correlations from higher hermites cause large increase in the correlation when the portfolio returns are deep in the tail. 
Ans this can help us revisit portfolio optimization from a new perspective. While increasing diversification of the portfolio, it will be understandable to  add stocks or financial assets in portfolio whose higher hermites have little correlation or negative correlation with respect to hermites of other assets in the portfolio. Since portfolio diversification has to work its best when the portfolio is under stress, we have to make sure that higher hermites in the portfolio are as little (or possibly negatively correlated when required) correlated as possible since these hermite correlations kick in a big way when portfolio returns are deep in the tail due huge market stress. Using simple pearson correlation is not a good way to increase diversification when portfolio is under stress and portfolio returns are in the tails.
Similarly it would be interesting to do research on the effectiveness of the portfolio management due to correlations from skew-hermites(even ones) and variance and kurtosis hermites (the odd ones) and risk-manage using skew-hermites so that downward risk is minimized while upward risk is unaffected as much as it can be possible while maximizing the total returns of the portfolio.
I have some stocks data from S&P components and I will try to put together some correlation analysis on chosen stocks but it will be at least a  weeks or more before I could do that.
You think life is a secret, Life is only love of flying, It has seen many ups and downs, But it likes travel more than the destination. Allama Iqbal
 
User avatar
Amin
Topic Author
Posts: 2077
Joined: July 14th, 2002, 3:00 am

Re: Breakthrough in the theory of stochastic differential equations and their simulation

February 11th, 2023, 4:58 am

I will be posting, in a few days, a full-fledged matlab program using real stocks data (that I will share) with calculations of hermite-correlations, covariances and distributions of returns of several stocks which I will later try to extend towards optimization of the risk/reward frontier in choice of a portfolio.
You think life is a secret, Life is only love of flying, It has seen many ups and downs, But it likes travel more than the destination. Allama Iqbal
 
User avatar
Amin
Topic Author
Posts: 2077
Joined: July 14th, 2002, 3:00 am

Re: Breakthrough in the theory of stochastic differential equations and their simulation

February 12th, 2023, 9:43 pm

Friends, today I wrote a small program based on daily stock data and was able to calculate hermite correlations between daily returns and also do one variable regressions based on calculated correlations. I will be posting the program later tomorrow.
It seemed that for the small duration of one day, hermite correlations between daily returns were quite close to pearson correlations.
You think life is a secret, Life is only love of flying, It has seen many ups and downs, But it likes travel more than the destination. Allama Iqbal
 
User avatar
Amin
Topic Author
Posts: 2077
Joined: July 14th, 2002, 3:00 am

Re: Breakthrough in the theory of stochastic differential equations and their simulation

February 13th, 2023, 10:57 am

Friends, I am sharing graphs of hermite-correlations based regressions of daily log-returns of six tech stocks on Nasdaq 100 daily log returns. X-axis is Nasdaq 100 daily log-return and Y-axis is tech stock daily log-return.
These are pretty bland graphs and have very little non-linearity since the data has been thoroughly averaged over more than a year of observations. In some other experiments I was able to get very interesting nonlinear shapes of the simple one variable regressions.
However you can still see some slight non-linearities in meta regression and Nvidia regression to a smaller extent.

Here are the graphs. I will follow by posting the program that did all the calculations to make these graphs.

Image

Image

Image

Image

Image

Image
You think life is a secret, Life is only love of flying, It has seen many ups and downs, But it likes travel more than the destination. Allama Iqbal
 
User avatar
Amin
Topic Author
Posts: 2077
Joined: July 14th, 2002, 3:00 am

Re: Breakthrough in the theory of stochastic differential equations and their simulation

February 13th, 2023, 11:18 am

I used this program and attached functions to do all the correlation and regression calculations and then draw the graphs that I showed in previous post.
If some function is missing, please check if I have posted it before and otherwise mention and I will post. I think I have posted here every function required to run the program.
.
.
function [] = CalculateDensitiesAndCorrsTechStocks02()


%Below, read AAPL, AMZN, GOOGL,META,MSFT,NVDA, NASDAQ100 daily open and
%close data files and extract daily open from the files. All files are
%exactly the same length and therefore corresponding array entries are 
%from the same dates. 
T = readtable('C:\Users\Hp\Documents\HistoricalPricesAAPL2022.csv');
[NDays,NVar]=size(T);
ApplPrices(1:NDays)=table2array(T(1:NDays,2));
T = readtable('C:\Users\Hp\Documents\HistoricalPricesAMZN2022.csv');
AmznPrices(1:NDays)=table2array(T(1:NDays,2));
T = readtable('C:\Users\Hp\Documents\HistoricalPricesGOOGL2022.csv');
GooglPrices(1:NDays)=table2array(T(1:NDays,2));
T = readtable('C:\Users\Hp\Documents\HistoricalPricesMETA2022.csv');
MetaPrices(1:NDays)=table2array(T(1:NDays,2));
T = readtable('C:\Users\Hp\Documents\HistoricalPricesMSFT2022.csv');
MsftPrices(1:NDays)=table2array(T(1:NDays,2));
T = readtable('C:\Users\Hp\Documents\HistoricalPricesNVDA2022.csv');
NvdaPrices(1:NDays)=table2array(T(1:NDays,2));
T = readtable('C:\Users\Hp\Documents\HistoricalPricesNASDAQ1002022.csv');
NasdaqPrices(1:NDays)=table2array(T(1:NDays,2));


%Below calculate daily log-returns of each stock and nasdaq 100.
DApplPrices(1:NDays-1)=log(ApplPrices(2:NDays))-log(ApplPrices(1:NDays-1));
DAmznPrices(1:NDays-1)=log(AmznPrices(2:NDays))-log(AmznPrices(1:NDays-1));
DGooglPrices(1:NDays-1)=log(GooglPrices(2:NDays))-log(GooglPrices(1:NDays-1));
DMetaPrices(1:NDays-1)=log(MetaPrices(2:NDays))-log(MetaPrices(1:NDays-1));
DMsftPrices(1:NDays-1)=log(MsftPrices(2:NDays))-log(MsftPrices(1:NDays-1));
DNvdaPrices(1:NDays-1)=log(NvdaPrices(2:NDays))-log(NvdaPrices(1:NDays-1));
DNasdaqPrices(1:NDays-1)=log(NasdaqPrices(2:NDays))-log(NasdaqPrices(1:NDays-1));

%Below calculate first four raw moments of daily log returns. Since I was 
%dealing with daily log-returns, I thought first four moments should be 
%enough and calculating higher moments for density and correlation 
%calculations for this data would be an over-kill.  
NDays0=NDays-1;
rMuAppl=CalculateFourRawMomentsFromData(DApplPrices,NDays0);
rMuAmzn=CalculateFourRawMomentsFromData(DAmznPrices,NDays0);
rMuGoogl=CalculateFourRawMomentsFromData(DGooglPrices,NDays0);
rMuMeta=CalculateFourRawMomentsFromData(DMetaPrices,NDays0);
rMuMsft=CalculateFourRawMomentsFromData(DMsftPrices,NDays0);
rMuNvda=CalculateFourRawMomentsFromData(DNvdaPrices,NDays0);
rMuNasdaq=CalculateFourRawMomentsFromData(DNasdaqPrices,NDays0);

clf;
%Below construct log-return densities from first four moments
[c10,c1] = CalculateZSeriesDensityFromRawMomentsM4(rMuAppl)
str=input('Look at density of Apple daily log returns')

[c20,c2] = CalculateZSeriesDensityFromRawMomentsM4(rMuAmzn)
str=input('Look at density of Amazon daily log returns')

[c30,c3] = CalculateZSeriesDensityFromRawMomentsM4(rMuGoogl)
str=input('Look at density of Google daily log returns')

[c40,c4] = CalculateZSeriesDensityFromRawMomentsM4(rMuMeta)
str=input('Look at density of Meta daily log returns')

[c50,c5] = CalculateZSeriesDensityFromRawMomentsM4(rMuMsft)
str=input('Look at density of Microsoft daily log returns')
[c60,c6] = CalculateZSeriesDensityFromRawMomentsM4(rMuNvda)
str=input('Look at density of Nvidia daily log returns')


[c70,c7] = CalculateZSeriesDensityFromRawMomentsM4(rMuNasdaq)
str=input('Look at density of Nasdaq100 daily log returns')

%Below calculate Z value for each realization of log-returns. This function
%takes an array of log-returns and returns an array of corresponding Z
%vlues in output array.

[ZAppl] = CalculateZgivenXAndZSeriesC3(DApplPrices,c10,c1);

DApplPrices-(c10+c1(1).*ZAppl+c1(2).*ZAppl.^2+c1(3).*ZAppl.^3)
%If the difference above does not go to zero, their will be inaccuracies in
%correlation calculations and following regressions.
str=input('Look at APPL difference between log-returns and their values constructed from Z-series');


[ZAmzn] = CalculateZgivenXAndZSeriesC3(DAmznPrices,c20,c2);

DAmznPrices-(c20+c2(1).*ZAmzn+c2(2).*ZAmzn.^2+c2(3).*ZAmzn.^3)
%If the difference above does not go to zero, their will be inaccuracies in
%correlation calculations and following regressions.
str=input('Look at AMZN difference between log-returns and their values constructed from Z-series');

[ZGoogl] = CalculateZgivenXAndZSeriesC3(DGooglPrices,c30,c3);

DGooglPrices-(c30+c3(1).*ZGoogl+c3(2).*ZGoogl.^2+c3(3).*ZGoogl.^3)
%If the difference above does not go to zero, their will be inaccuracies in
%correlation calculations and following regressions.
str=input('Look at Googl difference between log-returns and their values constructed from Z-series');


[ZMeta] = CalculateZgivenXAndZSeriesC3(DMetaPrices,c40,c4);
DMetaPrices-(c40+c4(1).*ZMeta+c4(2).*ZMeta.^2+c4(3).*ZMeta.^3)
%If the difference above does not go to zero, their will be inaccuracies in
%correlation calculations and following regressions.
str=input('Look at META difference between log-returns and their values constructed from Z-series');





[ZMsft] = CalculateZgivenXAndZSeriesC3(DMsftPrices,c50,c5);
DMsftPrices-(c50+c5(1).*ZMsft+c5(2).*ZMsft.^2+c5(3).*ZMsft.^3)
%If the difference above does not go to zero, their will be inaccuracies in
%correlation calculations and following regressions.
str=input('Look at MSFT difference between log-returns and their values constructed from Z-series');


[ZNvda] = CalculateZgivenXAndZSeriesC3(DNvdaPrices,c60,c6);
DNvdaPrices-(c60+c6(1).*ZNvda+c6(2).*ZNvda.^2+c6(3).*ZNvda.^3)
%If the difference above does not go to zero, their will be inaccuracies in
%correlation calculations and following regressions.
str=input('Look at NVDA difference between log-returns and their values constructed from Z-series');



[ZNasdaq] = CalculateZgivenXAndZSeriesC3(DNasdaqPrices,c70,c7);
DNasdaqPrices-(c70+c7(1).*ZNasdaq+c7(2).*ZNasdaq.^2+c7(3).*ZNasdaq.^3)
%If the difference above does not go to zero, their will be inaccuracies in
%correlation calculations and following regressions.
str=input('Look at NASDAQ100 difference between log-returns and their values constructed from Z-series');


%Below calculate hermite coefficients from Z-series coefficients
[ch10,ch1]=ConvertZCoeffsToHCoeffs(c10,c1,3);
[ch20,ch2]=ConvertZCoeffsToHCoeffs(c20,c2,3);
[ch30,ch3]=ConvertZCoeffsToHCoeffs(c30,c3,3);
[ch40,ch4]=ConvertZCoeffsToHCoeffs(c40,c4,3);
[ch50,ch5]=ConvertZCoeffsToHCoeffs(c50,c5,3);
[ch60,ch6]=ConvertZCoeffsToHCoeffs(c60,c6,3);
[ch70,ch7]=ConvertZCoeffsToHCoeffs(c70,c7,3);

%In what follows, I will calculate correlations of first three hermites
%between each of the six tech stocks and Nasdaq100 index. And finally I
%will draw scatter-plots of joint observations of daily log-returns of each of the
%tech stocks against Nasdaq100 index returns. And then construct the
%conditional regression based graph of each of the tech stock returns VS
%Nasdaq100 returns.

%Below calculate correlations of first three hermites between APPL
%log-returns and Nasdaq100 log rturns.
rho17_1=sum(ZAppl.*ZNasdaq)/sqrt(sum(ZAppl.^2).*sum(ZNasdaq.^2))
rho17_2=sum((ZAppl.^2-1).*(ZNasdaq.^2-1))/sqrt(sum((ZAppl.^2-1).^2).*sum(((ZNasdaq.^2)-1).^2))
rho17_3=sum((ZAppl.^3-3*ZAppl).*(ZNasdaq.^3-3*ZNasdaq))/sqrt(sum((ZAppl.^3-3.*ZAppl).^2).*sum((ZNasdaq.^3-3*ZNasdaq).^2))

%Below calculate correlations of first three hermites between AMZN
%log-returns and Nasdaq100 log rturns.
rho27_1=sum(ZAmzn.*ZNasdaq)/sqrt(sum(ZAmzn.^2).*sum(ZNasdaq.^2))
rho27_2=sum((ZAmzn.^2-1).*(ZNasdaq.^2-1))/sqrt(sum((ZAmzn.^2-1).^2).*sum(((ZNasdaq.^2)-1).^2))
rho27_3=sum((ZAmzn.^3-3*ZAmzn).*(ZNasdaq.^3-3*ZNasdaq))/sqrt(sum((ZAmzn.^3-3.*ZAmzn).^2).*sum((ZNasdaq.^3-3*ZNasdaq).^2))

%Below calculate correlations of first three hermites between Googl
%log-returns and Nasdaq100 log rturns.
rho37_1=sum(ZGoogl.*ZNasdaq)/sqrt(sum(ZGoogl.^2).*sum(ZNasdaq.^2))
rho37_2=sum((ZGoogl.^2-1).*(ZNasdaq.^2-1))/sqrt(sum((ZGoogl.^2-1).^2).*sum(((ZNasdaq.^2)-1).^2))
rho37_3=sum((ZGoogl.^3-3*ZGoogl).*(ZNasdaq.^3-3*ZNasdaq))/sqrt(sum((ZGoogl.^3-3.*ZGoogl).^2).*sum((ZNasdaq.^3-3*ZNasdaq).^2))

%Below calculate correlations of first three hermites between META
%log-returns and Nasdaq100 log rturns.

rho47_1=sum(ZMeta.*ZNasdaq)/sqrt(sum(ZMeta.^2).*sum(ZNasdaq.^2))
rho47_2=sum((ZMeta.^2-1).*(ZNasdaq.^2-1))/sqrt(sum((ZMeta.^2-1).^2).*sum(((ZNasdaq.^2)-1).^2))
rho47_3=sum((ZMeta.^3-3*ZMeta).*(ZNasdaq.^3-3*ZNasdaq))/sqrt(sum((ZMeta.^3-3.*ZMeta).^2).*sum((ZNasdaq.^3-3*ZNasdaq).^2))



%Below calculate correlations of first three hermites between MSFT
%log-returns and Nasdaq100 log rturns.

rho57_1=sum(ZMsft.*ZNasdaq)/sqrt(sum(ZMsft.^2).*sum(ZNasdaq.^2))
rho57_2=sum((ZMsft.^2-1).*(ZNasdaq.^2-1))/sqrt(sum((ZMsft.^2-1).^2).*sum(((ZNasdaq.^2)-1).^2))
rho57_3=sum((ZMsft.^3-3*ZMsft).*(ZNasdaq.^3-3*ZNasdaq))/sqrt(sum((ZMsft.^3-3.*ZMsft).^2).*sum((ZNasdaq.^3-3*ZNasdaq).^2))

%Below calculate correlations of first three hermites between NVDA
%log-returns and Nasdaq100 log rturns.ays0-1).^3-3*ZNasdaq(1:NDays0-1)))/sqrt(sum((ZMsft(2:NDays0).^3-3.*ZMsft(2:NDays0)).^2).*sum((ZNasdaq(1:NDays0-1).^3-3*ZNasdaq(1:NDays0-1)).^2))

rho67_1=sum(ZNvda.*ZNasdaq)/sqrt(sum(ZNvda.^2).*sum(ZNasdaq.^2))
rho67_2=sum((ZNvda.^2-1).*(ZNasdaq.^2-1))/sqrt(sum((ZNvda.^2-1).^2).*sum(((ZNasdaq.^2)-1).^2))
rho67_3=sum((ZNvda.^3-3*ZNvda).*(ZNasdaq.^3-3*ZNasdaq))/sqrt(sum((ZNvda.^3-3.*ZNvda).^2).*sum((ZNasdaq.^3-3*ZNasdaq).^2))


%Below do calculations to do regressions with daily log-returns of each
%tech stock as dependent variables and Nasdaq100 as independent variable.

dNn=.1/2;   % Normal density subdivisions width. would change with number of subdivisions
Nn=45*4;  % No of normal density subdivisions
NnMid=((1+Nn)/2)*dNn;
ZNasdaq0(1:Nn)=(((1:Nn)*dNn)-NnMid);

%Below reconstruct the Z-series representation of returns of each of the
%tech stock as a function of ZNasdaq, i.e the Z that underlies Nasdaq log
%returns. This is regression of each tech stock log-returns on log-returns of Nasdaq 100. 
XAppl(1:Nn)=(ch10)+rho17_1.*(ch1(1))*ZNasdaq0(1:Nn)+rho17_2.*ch1(2).*(ZNasdaq0(1:Nn).^2-1)+rho17_3.*ch1(3).*(ZNasdaq0(1:Nn).^3-3*ZNasdaq0(1:Nn));
XAmzn(1:Nn)=(ch20)+rho27_1.*(ch2(1))*ZNasdaq0(1:Nn)+rho27_2.*ch2(2).*(ZNasdaq0(1:Nn).^2-1)+rho27_3.*ch2(3).*(ZNasdaq0(1:Nn).^3-3*ZNasdaq0(1:Nn));
XGoogl(1:Nn)=(ch30)+rho37_1.*(ch3(1))*ZNasdaq0(1:Nn)+rho37_2.*ch3(2).*(ZNasdaq0(1:Nn).^2-1)+rho37_3.*ch3(3).*(ZNasdaq0(1:Nn).^3-3*ZNasdaq0(1:Nn));
XMeta(1:Nn)=(ch40)+rho47_1.*(ch4(1))*ZNasdaq0(1:Nn)+rho47_2.*ch4(2).*(ZNasdaq0(1:Nn).^2-1)+rho47_3.*ch4(3).*(ZNasdaq0(1:Nn).^3-3*ZNasdaq0(1:Nn));
XMsft(1:Nn)=(ch50)+rho57_1.*(ch5(1))*ZNasdaq0(1:Nn)+rho57_2.*ch5(2).*(ZNasdaq0(1:Nn).^2-1)+rho57_3.*ch5(3).*(ZNasdaq0(1:Nn).^3-3*ZNasdaq0(1:Nn));
XNvda(1:Nn)=(ch60)+rho67_1.*(ch6(1))*ZNasdaq0(1:Nn)+rho67_2.*ch6(2).*(ZNasdaq0(1:Nn).^2-1)+rho67_3.*ch6(3).*(ZNasdaq0(1:Nn).^3-3*ZNasdaq0(1:Nn));
XNasdaq(1:Nn)=ch70+ch7(1)*ZNasdaq0(1:Nn)+ch7(2).*(ZNasdaq0(1:Nn).^2-1)+ch7(3).*(ZNasdaq0(1:Nn).^3-3*ZNasdaq0(1:Nn))



clf
hold on

scatter(DNasdaqPrices(1:NDays0),DApplPrices(1:NDays0))
plot(XNasdaq(1:Nn),XAppl(1:Nn),'g')
title('Regression of daily log returns of Apple on Daily log-returns of NASDAQ 100'); 
str=input('Look at scatterplot and regression between Apple daily log returns and Nasdaq100 daily log-returns');

clf
hold on

scatter(DNasdaqPrices(1:NDays0),DAmznPrices(1:NDays0))
plot(XNasdaq(1:Nn),XAmzn(1:Nn),'g')
title('Regression of daily log returns of Amazon on Daily log-returns of NASDAQ 100');
str=input('Look at scatterplot and regression between Amazon daily log returns and Nasdaq100 daily log-returns');

clf
hold on

scatter(DNasdaqPrices(1:NDays0),DGooglPrices(1:NDays0))
plot(XNasdaq(1:Nn),XGoogl(1:Nn),'g')
title('Regression of daily log returns of Google on Daily log-returns of NASDAQ 100');
str=input('Look at scatterplot and regression between Google daily log returns and Nasdaq100 daily log-returns');

clf
hold on

scatter(DNasdaqPrices(1:NDays0),DMetaPrices(1:NDays0))
plot(XNasdaq(1:Nn),XMeta(1:Nn),'g')
title('Regression of daily log returns of Meta on Daily log-returns of NASDAQ 100');

str=input('Look at scatterplot and regression between Meta daily log returns and Nasdaq100 daily log-returns');

clf
hold on

scatter(DNasdaqPrices(1:NDays0),DMsftPrices(1:NDays0))
plot(XNasdaq(1:Nn),XMsft(1:Nn),'g')
title('Regression of daily log returns of Microsoft on Daily log-returns of NASDAQ 100');
str=input('Look at scatterplot and regression between Microsoft daily log returns and Nasdaq100 daily log-returns');

clf
hold on

scatter(DNasdaqPrices(1:NDays0),DNvdaPrices(1:NDays0))
plot(XNasdaq(1:Nn),XNvda(1:Nn),'g')
title('Regression of daily log returns of Nvidia on Daily log-returns of NASDAQ 100');
str=input('Look at scatterplot and regression between Nvidia daily log returns and Nasdaq100 daily log-returns');

clf
end

.
.
.
function [rMu] = CalculateFourRawMomentsFromData(DPrices,NDays0)

rMu(1)=sum(DPrices)/NDays0;
rMu(2)=sum(DPrices.^2)/NDays0;
rMu(3)=sum(DPrices.^3)/NDays0;
rMu(4)=sum(DPrices.^4)/NDays0;

end

.
.
.
function [c0,c] = CalculateZSeriesDensityFromRawMomentsM4(rMu)

    mOrder=4;
    [Mu1,cMu] = ConvertRawMomentsToCentralMoments(rMu,mOrder);

    sMu(1)=0;
    sMu(2)=1;
    sMu(3)=cMu(3)/cMu(2).^1.5;
    sMu(4)=cMu(4)/cMu(2).^2.0;
    %sMu(5)=cMu(5)/cMu(2).^2.5;
    %sMu(6)=cMu(6)/cMu(2).^3.0;
    %sMu(7)=cMu(7)/cMu(2).^3.5;
    %sMu(8)=cMu(8)/cMu(2).^4.0;
    
    
    
    iter=20;
    bGuess(1:3)=0;
    bGuess(1)=1;
    [c0,c] = PreSmoothingGuessAdvancedFromGuessBestNewC3(sMu,bGuess,iter);
    

    
    SeriesOrder=3;
    NMoments=4;    
    
    [F,dF] = CalculateMomentsAndDerivatives_0(sMu,c0,c,SeriesOrder,SeriesOrder,NMoments);

da(1,1)=c0;
da(2:SeriesOrder+1,1)=c(1:SeriesOrder);

[Moments] = CalculateMomentsOfZSeries(c0,c,SeriesOrder,NMoments);

%Replace with your own more intelligent objective function if you like.
ObjBest=100000*(abs(sMu(1)-Moments(1)))+abs(sMu(2)-Moments(2))+abs((sMu(3)-Moments(3))^(1/1.5))+abs((sMu(4)-Moments(4))^(1/2.0));

b0Best=c0;
bBest(1:SeriesOrder)=c(1:SeriesOrder);

nn=0;
while((nn<20)&&((abs(F(1,1))>.000000000001) || (abs(F(2,1))>.000000000001) || (abs(F(3,1))>.000000000001) || (abs(F(4,1))>.00000000001)  ))

    nn=nn+1;
    %Below Newton matrix equation to improve the Z-series coefficients guess at previous step.
    da=da-dF\F;

    %b0=median
    b0=da(1,1);
    b(1:SeriesOrder)=da(2:SeriesOrder+1,1);

    %[F,dF] = CalculateCumulantsAndDerivativesFromMoments_0(C,b0,b,SeriesOrder,SeriesOrder,NoOfCumulants);
    [F,dF] = CalculateMomentsAndDerivatives_0(sMu,b0,b,SeriesOrder,SeriesOrder,NMoments);
    [IsValidFlag] =1;% CheckIsValidDensity(b0,b);
    [Moments] = CalculateMomentsOfZSeries(b0,b,SeriesOrder,NMoments);

    ObjNew=100000*(abs(sMu(1)-Moments(1)))+abs(sMu(2)-Moments(2))+abs((sMu(3)-Moments(3))^(1/1.5))+abs((sMu(4)-Moments(4))^(1/2.0));
  
    if((ObjBest>ObjNew) &&( IsValidFlag))
      
       ObjBest=ObjNew;
       b0Best=b0;
       bBest(1:SeriesOrder)=b(1:SeriesOrder);
    end

    da(1,1)=b0;
    da(2:SeriesOrder+1,1)=b(1:SeriesOrder);

end
c0=b0Best;%Best;
c(1:SeriesOrder)=bBest;%Best(1:SeriesOrder);
    
    
    
    c0=c0*sqrt(cMu(2));
    c=c*sqrt(cMu(2));
    
    c0=c0+Mu1;

[FittedMoments] = CalculateMomentsOfZSeries(c0,c,SeriesOrder,NMoments);
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%Please Uncomment this if you want to see a comparison of input moments and
%fitted moments
rMu
FittedMoments
str=input('Look at comparison of fitted moments with input moments');
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%5
%COMMENT lines below if you do not want to see the density of new Z-series random variable with
%fitted moments.
%First calculate normal random variable on a grid below
dNn=.1/2;   % Normal density subdivisions width. would change with number of subdivisions
Nn=45*4;  % No of normal density subdivisions
NnMid=((1+Nn)/2)*dNn;
Z(1:Nn)=(((1:Nn)*dNn)-NnMid);
%Z
%str=input('Look at Z');
%Now calculate Y as a function of normal random variable.
Y(1:Nn)=c0;
for nn=1:3
    Y(1:Nn)=Y(1:Nn)+c(nn)*Z(1:Nn).^nn;
end

%Now take change of densities derivative of Y with respect to normal
DfY(1:Nn)=0;
for nn=2:Nn-1
    DfY(nn) = (Y(nn + 1) - Y(nn - 1))/(Z(nn + 1) - Z(nn - 1));
    %Change of variable derivative for densities
end
DfY(Nn)=DfY(Nn-1);
DfY(1)=DfY(2);

%Now calculate the density of Y from density of normal random variable
%using change of probability derivative.
pY(1:Nn)=0;
for nn = 1:Nn    
    pY(nn) = (normpdf(Z(nn),0, 1))/abs(DfY(nn));
end

plot(Y(1:Nn),pY(1:Nn),'b');
title('Density of the Z-Series Variable with Fitted Moments');

%str=input('Look at the density of Z-series random variable')
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

end

.
.
.
function [mu1,cMu] = ConvertRawMomentsToCentralMoments(Mu,mOrder)

%cu
%mu
% %mOrder
%str=input('Look at numbers');
mu1=Mu(1);
for mm=2:mOrder
    cMu(mm)=0;
    for jj=0:mm
        if(jj==0)
            cMu(mm)=cMu(mm)+ (-1)^(mm-jj).*factorial(mm)/factorial(jj)/factorial(mm-jj)*1*mu1.^(mm-jj);
        else
            cMu(mm)=cMu(mm)+ (-1)^(mm-jj).*factorial(mm)/factorial(jj)/factorial(mm-jj)*Mu(jj)*mu1.^(mm-jj);
        end
    end
end

.
.
.
function [c0,c] = PreSmoothingGuessAdvancedFromGuessBestNewC3(cmu,cin,iter)


%Mul=1.0;
SeriesOrder=3;
NMoments=4;

 EZ(1)=0;
 EZ(2)=1;
 for nn=3:NMoments*SeriesOrder+SeriesOrder+2
     if rem(nn,2)==1
         EZ(nn)=0;
     else
         EZ(nn)=EZ(nn-2)*(nn-1);
         EZ(nn);
     end
 end


c0=-(cin(2));  %This is for zero mean condition.
c=cin;
MaxIter=10;

for nn=1:iter     %increase the iterations over coefficients if neeeded


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

[M3,dc2] = CalculateParticularMomentAndDerivativeOfItsCoeff(c0,c,SeriesOrder,3,EZ);

tolerance=.0000001*cmu(3);
mm=0;
while ((abs(M3-cmu(3)) >tolerance ) && (mm<MaxIter))
mm=mm+1;
c(2)=c(2)-(M3-cmu(3))/dc2;

[M3,dc2] = CalculateParticularMomentAndDerivativeOfItsCoeff(c0,c,SeriesOrder,3,EZ);

end

c0=-(c(2));

[SecondMoment] = CalculateSecondMomentC3(c0,c);

c=c/sqrt(SecondMoment);
c0=c0/sqrt(SecondMoment);

%[c0,c] = IterateSmoothingGuessOverPreviousMoments(cmu,c,3,2,EZ);

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%5
[M4,dc3] = CalculateParticularMomentAndDerivativeOfItsCoeff(c0,c,SeriesOrder,4,EZ);

tolerance=.0000001*cmu(4);
mm=0;
while ((abs(M4-cmu(4)) >tolerance ) && (mm<MaxIter))
mm=mm+1;
c(3)=c(3)-(M4-cmu(4))/dc3;

[M4,dc3] = CalculateParticularMomentAndDerivativeOfItsCoeff(c0,c,SeriesOrder,4,EZ);

end

[SecondMoment] = CalculateSecondMomentC3(c0,c);

c=c/sqrt(SecondMoment);
c0=c0/sqrt(SecondMoment);


%[c0,c] = IterateSmoothingGuessOverPreviousMoments(cmu,c,4,2,EZ);


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%


end


end

.
.
.
function [EnMoment,dEnMomentdCoeff] = CalculateParticularMomentAndDerivativeOfItsCoeff(a0,a,SeriesOrder,nMoment,EZ)


% if(SeriesOrder>=8)
% a0=-(a(2)+3*a(4)+15*a(6)+105*a(8));% ---1
% end
% if(SeriesOrder<8)
% a0=-(a(2)+3*a(4)+15*a(6));% ---1
% end
% 
% EZ(1)=0;
% EZ(2)=1;
% for nn=3:NMoments*SeriesOrder+NZterms+2
%     if rem(nn,2)==1
%         EZ(nn)=0;
%     else
%         EZ(nn)=EZ(nn-2)*(nn-1);
%         EZ(nn);
%     end
% end
% EZ;

%EXZ(1,1)=1;
%for pp1=1:NZterms
%        EXZ(1,pp1+1)=EZ(pp1);
%end


a(SeriesOrder+1:nMoment*SeriesOrder+1+SeriesOrder)=0;
b0=a0;
b=a;

for mm=2:nMoment
    
    [b0,b] =SeriesProduct(a0,a,b0,b,SeriesOrder*mm+SeriesOrder+1);
     b(SeriesOrder*mm+1:nMoment*SeriesOrder+SeriesOrder+1)=0;
     if(mm==nMoment-1)
         dEnMomentdCoeff=b0.*EZ(nMoment-1);
         for pp2=1:SeriesOrder*nMoment-1
            
            dEnMomentdCoeff=dEnMomentdCoeff+b(pp2).*EZ(pp2+nMoment-1);
        end
        dEnMomentdCoeff=dEnMomentdCoeff*nMoment; 
     end
     if(mm==nMoment)
        EnMoment=b0;
        for pp2=1:SeriesOrder*nMoment
            EnMoment=EnMoment+b(pp2).*EZ(pp2);
    
        end
     end
end   

    

end

.
.
.
function [c0,c] =SeriesProduct(a0,a,b0,b,SeriesOrder)


c0=a0*b0;
c(1:SeriesOrder)=0.0;
for nn=1:SeriesOrder
    c(nn)=c(nn)+a0*b(nn);
    c(nn)=c(nn)+b0*a(nn);
    for kk=1:nn-1
        c(nn)=c(nn)+a(kk)*b(nn-kk);
    end
end

end

.
.
.
function [SecondMoment] = CalculateSecondMomentC3(a0,a)


SecondMoment=a0^2+a(1)^2 +2* a0* a(2) +3*(a(2).^2+2* a(1).* a(3))+ ...
    15*(a(3).^2) ;

end

.
.
.
function [F,dF] = CalculateMomentsAndDerivatives_0(Ms,a0,a,SeriesOrder,NZterms,NMoments)
%[F,dF] = CalculateCumulantsAndDerivativesFromMoments(C,a0,a,SeriesOrder,SeriesOrder,NoOfCumulants);

%a(1)=a1;
%a(2)=a2;
%a(3)=a3;
%a(4)=a4;
%a(5)=a5;
%a(6)=a6;
%a(7)=a7;



%aa0=a0;
if(NMoments>8)
a0=-(a(2)+3*a(4)+15*a(6)+105*a(8));% ---1
end
if(NMoments==8)
a0=-(a(2)+3*a(4)+15*a(6));% ---1
end
if(NMoments==6)
a0=-(a(2)+3*a(4));% ---1
end

EZ(1)=0;
EZ(2)=1;
for nn=3:NMoments*SeriesOrder+NZterms+2
    if rem(nn,2)==1
        EZ(nn)=0;
    else
        EZ(nn)=EZ(nn-2)*(nn-1);
        EZ(nn);
    end
end
EZ;

EXZ(1,1)=1;
for pp1=1:NZterms
        EXZ(1,pp1+1)=EZ(pp1);
end


a(SeriesOrder+1:NMoments*SeriesOrder+1)=0;
b0=a0;
b=a;

for mm=1:NMoments
    if(mm>1)
        [b0,b] =SeriesProduct(a0,a,b0,b,SeriesOrder*mm);
        b(SeriesOrder*mm+1:NMoments*SeriesOrder+1)=0;
    end
   % b0
   % b
%str=input('Look at numbers')    
    EXZ(mm+1,1)=b0;
    for pp2=1:SeriesOrder*mm
        
            EXZ(mm+1,1)=EXZ(mm+1,1)+b(pp2).*EZ(pp2);
    end
    for pp1=1:NZterms
        EXZ(mm+1,pp1+1)=b0.*EZ(pp1);
        for pp2=1:SeriesOrder*mm
        
            EXZ(mm+1,pp1+1)=EXZ(mm+1,pp1+1)+b(pp2).*EZ(pp2+pp1);
        end
    end
end

%u1=EXZ(2,1);
u2=EXZ(3,1);
u3=EXZ(4,1);
u4=EXZ(5,1);

u1=a0+a(2);
if(SeriesOrder>=4)
u1=a0+a(2)+3*a(4);
end
if(SeriesOrder>=6)
u1=u1+15*a(6);
end
if(SeriesOrder>=8)
u1=u1+105*a(8);
end


%k2=u2-u1^2;
%k3=u3-3*u2*u1+2*u1^3;
%k4=u4-4*u3*u1-3*u2^2+12*u2*u1^2-6*u1^4;


du1(1)=1;%----2
% du1(2)=0;%----2
% du1(3)=1;%----2
% du1(4)=0;%----2
% du1(5)=1;%----2
% du1(6)=0;%----2
% du1(7)=15;%----2
% du1(8)=0;%----2
% du1(9)=105;%----2
% du1(10)=0;%----2
 du2(1)=2*EXZ(2,1);
 du3(1)=3*EXZ(3,1);
 du4(1)=4*EXZ(4,1);


%du1(1)=1;%----2
%du2(1)=0;
%du3(1)=0;
%du4(1)=0;

for mm=2:SeriesOrder+1
du1(mm)=EXZ(1,mm);
du2(mm)=2*EXZ(2,mm);
du3(mm)=3*EXZ(3,mm);
du4(mm)=4*EXZ(4,mm);

end


if(NMoments>=5)
u5=EXZ(6,1);


%du5(1)=5*EXZ(5,1);

for mm=1:SeriesOrder+1
du5(mm)=5*EXZ(5,mm);
end

end

if(NMoments>=6)
u6=EXZ(7,1);
%du6(1)=0;

for mm=1:SeriesOrder+1
du6(mm)=6*EXZ(6,mm);
end

end


if(NMoments>=7)
u7=EXZ(8,1);
%str=input('Look at k7 and k71')

%du7(1)=0;

for mm=1:SeriesOrder+1
du7(mm)=7*EXZ(7,mm);
end

end
if(NMoments>=8)

u8=EXZ(9,1);
for mm=1:SeriesOrder+1
du8(mm)=8*EXZ(8,mm);
end

end

if(NMoments>=9)

u9=EXZ(10,1);
for mm=1:SeriesOrder+1
du9(mm)=9*EXZ(9,mm);
end

end
if(NMoments>=10)

u10=EXZ(11,1);
for mm=1:SeriesOrder+1
du10(mm)=10*EXZ(10,mm);
end

end




 F(1,1)=u1-Ms(1);
 F(2,1)=u2-Ms(2);
 F(3,1)=u3-Ms(3);
 F(4,1)=u4-Ms(4);
 if(NMoments>=5)
 F(5,1)=u5-Ms(5);
 end
 if(NMoments>=6)
 F(6,1)=u6-Ms(6);
 end
 if(NMoments>=7)
 F(7,1)=u7-Ms(7);
 end
 if(NMoments>=8)
 F(8,1)=u8-Ms(8);
 end
 if(NMoments>=9)
 F(9,1)=u9-Ms(9);
 end
 if(NMoments>=10)
 F(10,1)=u10-Ms(10);
 end
 
     for mm=1:SeriesOrder+1
        dF(1,mm)=du1(mm);
        dF(2,mm)=du2(mm);
        dF(3,mm)=du3(mm);
        dF(4,mm)=du4(mm);
        if(NMoments>=5)
        dF(5,mm)=du5(mm);
        end
        if(NMoments>=6)
        dF(6,mm)=du6(mm);
        end
        if(NMoments>=7)
        dF(7,mm)=du7(mm);
        end
        if(NMoments>=8)
        dF(8,mm)=du8(mm);
        end
        if(NMoments>=9)
        dF(9,mm)=du9(mm);
        end
        if(NMoments>=10)
        dF(10,mm)=du10(mm);
        end
     end

end

.
.
.
function [Moments] = CalculateMomentsOfZSeries(a0,a,SeriesOrder,NMoments)


%aa0=a0;
%a0=0;% ---1

EZ(1)=0;
EZ(2)=1;
%LogEZ(2)=0;
for nn=3:NMoments*SeriesOrder
    if rem(nn,2)==1
        EZ(nn)=0;
    else
        EZ(nn)=EZ(nn-2)*(nn-1);
        %LogEZ(nn)=log(EZ(nn-2))+log(nn-1);
        %EZ(nn);
    end
end
%EZ
%EZ(1:30)
%str=input('Look at numbers')
a(SeriesOrder+1:SeriesOrder*NMoments+1)=0;
b0=a0;
b=a;


for mm=1:NMoments
    if(mm>1)
        %[b0,b] =SeriesProductLogarithmic(a0,a,b0,b,SeriesOrder*mm);
        
        [b0,b] =SeriesProduct(a0,a,b0,b,SeriesOrder*mm);
        
        %b0-bb0
        
        %b-bb
        
        %str=input('Look at two products')
        
        
        b(SeriesOrder*mm+1:SeriesOrder*NMoments+1)=0;
    end
   % Logb=log(abs(b));
   % Signb=sign(b);
    EXZ(mm,1)=b0;
    for pp2=1:SeriesOrder*mm
        if rem(pp2,2)==0
            %EXZ(mm,1)=EXZ(mm,1)+exp(Logb(pp2)+LogEZ(pp2)).*Signb(pp2);
            EXZ(mm,1)=EXZ(mm,1)+b(pp2).*EZ(pp2);
 %           b(pp2).*EZ(pp2)
 %           exp(Logb(pp2)+LogEZ(pp2)).*Signb(pp2)
 %           mm
 %           pp2
            %str=input('Look at moment values')
        end
    end
end

    
for nn=1:NMoments
    Moments(nn)=EXZ(nn,1);
end



end

.
.
.
function [Z] = CalculateZgivenXAndZSeriesC3(X,c0,c)


Z=(X-c0)/c(1)-(c(2) * (X-c0).^2)/c(1).^3+(((2* c(2).^2)./c(1).^2-c(3)./c(1)).* (X-c0).^3)./c(1).^3;

for nn=1:30

    X0=c0+c(1)*Z+c(2)*Z.^2+c(3).*Z.^3;
    dX0dZ=c(1)+2.*c(2)*Z+3.*c(3).*Z.^2;
    Z=Z-(X0-X)./(dX0dZ);
end



end

.
.
.
function [aH0,aH] = ConvertZCoeffsToHCoeffs(a0,a,SeriesOrder)

if(SeriesOrder==3)
aH0=a0+a(2);
aH(1)=a(1)+3*a(3);
aH(2)=a(2);
aH(3)=a(3);
end


if(SeriesOrder==4)
aH0=a0+a(2)+3*a(4);
aH(1)=a(1)+3*a(3);
aH(2)=a(2)+6*a(4);
aH(3)=a(3);
aH(4)=a(4);
end


if(SeriesOrder==5)
aH0=a0+a(2)+3*a(4);
aH(1)=a(1)+3*a(3)+15*a(5);
aH(2)=a(2)+6*a(4);
aH(3)=a(3)+10*a(5);
aH(4)=a(4);
aH(5)=a(5);
end



if(SeriesOrder==6)
aH0=a0+a(2)+3*a(4)+15*a(6);
aH(1)=a(1)+3*a(3)+15*a(5);
aH(2)=a(2)+6*a(4)+45*a(6);
aH(3)=a(3)+10*a(5);
aH(4)=a(4)+15*a(6);
aH(5)=a(5);
aH(6)=a(6);
end


if(SeriesOrder==7)
aH0=a0+a(2)+3*a(4)+15*a(6);
aH(1)=a(1)+3*a(3)+15*a(5)+105*a(7);
aH(2)=a(2)+6*a(4)+45*a(6);
aH(3)=a(3)+10*a(5)+105*a(7);
aH(4)=a(4)+15*a(6);
aH(5)=a(5)+21*a(7);
aH(6)=a(6);
aH(7)=a(7);
end



end

.
.
.
You think life is a secret, Life is only love of flying, It has seen many ups and downs, But it likes travel more than the destination. Allama Iqbal
 
User avatar
Amin
Topic Author
Posts: 2077
Joined: July 14th, 2002, 3:00 am

Re: Breakthrough in the theory of stochastic differential equations and their simulation

February 13th, 2023, 1:14 pm

Sorry friends, I forgot to attach the tech stocks data and nasdaq 100 data. Here they are in this small zip file. These are csv files. I downloaded this data yesterday from WSJ web site.
CSVDataFiles.zip
(43.32 KiB) Downloaded 21 times
I have some other very interesting ideas about calculation of dynamic correlations that prevail for reasonably small periods of time. I will come back with a detailed post in a day. 
For example, there is huge correlation between Nasdaq 100 and all stocks when calculated over a period of more than one year. However one period ahead serial correlations when calculated over the period of more than a year was practically zero. However there were smaller stretches of data when one period ahead serial correlations were highly significant and that led me to ask myself how to be able to infer dynamic serial correlations from smaller stretches of data in a reasonable way. I will do my experiments and come back with a more detailed post soon and a program in a few days.
You think life is a secret, Life is only love of flying, It has seen many ups and downs, But it likes travel more than the destination. Allama Iqbal
 
User avatar
Amin
Topic Author
Posts: 2077
Joined: July 14th, 2002, 3:00 am

Re: Breakthrough in the theory of stochastic differential equations and their simulation

February 14th, 2023, 10:59 am

Friends, I continued to think about calculation of dynamic correlations and want to share my ideas with friends.
First please let me explain my framework. I will explain calculation of correlation between two stocks but it can easily be generalized to calculation of dynamic correlations between a portfolio of stocks based on their joint time series. I suppose that we are given diagonal variances of both stocks independently. For example we may independently calculate the diagonal variances from several months of data but we want to calculate correlations only from data of past one week or ten or fifteen days. In other words, we suppose that correlations are more local and changing more quickly. If there is some structure in diagonal variances as we have in stochastic volatility, we suppose that this effect has been independently calculated for both diagonal variances before we want to calculate the dynamic correlations over shorter stretches. So first observation is that we have decoupled calculations of  diagonal variances and calculation of dynamic correlations and diagonal variances that enter the calculation of dynamic correlations have already been calculated.

First some background on two variable regressions. Suppose, I am regressing Amazon stock on Nasdaq 100 index.
Variance of Nasdaq 100 = [$]{\sigma_{N}}^2[$]
Variance of Amazon = [$]{\sigma_{A}}^2[$]
Correlation between Amazon and Nasdaq = [$]\rho_{N,A}[$]
Covariance between Amazon and Nasdaq= [$]\rho_{N,A} \, \sigma_{N} \,\sigma_{A}[$]

We know from regression lemma that (please note that variables with a bar are estimated values and variables without bar are observed values)

[$]\bar{X_A} \, = \frac{Cov_{N,A}}{Var_N} \, (X_N \, - \mu_{N})[$]
[$] \, = \frac{\rho_{N,A} \, \sigma_{N} \,\sigma_{A}}{{\sigma_{N}}^2} \, (X_N \, - \mu_{N})[$]
[$] \, = \frac{\rho_{N,A} \, \,\sigma_{A}}{\sigma_{N}} \, (X_N \, - \mu_{N})[$]

Incidentally we can further simplify above in context of our Z-series framework, this time with only one polynomial by noting that 
[$] X_N \, - \mu_{N} \, = \,  \sigma_{N} \, Z_N[$] and replacing it in last line of above equation block as

[$]  \bar{X_A} \, \, = \frac{\rho_{N,A} \, \,\sigma_{A}}{\sigma_{N}} \, ( \sigma_{N} \, Z_N)[$]
[$] \, = \rho_{N,A} \, \,\sigma_{A} \,  \, Z_N[$] 

I added last few lines because this was basis for my hermite polynomial based regression program that I uploaded yesterday (here with just one polynomial).
Coming back to classic form, we have
[$] \bar{X_A} \,\, = \frac{\rho_{N,A} \, \,\sigma_{A}}{\sigma_{N}} \, (X_N \, - \mu_{N})[$]

We notice that observed value of [$]X_A[$] is equal to predicted from regression value of [$]\bar{X_A}[$] and a residual [$]{R_A}[$] given as
[$]X_A \, = \, \bar{X_A} \, + \, {R_A}[$]

We want to calculate a dynamic correlation so that over the small stretch of data (in which we are calculating dynamic correlation), our predicted values are maximized and residual values are minimized. This is done by minimizing the sum of squared residuals. This minimization from regressions is made possible since we had made calculations of diagonal variances exogenously from a possibly larger set of data.

Doing analytics, we get sum of squared residuals (over the time interval that is relevant to us for calculation of dynamic correlations) as
[$] \sum \, {R_A}^2 \, = \sum {(\, X_A \, - \, \bar{X_A})}^2 \,[$]
[$] \sum \, {R_A}^2 \, = \sum {(\, X_A \, - \, \big[ \frac{\rho_{N,A} \, \,\sigma_{A}}{\sigma_{N}} \, (X_N \, - \mu_{N}) \big])}^2 \,[$]

All the values on rhs have either been observed or calculated earlier from data independently. So we minimize the sum of residuals with respect to correlation parameter and apply the first order condition as

[$] \frac{\partial [\sum \, {R_A}^2]}{\partial \rho_{N,A}} \, =\frac{\partial \big[ \sum {(\, X_A \, - \, \big[ \frac{\rho_{N,A} \, \,\sigma_{A}}{\sigma_{N}} \, (X_N \, - \mu_{N}) \big])}^2 \,\big]}{\partial \rho_{N,A}}=0[$]

Solution of the above FOC would give the dynamic correlation that maximizes the predicted value of Amazon stock over the short stretch of data and minimizes the sum of squared residuals. 
You can also apply above analytics to all mutual dynamic correlations in a basket of stocks by changing as in modelling a multivariable regression(with vectors and matrices) as opposed to single variable regression.

I believe that minimization of squared residuals is underemphasized and can also be used intelligently in changepoint detection and regime change detection. If you can cast your model in a two regime framework or changepoint framework and apply minimization of sum of squared residuals, the hypothesis of regime change can easily be accepted if there is a substantial decrease in sum of squared residuals due to regime change applied to the model.
You think life is a secret, Life is only love of flying, It has seen many ups and downs, But it likes travel more than the destination. Allama Iqbal
 
User avatar
Amin
Topic Author
Posts: 2077
Joined: July 14th, 2002, 3:00 am

Re: Breakthrough in the theory of stochastic differential equations and their simulation

February 14th, 2023, 5:18 pm

Friends, I continued to think about calculation of dynamic correlations and want to share my ideas with friends.
First please let me explain my framework. I will explain calculation of correlation between two stocks but it can easily be generalized to calculation of dynamic correlations between a portfolio of stocks based on their joint time series. I suppose that we are given diagonal variances of both stocks independently. For example we may independently calculate the diagonal variances from several months of data but we want to calculate correlations only from data of past one week or ten or fifteen days. In other words, we suppose that correlations are more local and changing more quickly. If there is some structure in diagonal variances as we have in stochastic volatility, we suppose that this effect has been independently calculated for both diagonal variances before we want to calculate the dynamic correlations over shorter stretches. So first observation is that we have decoupled calculations of  diagonal variances and calculation of dynamic correlations and diagonal variances that enter the calculation of dynamic correlations have already been calculated.

First some background on two variable regressions. Suppose, I am regressing Amazon stock on Nasdaq 100 index.
Variance of Nasdaq 100 = [$]{\sigma_{N}}^2[$]
Variance of Amazon = [$]{\sigma_{A}}^2[$]
Correlation between Amazon and Nasdaq = [$]\rho_{N,A}[$]
Covariance between Amazon and Nasdaq= [$]\rho_{N,A} \, \sigma_{N} \,\sigma_{A}[$]

We know from regression lemma that (please note that variables with a bar are estimated values and variables without bar are observed values)

[$]\bar{X_A} \, = \frac{Cov_{N,A}}{Var_N} \, (X_N \, - \mu_{N})[$]
[$] \, = \frac{\rho_{N,A} \, \sigma_{N} \,\sigma_{A}}{{\sigma_{N}}^2} \, (X_N \, - \mu_{N})[$]
[$] \, = \frac{\rho_{N,A} \, \,\sigma_{A}}{\sigma_{N}} \, (X_N \, - \mu_{N})[$]

Incidentally we can further simplify above in context of our Z-series framework, this time with only one polynomial by noting that 
[$] X_N \, - \mu_{N} \, = \,  \sigma_{N} \, Z_N[$] and replacing it in last line of above equation block as

[$]  \bar{X_A} \, \, = \frac{\rho_{N,A} \, \,\sigma_{A}}{\sigma_{N}} \, ( \sigma_{N} \, Z_N)[$]
[$] \, = \rho_{N,A} \, \,\sigma_{A} \,  \, Z_N[$] 

I added last few lines because this was basis for my hermite polynomial based regression program that I uploaded yesterday (here with just one polynomial).
Coming back to classic form, we have
[$] \bar{X_A} \,\, = \frac{\rho_{N,A} \, \,\sigma_{A}}{\sigma_{N}} \, (X_N \, - \mu_{N})[$]

We notice that observed value of [$]X_A[$] is equal to predicted from regression value of [$]\bar{X_A}[$] and a residual [$]{R_A}[$] given as
[$]X_A \, = \, \bar{X_A} \, + \, {R_A}[$]

We want to calculate a dynamic correlation so that over the small stretch of data (in which we are calculating dynamic correlation), our predicted values are maximized and residual values are minimized. This is done by minimizing the sum of squared residuals. This minimization from regressions is made possible since we had made calculations of diagonal variances exogenously from a possibly larger set of data.

Doing analytics, we get sum of squared residuals (over the time interval that is relevant to us for calculation of dynamic correlations) as
[$] \sum \, {R_A}^2 \, = \sum {(\, X_A \, - \, \bar{X_A})}^2 \,[$]
[$] \sum \, {R_A}^2 \, = \sum {(\, X_A \, - \, \big[ \frac{\rho_{N,A} \, \,\sigma_{A}}{\sigma_{N}} \, (X_N \, - \mu_{N}) \big])}^2 \,[$]

All the values on rhs have either been observed or calculated earlier from data independently. So we minimize the sum of residuals with respect to correlation parameter and apply the first order condition as

[$] \frac{\partial [\sum \, {R_A}^2]}{\partial \rho_{N,A}} \, =\frac{\partial \big[ \sum {(\, X_A \, - \, \big[ \frac{\rho_{N,A} \, \,\sigma_{A}}{\sigma_{N}} \, (X_N \, - \mu_{N}) \big])}^2 \,\big]}{\partial \rho_{N,A}}=0[$]

Solution of the above FOC would give the dynamic correlation that maximizes the predicted value of Amazon stock over the short stretch of data and minimizes the sum of squared residuals. 
You can also apply above analytics to all mutual dynamic correlations in a basket of stocks by changing as in modelling a multivariable regression(with vectors and matrices) as opposed to single variable regression.

I believe that minimization of squared residuals is underemphasized and can also be used intelligently in changepoint detection and regime change detection. If you can cast your model in a two regime framework or changepoint framework and apply minimization of sum of squared residuals, the hypothesis of regime change can easily be accepted if there is a substantial decrease in sum of squared residuals due to regime change applied to the model.
.
Friends, there was a minor error in some equations above and I am copying corrected version of the above post below. I missed the mean of dependent variable that is Amazon stock everywhere in the equations. Below is the corrected post.
.
Friends, I continued to think about calculation of dynamic correlations and want to share my ideas with friends.
First please let me explain my framework. I will explain calculation of correlation between two stocks but it can easily be generalized to calculation of dynamic correlations between a portfolio of stocks based on their joint time series. I suppose that we are given diagonal variances of both stocks independently. For example we may independently calculate the diagonal variances from several months of data but we want to calculate correlations only from data of past one week or ten or fifteen days. In other words, we suppose that correlations are more local and changing more quickly. If there is some structure in diagonal variances as we have in stochastic volatility, we suppose that this effect has been independently calculated for both diagonal variances before we want to calculate the dynamic correlations over shorter stretches. So first observation is that we have decoupled calculations of  diagonal variances and calculation of dynamic correlations and diagonal variances that enter the calculation of dynamic correlations have already been calculated.

First some background on two variable regressions. Suppose, I am regressing Amazon stock on Nasdaq 100 index.
Variance of Nasdaq 100 = [$]{\sigma_{N}}^2[$]
Variance of Amazon = [$]{\sigma_{A}}^2[$]
Correlation between Amazon and Nasdaq = [$]\rho_{N,A}[$]
Covariance between Amazon and Nasdaq= [$]\rho_{N,A} \, \sigma_{N} \,\sigma_{A}[$]

We know from regression lemma that (please note that variables with a bar are estimated values and variables without bar are observed values)

[$]\bar{X_A} \, = \mu_{A} \, + \, \frac{Cov_{N,A}}{Var_N} \, (X_N \, - \mu_{N})[$]
[$] \, = \mu_{A} \, + \,\frac{\rho_{N,A} \, \sigma_{N} \,\sigma_{A}}{{\sigma_{N}}^2} \, (X_N \, - \mu_{N})[$]
[$] \, = \mu_{A} \, + \,\frac{\rho_{N,A} \, \,\sigma_{A}}{\sigma_{N}} \, (X_N \, - \mu_{N})[$]

Incidentally we can further simplify above in context of our Z-series framework, this time with only one polynomial by noting that 
[$] X_N \, - \mu_{N} \, = \,  \sigma_{N} \, Z_N[$] and replacing it in last line of above equation block as

[$]  \bar{X_A} \, \, = \mu_{A} \, + \,\frac{\rho_{N,A} \, \,\sigma_{A}}{\sigma_{N}} \, ( \sigma_{N} \, Z_N)[$]
[$] \, = \mu_{A} \, + \,\rho_{N,A} \, \,\sigma_{A} \,  \, Z_N[$] 

I added last few lines because this was basis for my hermite polynomial based regression program that I uploaded yesterday (here with just one polynomial).
Coming back to classic form, we have
[$] \bar{X_A} \,\, =\mu_{A} \, + \, \frac{\rho_{N,A} \, \,\sigma_{A}}{\sigma_{N}} \, (X_N \, - \mu_{N})[$]

We notice that observed value of [$]X_A[$] is equal to predicted from regression value of [$]\bar{X_A}[$] and a residual [$]{R_A}[$] given as
[$]X_A \, = \, \bar{X_A} \, + \, {R_A}[$]

We want to calculate a dynamic correlation so that over the small stretch of data (in which we are calculating dynamic correlation), our predicted values are maximized and residual values are minimized. This is done by minimizing the sum of squared residuals. This minimization from regressions is made possible since we had made calculations of diagonal variances exogenously from a possibly larger set of data.

Doing analytics, we get sum of squared residuals (over the time interval that is relevant to us for calculation of dynamic correlations) as
[$] \sum \, {R_A}^2 \, = \sum {(\, X_A \, - \, \bar{X_A})}^2 \,[$]
[$] \sum \, {R_A}^2 \, = \sum {(\, X_A \, - \, \big[ \mu_{A} \, + \,\frac{\rho_{N,A} \, \,\sigma_{A}}{\sigma_{N}} \, (X_N \, - \mu_{N}) \big])}^2 \,[$]

All the values on rhs have either been observed or calculated earlier from data independently. So we minimize the sum of residuals with respect to correlation parameter and apply the first order condition as

[$] \frac{\partial [\sum \, {R_A}^2]}{\partial \rho_{N,A}} \, =\frac{\partial \big[ \sum {(\, X_A \, - \, \big[ \mu_{A} \, + \, \frac{\rho_{N,A} \, \,\sigma_{A}}{\sigma_{N}} \, (X_N \, - \mu_{N}) \big])}^2 \,\big]}{\partial \rho_{N,A}}=0[$]

Solution of the above FOC would give the dynamic correlation that maximizes the predicted value of Amazon stock over the short stretch of data and minimizes the sum of squared residuals. 
You can also apply above analytics to all mutual dynamic correlations in a basket of stocks by changing as in modelling a multivariable regression(with vectors and matrices) as opposed to single variable regression.

I believe that minimization of squared residuals is underemphasized and can also be used intelligently in changepoint detection and regime change detection. If you can cast your model in a two regime framework or changepoint framework and apply minimization of sum of squared residuals, the hypothesis of regime change can easily be accepted if there is a substantial decrease in sum of squared residuals due to regime change applied to the data.
You think life is a secret, Life is only love of flying, It has seen many ups and downs, But it likes travel more than the destination. Allama Iqbal
 
User avatar
Amin
Topic Author
Posts: 2077
Joined: July 14th, 2002, 3:00 am

Re: Breakthrough in the theory of stochastic differential equations and their simulation

February 18th, 2023, 6:24 pm

Friends, I did not have a very good day today. Mind control agents have sharply increased mind control activity very recently. I had great difficulty thinking and concentrating and I lost threads of my thoughts again and again since mind control agents kicked important neurotransmitters out of their pathways using EM resonance. It seemed that mind control agents have a great urgency to somehow not let me continue my research. However I was still able to gain clarity about my many things after repeated efforts. But mind control agents have increased their attempts to make it difficult for me to do good research.

Here are my near future plans for research. I want to complete my programs for multivariable regressions in multiple hermite polynomial setting. And I want to do it both by using average correlations (of different hermite polynomials) and optimized correlations (of different hermite polynomials) by minimizing least squres in multivariable regression setting.
I also want to share other interesting observations with friends. I ran a simulation over the attached data(in a recent previous post) of Nasdaq tech stocks. The data stretches for little over a year and a month. I calculated skew and made a strategy to shift money in stock with the max raw third moment (of daily log returns) among six stocks and Nasdaq index. I made local calculations of third moment and you will have to play with number of days required for calculation of local values of third raw moment of log returns that optimizes the strategy (though this sensitivity is not huge which means that strategy is reasonably robust). I also took Nasdaq 100 as an asset in which you can shift money by buying the index when Nasdaq 100 log returns would have more skew than any of the other six stocks in the sample. One dollar of notional grew into  2.69 dollars of notional after 13.5 months of time with above strategy. Though all calculations were in an idealized setting, I am sharing it for friends who find it interesting and want to explore it further. The strategy probably worked well since I was picking the (locally) best stock among six very strong tech stocks and Nasdaq 100 (considered as an asset in which you can park your money)
Another interesting thing for algorithmic trading would be to look at local autocorrelation (and local serial cross correlations) of skew hermite polynomials and include that in the analysis. 
You think life is a secret, Life is only love of flying, It has seen many ups and downs, But it likes travel more than the destination. Allama Iqbal