I have suggested in a previous post that we can solve for all of the 6X6 coefficients of 2D Hermite-series from knowledge of \(B_1,\ldots\,,\,B_6\) and \(C_1,\ldots\,,\,C_6\). This is because to satisfy squared sum across both rows and columns of a 2D Hermite-series, the squared terms inside the matrix have to be in specific ratios. We can use this knowledge in other ways to make numerical methods. For example we can find coefficients \(C_1,\ldots\,,\,C_6\) from one dimensional Hermite-series of \(Ydecorr|Z_y\). We had been using this coefficients as upper bound on row variances when our iterative method improved dramatically after introducing those analytic bounds. And instead of optimizing over 36 coefficients, we simply optimize over six variables given by \(B_1,\ldots\,,\,B_6\) . We have defined these variables  \(B_1,\ldots\,,\,B_6\) in a previous post.

In this way, we need to do only six dimensional optimization in our iterative method for fitting of 6X6 2D Hermite-series instead of optimizing over 36 values. I expect that this new technique would be both faster and more accurate and we would simultaneously fit to all cross-moments as well instead of relying on poorly worked out regressions with high variance in regression coefficients. 

I would write a detailed note about it in a day with all the equations.

I suggest the we remove the 1st order correlation effect of X on Y and write Zero mean Ydecorr as


\begin{align}
\left[Ydecorr|X\right](\tilde{Z}_y \, , \, Z_x)\, \,&=\, \, \Big[ \, b_{1,0} \, + \, b_{1,1} \,H_1(Z_x) \,+\, b_{1,2} \, H_2(Z_x) \, + \, b_{1,3} \,H_3(Z_x)\,\,+\,\ldots\, \Big] \,H_1(\tilde{Z}_y) \, \\
&+\, \Big[ \, b_{2,0} \, + \, b_{2,1} \,H_1(Z_x) \,+\, b_{2,2} \, H_2(Z_x) \, \, + \, b_{2,3} \,H_3(Z_x)\,\,+\,\ldots\, \Big] \, H_2(\tilde{Z}_y) \, \\
&+\, \Big[ \, b_{3,0} \, + \, b_{3,1} \,H_1(Z_x) \,+\, b_{3,2} \, H_2(Z_x) \, \, + \, b_{3,3} \,H_3(Z_x)\,\,+\,\ldots\, \Big] \, H_3(\tilde{Z}_y) \, \\
&+\,\,\,\ldots\, \\
\end{align} 

Expectation of second moment of  \(Ydecorr\) as a function of squared hermite polynomials is given by formula


\begin{align}
E\left[{Ydecorr}^2|X\right](\tilde{Z}_y \, , \, Z_x)\, \,&=\, \, E \Bigg[ \Big[ \, {b_{1,0}}^2 \, + \, {b_{1,1}}^2 \,{H_1(Z_x)}^2 \,+\, {b_{1,2}}^2 \, {H_2(Z_x)}^2 \, + \, {b_{1,3}}^2 \,{H_3(Z_x)}^2\,\,+\,\ldots\, \Big] \,{H_1(\tilde{Z}_y)}^2 \, \\
&+\,  \Big[ \, {b_{2,0}}^2 \, + \, {b_{2,1}}^2 \,{H_1(Z_x)}^2 \,+\, {b_{2,2}}^2 \, {H_2(Z_x)}^2 \, + \, {b_{2,3}}^2 \,{H_3(Z_x)}^2\,\,+\,\ldots\, \Big] \,{H_2(\tilde{Z}_y)}^2 \, \\
&+\,  \Big[ \, {b_{3,0}}^2 \, + \, {b_{3,1}}^2 \,{H_1(Z_x)}^2 \,+\, {b_{3,2}}^2 \, {H_2(Z_x)}^2 \, + \, {b_{3,3}}^2 \,{H_3(Z_x)}^2\,\,+\,\ldots\, \Big] \,{H_3(\tilde{Z}_y)}^2 \, \\
&+\,\,\,\ldots\, \Bigg] \\
\end{align} 


In one dimension, we know that 
\begin{align}
E\left[{Ydecorr}^2|X\right](\tilde{Z}_y \,)\, \,&=\, \, E \Big[ \, {c_{0}}^2 \, + \, {c_{1}}^2 \,{H_1(\tilde{Z_y})}^2 \,+\, {c_{2}}^2 \, {H_2(\tilde{Z_y})}^2 \, + \, {c_{3}}^2 \,{H_3(\tilde{Z}_y)}^2\,\,+\,\ldots\, \Big] \\
&=\, \,  \Big[ \, {c_{0}}^2 \, + \, {c_{1}}^2 \, \,+\,2\, {c_{2}}^2 \,  \, + \,6\, {c_{3}}^2 \,\,\,+\,\ldots\, \Big]
\end{align}

In above \(c_n\) are coefficients of 1 dimensional hermite series of Ydecorr.

Suppose in other dimension, we have

\begin{align}
E\left[{Ydecorr}^2|X\right]({Z}_x \,)\, \,&=\, \, E \Big[ \, {d_{0}}^2 \, + \, {d_{1}}^2 \,{H_1({Z_x})}^2 \,+\, {d_{2}}^2 \, {H_2({Z_x})}^2 \, + \, {d_{3}}^2 \,{H_3({Z}_x)}^2\,\,+\,\ldots\, \Big] \\
&=\, \,  \Big[ \, {d_{0}}^2 \, + \, {d_{1}}^2 \, \,+\,2\, {d_{2}}^2 \,  \, + \,6\, {d_{3}}^2 \,\,\,+\,\ldots\, \Big]
\end{align}

We normalize the above sum to one and write
\[{\bar{d}_n}^2\,=\,\frac{{d_n}^2}{ \Big[ \, {d_{0}}^2 \, + \, {d_{1}}^2 \, \,+\,2\, {d_{2}}^2 \,  \, + \,6\, {d_{3}}^2 \,\,\,+\,\ldots\, \Big]}\]

Then I conjencture that two dimensional expectation could be fulfilled by a structure of the kind (The first term in large brackets is already equal to second moment from one dimension calibration and second term in other large brackets has expectation of its variance/2nd moment equal to unity and does not alter the second moment but is chosen so that cross-moments of Ydecorr with X are as perfectly satisfied as possible)

\[E\left[{Ydecorr}^2|X\right](\tilde{Z}_y \, , \, Z_x)\, \,=\, \,  \Big[ \, {c_{0}}^2 \, + \, {c_{1}}^2 \, \,+\,2\, {c_{2}}^2 \,  \, + \,6\, {c_{3}}^2 \,\,\,+\,\ldots\, \Big] \Big[ \, {\bar{d}_{0}}^2 \, + \, {\bar{d}_{1}}^2 \, \,+\,2\, {\bar{d}_{2}}^2 \,  \, + \,6\, {\bar{d}_{3}}^2 \,\,\,+\,\ldots\, \Big]\]

This would imply that (up to sign that has to be decided separately) coefficients of 2D hermite matrix are given as

\[\,b_{n,m}=\,c_n\,\bar{d}_m \]

I really think that above formulas for 2D Coefficients should work. At least they make a solution to the variance of 2D Z-series. We can iterate over six \(\bar{d}_m\) in our iterative program(with the constraint that their weighted sum or 2nd moment is unity as earlier described) and settle at values that give us best fit to the cross-moments. Even if we could not perfectly calibrate the cross-moments with above six coefficients, they can give us a very good and cheap initial guess that could be used in full-blown optimization of 36 coefficients. 

Again our proposed solution for 2D Hermite series is akin to product of two single dimensional hermite-series, the first of which is one dimensional hermite-series of the random variable and the second one is a normalized variance hermite-series in other variable \(Z_x\) so that cross-moments with X are as well calibrated as possible. Even if it does not perfectly satisfy the cross-moments, we can use it as a good initial guess for the full-blown optimization over all 36 different coefficients.

I will try working out on matlab programs how the above idea goes. If it does well, we can even write a special Newton calibration like 1D calibration that solves for the coefficients of the 2D Hermite-series by exploiting the ideas given in this post.

Friends, if we think more deeply about our problem, we realize that there need to be some mathematical structure and relationships between different coefficients of various hermites when we try to infer parameters of 2D Hermite-Series.

I should have realized it very long ago but we can recall from the time when we were modelling SDEs to higher order that coefficients of higher hermites in SDEs were based on derivatives of the SDE function of first hermite. Granted that SDEs evolution have a structure in time but we are dealing with smooth and continuous functions and they should also have a structure in which coefficients of higher  hermites depend upon derivatives of some expression of smaller hermites.

I will give a very simple example of an SDE

\[dy\,=\,\sigma(y)\, dz_y\,=\, \sigma(y)\, H_1(Z_y) \,\sqrt{\Delta(t)}+\, {\sigma(y)}' \sigma(y)\, H_2(Z_y) \,\frac{\Delta(t)}{2}+\, {\sigma(y)}'' {\sigma(y)}^2 \, H_1(Z_y) \,\Delta(t) + \, \left[ {\sigma(y)}'' +{{\sigma(y)}'}^2 \right] \,H_3(Z_y)\,\,\frac{{\Delta(t)}^{1.5}}{6} \]

I think in ordinary densities when there is no time evolution, we should still have derivatives structures in higher hermites possibly as given below

\[y\,=\, \frac{d\sigma(y)}{dZ_y}\, Z_y \,+\,\,\frac{1}{2}\, \frac{d^2{\sigma(y)}}{{dZ_y}^2} \, {Z_y}^2  + \,\frac{1}{6}\,  \frac{d^3{\sigma(y)}}{{dZ_y}^3} \, {Z_y}^3\, \]

In the above \(\sigma(y)\) is not known but once it is known as a function, coefficients of higher hermites are no longer free parameters but are given by some expression of derivatives of the first hermite. 

This should also be true in two dimensional and higher dimensional Hermite-series. 

We can take a reasonably complex functional form of dependence between underlying variable  Y and \({Z_y}\) and then we would be free to choose parameters of the first order expansion of Y as a function of Z and all the higher order expansions coefficients would be derivatives of the first order expansion of Y as a function of Z. This expansion may not be helpful in one dimension since but in higher dimensions we would need to find only six or eight parameters and coefficients of higher hermites would be derivatives of first order expansions.

OK I can write the whole thing with more clarity. It is like saying that I find it very difficult to optimize for 8X8 coefficients of a 2D Taylor series but I can possibly try a more simpler analytic expression with lesser number of free parameters for the original function and take its cross-derivatives and optimize on the smaller number of free parameters of the analytic function rather than having to find all the huge number of coefficients of 2D Hermite-Series through optimization. 

Again, we do not start by assuming a perfect relationship but rather a functional form that has its own free parameters that are much smaller than 2D Taylor series free parameters and optimize over the free parameters of the functional form for a robust calibration.

Though the idea is very simple, I believe it can hugely simplify our optimizations and calibration of hermite-series. I will come up with several interesting examples in a day or two.

For example, we can try a simple functional form for our two dimensional Hermite series Ydecorr (that we were trying to find 36/64 coefficients of) as

\[\text{Ydecorr}(Z_y,Z_x)\,=\,a_0\, {(c_1+b_0\,Z_y)}^{\alpha_1} \,+ \,a_1\, {(c_2+b_1\,Z_y)}^{\alpha_2}\,+\, a_2\,{(c_3+b_3\,Z_y)}^{\beta_1} \,\,{(c_4+b_4\,Z_x)}^{\gamma_1} \, \\+\, a_3\,{(c_5+b_5\,Z_y)}^{\beta_2} \,\,{(c_6+b_6\,Z_x)}^{\gamma_2} \,\]

But the above form still has a lot of free parameters. As a start, we can try a simpler functional form of the kind given below as

\[\text{Ydecorr}(Z_y,Z_x)\,=\,a_0\, {(c_1+b_1\,Z_y)}^{\alpha_1} \,+\, a_2\,{(c_3+b_3\,Z_y)}^{\beta_1} \,\,{(c_4+b_4\,Z_x)}^{\gamma_1} \, \,\]

and try to see how things go. If it works and we can find ways to simple solutions, we could later increase complexity.

If I try to write a Newton Solver for above or other similar forms, good thing is that it will be quite reasonably over determined system when we try to fit it to thirty-six or sixty-four cross-moments. An over-determined system is usually good for stability. And its higher powers would usually be reasonably tame and would not be explosive either.

Friends, very sorry for not being able to write a post for so many days. A few days ago, my mother was diagnosed of breast cancer and I have been spending a lot of time going to Labs to take her further tests. It is still not clear at what stage the cancer is and everyone in my family hopes that her cancer is curable and she would be fine after getting appropriate medical treatment. It would be clear in about a day when reports of some other tests arrive that we would know about the stage of her cancer.

Friends, I have been thinking more about our optimization problem. Ok, in the objective function for the optimization of coefficients of 2D Hermite Series, we have been taking sum of squared differences between model cross moments and target cross moments. I believe that if we take the objective function over cross moment correlations(in the sense of cross-moments divided by 1D moments of appropriate order of both variables in cross-moments), the optimization would be far more well-behaved. We can take objective function as sum of squared differences between moment correlations and I really think that problem would be highly tamed for both our iterative method and the Newton would also possibly work much better. I hope that it might become a far easier optimization because I think cross-moments correlation would be linear in hermite-polynomials as opposed to higher powers when we fit directly to cross-moments. 

It is not very difficult to make these changes and run the program again and I hope to post a new program as soon as I get some free time. I am very hopeful that dimensionless correlations would be far better well behaved than the objective function taken over blowing higher moments.

As soon as I get my mother's further reports, I will post some details and hope some of the more knowledgable friends would mention anything about possibilities of her treatment.