Friends, I continued to think about yesterday's ideas and seem to possibly have more insight about using Taylor series for orthogonal regressions. But still this is non-tested theory and I might be totally off and therefore I want to request friends to pardon any possible mistakes in my exposition.
In deterministic setting, if we want to do an orthogonal regression between explanatory variables and explained variable, we first have to choose an appropriate base function into which we will Taylor-expand our variables. (Non-linearity of a variable is with respect to another variable that is related through a non-linear function. However non-linear a variable with respect to some other variable, it is perfectly linear on its own axis and obviously cannot be taylor-expanded onto itself.) So first we have to find an appropriate base function with respect to which we will Taylor-expand explained and explanatory variables. In our orthogonal stochastic hermite regressions, the base variable was a standard gaussian into which we expanded explained and explanatory variables.
To keep things simple, we suppose that we are orthogonal-Taylor-regressing one explained variable Y on one explanatory variable X. Suppose we choose base function as [$] \, f(u)\, = \, {u}^{\frac{1}{4}} \, [$] . Again, there can be many possible choices of base functions that can even possibly be better but we are giving this one-fourth power as one possible base function.
We want to orthogonal-Taylor-Regress Y on X. Using the base function mentioned above, we have
[$] \, W \, = \, {X}^{\frac{1}{4}} \, [$] meaning [$]\, X \, = \, W^4 \, [$]
and
[$] \, V \, = \, {Y}^{\frac{1}{4}} \, [$] meaning [$]\, Y \, = \, V^4 \, [$]
W is base variable for X while V is base variable for Y. In our hermite-orthogonal-regressions independent Z's were base variables.
We Taylor-expand X in its base variable W and we also Taylor-expand Y in its base variable V as (around appropriate series centre points. We would have to work these things out where to place center points of series. These centre points corresponded to medians in our Z-series for hermite-orthogonal-regressions.)
following our previous equation [$]\, X \, = \, W^4 \, [$], we taylor expand around appropriate expansion point [$]W_0(X_0) [$] as
[$]X \, = \, W_0 \, + \, 4 \, {W_0}^3 \, (W \, - \, W_0) \, + 1/2 \,*\, 12 \, {W_0}^2 \, {(W \, - \, W_0)}^2+ 1/6 \,*\, 24 \, {W_0} \, {(W \, - \, W_0)}^3+ 1/24 \,*\, 24 \, {(W \, - \, W_0)}^4 [$]
which can be sucintly written as
[$]X \, = \, b_0 \, + \, b_1 \, (W \, - \, W_0) \, + b_2\, {(W \, - \, W_0)}^2+ b_3 \, {(W \, - \, W_0)}^3+ b_4 \, {(W \, - \, W_0)}^4 [$]
Similarly, following above steps again, we can Taylor-expand Y in terms of V as
[$]Y \, = \, c_0 \, + \, c_1 \, (V \, - \, V_0) \, + c_2\, {(V \, - \, V_0)}^2+ c_3 \, {(V \, - \, V_0)}^3+ c_4 \, {(V \, - \, V_0)}^4 [$]
Please notice the similarity of Taylor-expansions of X and Y with our previous Z-series expansions.
Now to set up the regression, we would have to convert X and Y data points into W and V data points as we had to do in the inversion of Z-series. Our final data before regression would be in terms of W and V variables (as opposed to Z1 and Z2 in case of hermite Z-series orthogonal regressions)
Now, in order to regress Y on X, we can simply do four regressions, with each regression between appropriate expansion powers. These regressions would be between (just like we have to take into account covariances, we would have to take into account coefficients on each of these pairs mentioned below)
[$](V \, - \, V_0)[$] and [$](W \, - \, W_0)[$]
[$]{(V \, - \, V_0)}^2[$] and [$]{(W \, - \, W_0)}^2[$]
[$]{(V \, - \, V_0)}^3[$] and [$]{(W \, - \, W_0)}^3[$]
[$]{(V \, - \, V_0)}^4[$] and [$]{(W \, - \, W_0)}^4[$]
There will be four pairs of orthogonal regressions since the base function we chose allowed for expansion up to fourth power only. Some Other base functions could possibly do better. But these details we will continue to know in coming weeks and months. We could combine the results from above orthogonal regressions to come up with one estimator for the explained variable.)
Friends above is my brief plan of attack for orthogonal Taylor regressions but I might have made some mistakes or something might possibly be wrong so please pardon any mistakes.
I will try to come up with a Taylor orthogonal regression program in next 3-4 days.
As I see, if we take deterministic functions without any noise in data, and do above steps appropriately by choosing an appropriate expansion point and other things right, we might possibly get a fourth order accurate result from this regression if the data is from deterministic functions without noise.
.
.
Friends, I suggested in the copied paste that
We want to regress using taylor basis functions and
our base variable for explanatory variable could possibly be [$] \, W \, = \, {X}^{\frac{1}{4}} \, [$] meaning original variable would be given as [$]\, X \, = \, W^4 \, [$]
There would usually not be any integer power function and we have no way to know whether X can be denoted as an integer power function as in above.
But we can easily assume that X is a polynomial function of W. If we want to take analytics to fourth power, this polynomial relationship can be stated as
[$]X\, = \, a \, W^4 \, + \, b \, W^3 \, + \, c\, W^2 \, + \, d \, W \, + \, e [$]
and we have to find W through inverse function using numerical inversion of X since data is given in X coordinates.(Similarly we would have a different polynomial function usually for same order relating to base variables for all explained and explanatory variables)
Now we can Taylor expand X as
[$] \overline{X(W)}\, =\,X(W_0) \, + \frac{dX}{dW}(W_0) \, (W\, - \, W_0) \,+ \frac{1}{2} \, \frac{d^2 X}{dW^2}(W_0) \, {(W\, - \, W_0)}^2 \,+\frac{1}{6} \, \frac{d^3 X}{dW^3}(W_0) \, {(W\, - \, W_0)}^3 \,[$]
[$]+ \frac{1}{24} \,\frac{d^4 X}{dW^4}(W_0) \, {(W\, - \, W_0)}^4 \,[$]
overline on X denotes that this value of X is derived from Taylor expansion given the value of W (which is found from X after numerical inversion of original polynomial function relationship)
where
[$]\,X(W_0) \,=\, a \, {W_0}^4 \, + \, b \, {W_0}^3 \, + \, c\, {W_0}^2 \, + \, d \, W_0 \, + \, e [$]
[$]\,\frac{dX}{dW}(W_0) \,=\,4 \, a \, {W_0}^3 \, + \, 3\, b \, {W_0}^2 \, + \,2\, c\, {W_0} \, + \, d \, [$]
[$]\,\frac{d^2 X}{dW^2}(W_0) \,=\,12 \, a \, {W_0}^2 \, + \, 6 \, b \, {W_0} \, + \,2\, c\, [$]
[$]\,\frac{d^3 X}{dW^3}(W_0) \,=\,24 \, a \, {W_0} \, + \, 6 \, b \, [$]
[$]\,\frac{d^4 X}{dW^4}(W_0) \,=\,24 \, a \, [$]
We start with guess coefficients a, b, c, d, and e. We now take all the data values [$]X_n[$] and their respective Taylor derived values [$]\overline{X_n}[$] and perturb and optimize for the coefficients a, b, c, d, and e until [$]\sum {(X_n\, - \, \overline{X_n})}^2 [$] is minimized.
The true coefficients for X as a function of W for a particular expansion order would be the ones that minimize [$]\sum {(X_n\, - \, \overline{X_n})}^2 [$] meaning our chosen coefficients make the polynomial base function calculation of the data variables as close to the data observations as possible.
Even if Taylor does not work perfectly for regression(something we have yet to check in our experiments), we could easily change the above calculations into Legendre orthogonal polynomial basis and run an orthogonal regression as we did with hermite polynomials.
Above was just a post explaining my strategy to find the best Taylor expansion of a certain order given some numerical data. I will come with a new program with Taylor expansion and regression in Taylor expanded basis or possibly also in Legendre orthogonal basis in a few days.
You think life is a secret, Life is only love of flying, It has seen many ups and downs, But it likes travel more than the destination. Allama Iqbal