Friends, I am writing a small post trying to explain the basics of how to convert functions and polynomials of one density with continuous derivatives into polynomials of another density with continuous derivatives. I will not go deep into mathematical calculations but try to give idea about logic and methods involved in such change of probability distributions.

Suppose we have one standard normal density and we write it as

[$]p_Z(z) \, = \, \frac{1}{\sqrt{2 \pi}} \, \exp(-\, \frac{z^2}{2}) [$]

and we have a gamma density and we write it as

[$]p_G(g) \, = \, \frac{1}{\Gamma(k)\, {\theta}^k}\, g^{(k-1)} \, \exp(-\frac{g}{\theta}) [$]

We know the change of variables equation for continuous univariate probability densities as

[$]\, p_G(g) \, |\frac{dg}{dz}|\, = \, p_Z(z) [$] Eq(1)

We select a common CDF point for both densities where we find derivatives between gamma variable g and standard normal variable z. In my experiments I found that median of both densities was a very good choice. Median of many densities is not known analytically but can be calculated with very high precision using numerical methods if CDF is known. We suppose that pdf of both densities involved is known in closed from and we can analytically take all order of derivatives of each pdf.

From Eq(1) we can find first derivative between both densities at any common CDF point which is median in our special case. For that we just input values of z and g at their respective medians in the respective probability distribution functions in Eq(1) and back out the value of [$]|\frac{dg}{dz}|\,[$]. We will have to be careful about absolute value but if both densities are increasing at median, [$]\frac{dg}{dz}\,[$] will always be positive and we would not need to worry about absolute sign. It turns out that we can find all higher derivatives [$]\frac{d^{n} g}{dz^n}\,[$] by differentiating the Eq(1) n-1 times on both sides and doing some algebra. Please read posts 1637, 1638, 1639 for details.

viewtopic.php?t=99702&start=1635#p873552
viewtopic.php?t=99702&start=1485#p871049
Alternatively we could also have found [$]\frac{dz}{dg}\,[$] and all of the nth derivatives [$]\frac{d^{n} z}{dz^n}\,[$] by writing Eq(1) differently as

[$]\, p_G(g) \, = \, p_Z(z) |\frac{dz}{dg}|\,[$] Eq(2)

It turns out that differentiating many densities becomes very tedious after 9th or tenth derivatives but an algorithm can easily be written for respective densities that can differentiate them several tens of times for arbitrary precision.

Once we have found derivatives between both densities to sufficient order, we can use them to represent for example functions of standard normal as polynomials in gamma density variable or we could represent functions of gamma density as polynomials in standard normal random variable. Of course, some restrictions would apply but the conversions can be useful in a large number of settings.

Let us suppose we have a function or a polynomial in Gamma density variable written as [$]f(g)[$] and we want to represent it as polynomial function h(z) of standard normal variable z. Again we would do this polynomial expansion around common CDF points which we prefer to choose as median of both densities. In order to do that , we would have to write

[$]h(z)=f(g(z))[$] Eq(3)

We will expand the right hand side of Eq(3) in a taylor series and it is given to fourth order as (please note that derivatives between gamma random variable g and normal random variable z of all orders [$]\frac{d^n g}{dz^n}[$] have previously been calculated at a common CDF point possibly median either from differentiating Eq(1) or Eq(2) on both sides repeatedly and using some algebra) Here [$]z_0[$] is median of standard normal density.

[$]h(z)\, = \, f(g(z)) \, = \, f(g(z_0))+(z-z_0) \, \frac{dg}{dz} \, f'(g(z_0)) \\

+\frac{1}{2} (z-z_0)^2 \big[ \, {(\frac{dg}{dz})}^2 f''(g(z_0))+ \, \frac{d^2g}{dz^2} \, f'(g(z_0))\big] \\

+\frac{1}{6} (z-z_0)^3 \big[f^{(3)}(g(z_0)) \, {( \frac{dg}{dz})}^3 \, +3 \, \frac{dg}{dz} \, \, \frac{d^2g}{dz^2} \, f''(g(z_0))+ \, \frac{d^3g}{dz^3} \, f'(g(z_0))\big]\\

+\frac{1}{24} (z-z_0)^4 \big[f^{(4)}(g(z_0)) \, {(\frac{dg}{dz})}^4 \,+6 f^{(3)}(g(z_0)) \, {(\frac{dg}{dz})}^2 \, \, \frac{d^2g}{dz^2} \,+3 \, {(\frac{d^2g}{dz^2})}^2 \, f''(g(z_0))+4 \, \frac{d^3g}{dz^3} \, \, \frac{dg}{dz} \, f''(g(z_0))+\, {(\frac{d^4g}{dz^4})} \, f'(g(z_0))\big]\\

+O\left((z-z_0)^5\right)[$]

We can write algorithm for an automated program that expands the above series to arbitrary order as opposed to fourth order we have done in previous equation.

Similarly we could have just as easily converted many functions of standard normal into functions of gamma density using the same logic.

Once we have found a polynomial representation in variable of a particular density, we could convert it easily into a series in orthogonal polynomials of the same density. This is because span of all nth order polynomials is the same and everything that can be represented by an nth order polynomial, can also be represnted as a series in first n orthogonal polynomials and it just requires some algebra (we have always been converting all nth order polynomial functions of standard normal in the form of hermite polynomials series with first n polynomials).

You think life is a secret, Life is only love of flying, It has seen many ups and downs, But it likes travel more than the destination. Allama Iqbal