]]>

]]>

Statistics: Posted by Cuchulainn — Yesterday, 3:15 pm

]]>

Statistics: Posted by tagoma — Yesterday, 2:10 pm

]]>

Statistics: Posted by tagoma — Yesterday, 1:43 pm

]]>

These ideas can easily be applied to Legendre polynomials in Uniform measure and Hermite polynomials in gaussian measure. I believe these ideas can also be used in a reasonable way to find a very good guess that can be used with moment matching optimizations to fit the coefficients of orthogonal polynomials on normal or uniform distribution.

We start with the basic premises that whatever the distribution of data is, the variable in Legendre polynomial used to fit the data should (ideally) be a perfect uniform density after we invert the Legendre polynomial representation (or alternatively the U-series representation) of data. Similarly when a well-fitted Hermite polynomial representation (or its associated Z-series) is inverted, it should ideally give us a perfect normal distribution. Alternatively it means that Legendre polynomials and Hermite polynomials used to fit the data (and find their coefficients representing the data) are from true uniform and true normal distribution respectively.

We first create the perfect uniform measure for Legendre polynomials and a perfect gaussian measure in case of hermite polynomials. If there are twenty points in the data and we are assuming a perfect uniform distribution, each point should have probability equal to 1/20 and the grid should be equidistant as well. Similarly in case of a perfect gaussian distribution we will need a twenty points grid so that probability mass in each grid point is 1/20 but these grid centre points would be very close to each other around the median and very far from each other in the tails since they are dictated by a discretized gaussian density. These are both discretized version of perfect uniform and perfect gaussian density. We use these discretized versions of Gaussian density and uniform density to form discretized versions of Hermite polynomials and Legendre polynomial in data.

In reality the data is random and never equidistant or equiprobable ever but we use this idea of discretized version of the uniform and normal distributions as an ideal situation from which we find coefficients of respective polynomials by minimizing least squares.

Again if we have twenty data points, we sort them in increasing order and associate a uniform on discretized uniform density with each of the data points.

First polynomial basis is linear in both hermite and Legendre and is given as distance from median of the density. But this first hermite/legendre polynomial is more difficult and usually its coefficient cannot be found by minimizing least squares. The coefficient of first orthogonal polynomial affects all of the higher polynomials and has to be selected by trial and error. We intend to start with a suitable number for coefficient of first hermite or legendre polynomial and then find coefficients of all of the higher order polynomials by least square minimization. We note how good the fit to data is for first few orthogonal polynomials. We repeat the process by perturbing the coefficients of first hermite/Legedre polynomial in the direction of better fit and regress higher order polynomials and again find least square fit to data. We choose the optimization associated with that coefficient of first orthogonal polynomial where fit to data from all polynomials least square fit is the best. So again first coefficient (of first orthogonal polynomial) is chosen by trial ad error in direction of over all best fit to data and rest of the higher order polynomial coefficients are fitted by minimizing least squares to data.

Again the choice of coefficient associated with first hermite/Legendre polynomial is the most difficult and basically pins the linear span of true Gaussian or Uniform density.

Suppose we have twenty data points and we want to fit coefficients of Legendre polynomials to describe the data. The data is sorted in increasing order by

[$]X_n[$] where n ranges from one to twenty. We have our ideally discretized uniform density with equidistant grid of twenty points and 1/20 probability mass in each grid cell.

This Un would be centers of each equiprobable grid cells.

U(1)=-.95,U(2)=-.85,,U(3)=-.75, ... , U(10)=-.05, U(11)=.05, .... , U(20)=.95

With each [$]X_n[$], there is associated a [$]U_n[$] so that

[$]X_n \, - \, X_0 = c_1 \, (U_n \, - \, U_0) \, + R_{1,n}[$]

This [$]c_1[$] is chosen by trial and error and [$]R_{1,n}[$] denotes defect in fit of nth point with respect to first Legendre polynomial.

Now we choose coefficient of second Legendre polynomial by regressing the defect of data points from first polynomial to higher polynomials.

[$]c_2[$] is chosen so that expression below is minimized

[$]\sum_{n=1}^{n=20} \, {(R_{1,n} \, - \, c_2 \, L_2(U_n))}^2 [$]

Where [$]L_2(U_n)[$] is value of second Legendre polynomial associated with value of Uniform associated with nth grid cell on our idealized uniform density.

Once we have found coefficient [$]c_2[$] associated with second Legendre polynomial, we find further remainder as

[$]R_{1,n} \, - \, c_2 \, L_2(U_n)= R_{2,n}[$]

Now we fit third Legendre polynomial on this remainder from fitting of second Legendre polynomial by minimizing the following expression in least square sense as

[$]\sum_{n=1}^{n=20} \, {(R_{2,n} \, - \, c_3 \, L_3(U_n))}^2 [$]

This way we can fit as many Legendre polynomials as we want. If we fit m polynomials, we will get the following number for criteria of fit

[$]\sum_{n=1}^{20} \, {R_{m,n}}^2 [$]

But our choice of first coefficient [$]c_1[$] was based on trial and error. We now perturb the first coefficient and redo the whole procedure for fitting the coefficients on first m Legendre polynomials and note criteria of fit. We continue to perturb it in the direction of minimizing criteria of fit and this whole process is repeated again and again until criteria of fit statistic is minimized.

We have an ideal uniform measure for the underlying but data is usually too random and therefore we can get resulting coefficients of Legendre polynomials from above procedure and possibly use them as initial guess for moment matching procedure of the U-series.

Similar process can easily be used with normal distribution and hermite polynomials. In which we can arrange data in increasing order by sorting and then associate a discretized normal (on our perfect normal density with equal probability mass in each grid cell.) We will have to choose coefficient of first hermite polynomial by trial and error and then choose the coefficients of higher polynomials by minimizing least squares. And repeating the whole process by perturbing c1 and redoing calculations of coefficients of higher hermites so that criteria of fit with the whole process is optimized.

I will have to get an antipsychotic injection in two days, and I am travelling tomorrow to Islamabad, therefore it may take me a week or so to finish this program. I will be posting it here on the forum as always.

Statistics: Posted by Amin — Yesterday, 1:33 pm

]]>

Statistics: Posted by tagoma — Yesterday, 10:48 am

]]>

Statistics: Posted by tagoma — Yesterday, 10:43 am

]]>

VivienB: well, some do, some don't.In particular, compile-time generic programming (with the binding & dispatch performed statically but general enough so as to allow for compile-time laziness) is not as common.Let's take Template Haskell as an example: http://www.haskell.org/haskellwiki/Temp ... askellThis is closely associated with type classes,[/b] for which, again, while comparable features are widespread, things are rather more interesting as far as the compile-time variant is concerned.Also relevant in this context:- Multi-parameter type classes- (Indexed) Type familiesI'm wondering, how many other FP languages implement comparable features?// I think GP is far more than "generics" implemented by some programming languages. In particular, I find run-time attempts at the implementation particularly limited and generally uninteresting.From the PLT (programming language theory) point of view, Harper (2014) provides an interesting way to look at it (succinct if you're coming from an FP background):QuoteThe generic extension of a type operator is an example of the concept of a functor in category theory (MacLane, 1998). Generic programming is essentially functorial programming, exploiting the functorial action of polynomial type operators (Hinze and Jeuring, 2003).If you're curious / would like more information, take a look at Chapter 13 (Generic Programming), which is the source of the quote:Harper (2014) "Practical Foundations for Programming Languages" Second Editionhttp://www.cs.cmu.edu/~rwh/plbook/book.pdf

Yes, by George!Haskell Type classes are the omphalos of C++20 Concepts.

Got it, QED

Why did you guys keep the rest of us in the dark for so long?

//

Statistics: Posted by Cuchulainn — March 25th, 2023, 9:38 pm

]]>

]]>

[$]g(Z) \, = \frac{1}{\sqrt{2 \, \pi}}\, \exp(-.5 \, Z^2)\,[$]

and

[$] f(U) \, = \, \frac{1}{2} [$] for [$]\, -1 \, \geq \, U \, \geq \, 1[$] zero otherwise

Through change of variable derivative relationship between densities we know that

[$]g(Z) \, dZ \, = f(U) \, dU \,[$]

or

[$]g(Z) \, \frac{dZ}{dU} \, = f(U) \,[$]

The above relationship is related through common cdf or common probability mass. For details please look up in wikipedia.

At a point related through common CDF, if [$]G(Z_0) \, = \, F(U_0) \, \,[$] where G and F represent their respective CDF, we have

[$]g(Z_0) \, \frac{dZ}{dU} \, = f(U_0) \, \,[$]

or

[$]\frac{dZ}{dU} \, = \frac{f(U_0)}{g(Z_0)} \, \,[$]

The above derivative is valid at common CDF points. In our expansions, we take this derivative at median of both densities which are both incidentally zero.

Sorry friends, but we cannot take higher derivatives of standard normal with respect to standard uniform. We cannot differentiate uniform density. We could have equated a zero derivative with second order derivative equation on other (standard normal )side but first hermite (which is derivative of standard normal density) also goes to zero at median. So we cannot find higher derivatives of standard normal with respect to standard uniform at median.

Sorry for this since I was erroneously thinking we could take higher derivatives of standard normal with respect to uniform even if we could not take derivatives of uniform.

I would have to try some other way to solve this problem.

I still think that earlier exposition could be useful in many ways especially with densities that take continuous derivatives.

I will look at things more carefully and try to see how we can solve this problem.

It would be possibly interesting to see if using just first derivatives in Taylor series of functions of normal might possibly help us in getting a decent initial point for optimizations for U-series.

Statistics: Posted by Amin — March 25th, 2023, 3:25 pm

]]>

[$]\, Z(U)\, = \, Z(U_0) \, + \, \frac{dZ(U_0)}{dU} \, (U \, - \, U_0) \, + \, \frac{1}{2} \, \frac{d^2Z(U_0)}{dU^2} \, {(U \, - \, U_0)}^2[$]

[$] \, + \, \frac{1}{6} \, \frac{d^3Z(U_0)}{dU^3} \, {(U \, - \, U_0)}^3\, + \, \frac{1}{24} \, \frac{d^4Z(U_0)}{dU^4} \, {(U \, - \, U_0)}^4 + \ldots [$]

One might have to consider that converting from an exponential measure to uniform measure might require reasonably large number of derivatives. But if you want to represent your data in uniform measure (and in related Legendre polynomials), you might have to sacrifice some brevity associated with polynomial representation associated with standard normal measure. I have to understand that this is usually related to data itself and if I directly fit the data that is (normal-like enough), I will still need a large degree of polynomial in standard uniform to fit the data (It does not have to do with that we are converting Z-series to U-series) A higher order representation in uniform series is a feature of the data.

It turns out that we can easily find the derivatives [$]\, \frac{dZ(U_0)}{dU} [$] , [$]\frac{d^2Z(U_0)}{dU^2}[$] , [$]\frac{d^3Z(U_0)}{dU^3}[$] and all other higher derivatives. Though we can expand around any point on both standard normal and standard uniform density associated with a common CDF, I would generally prefer to almost always expand around points associated with median of both densities. In standard normal and in our version of standard uniform, both these densities have a median at zero.

For functions of standard normal variable (As Z-series representation of a random variable is a polynomial function of standard normal variable), we will use a version of Taylor given as below. Here [$]Z_0[$] is point on normal density associated with expansion point [$]U_0[$] and I would prefer that [$]Z_0[$] and [$]U_0[$] both are median of respective densities (and hence associated with common CDF). Particularly [$]\, f(Z(U_0)) \,= \, f(Z_0) [$]

[$]\, f(Z(U))\, = \, f(Z(U_0)) \, + \, \frac{df(Z_0)}{dZ}\, \frac{dZ(U_0)}{dU} \, (U \, - \, U_0) \, + \, \frac{1}{2} \, \, \Big[ \frac{d^2f(Z_0)}{dZ^2}\, \big[\frac{dZ(U_0)}{dU} \big]^2 +\,\frac{df(Z_0)}{dZ}\, \frac{d^2Z(U_0)}{dU^2} \Big] \, {(U \, - \, U_0)}^2[$]

[$] \, + \, \frac{1}{6} \,\Big[ \frac{d^3f(Z_0)}{dZ^3}\, \big[\frac{dZ(U_0)}{dU} \big]^3 +3 \, \frac{d^2f(Z_0)}{dZ^2}\, {\frac{dZ(U_0)}{dU}} \, {\frac{d^2Z(U_0)}{dU^2}} +\frac{df(Z_0)}{dZ}\, \frac{d^3Z(U_0)}{dU^3} \,\Big] \, {(U \, - \, U_0)}^3\, + \ldots [$]

In the next post, I explain how to find derivatives of standard normal with respect to standard uniform by relating standard normal density to standard uniform density through change of variables derivatives between both densities.

Statistics: Posted by Amin — March 25th, 2023, 1:37 pm

]]>

To give friends some ideas about how to convert a standard normal variable (or any other density with continuous derivatives) into (our version of) standard uniform, I want friends to read some preliminaries including few older posts where I have mentioned how to do this.

Below I am directly copying the contents of this post so it is easier to read. Here is the original post 1637: https://forum.wilmott.com/viewtopic.php?f=4&t=99702&start=1635#p873552

I will copy contents of the above post below and then explain how a Z-series can be converted directly into a U-series in the next post. This means that if we are given a polynomial representation of our data in terms of polynomial in powers of standard normal, we can describe the same data by a polynomial in standard uniform. Preliminaries first and then I give the details in next post.

The idea is to expand standard normal and its powers as a Taylor series in (our version of) standard Uniform variable. The derivatives of Z with respect to U of all orders required for this Taylor expansion are found by simple analytics related to change of density derivative of standard normal density with respect to standard uniform density. Please read the copied post below and I explain the ideas in more details in next post in the context of conversion of Z-series in U-series.

Below is the copy of post 1637.

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Friends, I had tried this idea earlier when I was finding Z-series of density from its moments in posts 1491 and 1492. But there I had tried to generate a base density from solution of linear equations applied to match the moments of the random variable. The idea of expanding any random variable with analytic density at its median and finding its derivatives with respect to standard normal random variable to from a Z-series of random variable was perfectly sound but the whole thing did not work as moments calculated from linear equations were totally rubbish.

https://forum.wilmott.com/viewtopic.php?f=4&t=99702&start=1485#p871047

https://forum.wilmott.com/viewtopic.php?f=4&t=99702&start=1485#p871049

Here I copy relevant parts of those above posts.

From post 1491:

Friends, we have earlier learnt when we want to represent a stochastic random variable X, in the form of a Z-series, the form of equation for X is given as

[$]X(Z)=a_0 \,+ a_1 \, Z + a_2 \, Z^2 + a_3 \, Z^3 + a_4 \, Z^4 +\, ... [$]

The first, second and third derivatives of w(t) w.r.t Z are given as

[$]\frac{\partial X}{\partial Z}= a_1 \, + 2 \,a_2 \, Z +3 \, a_3 \, Z^2 +4 \, a_4 \, Z^3 +\, ... [$]

[$]\frac{\partial^2 X}{\partial Z^2}= 2 \,a_2 \, +6 \, a_3 \, Z +12 \, a_4 \, Z^2 +\, ... [$]

[$]\frac{\partial^3 X}{\partial Z^3}= 6 \, a_3 \, +24 \, a_4 \, Z \,+60 \, a_5 \, Z^2 +\, ... [$]

IF we do all the calculations of series evolution equation at median where Z=0. here all the derivatives are given only by leading coefficient associated with zeroth power of Z. So at Z=0,

[$]w(t)=a_0 \,[$]

[$]\frac{\partial X}{\partial Z}= a_1 \, [$]

[$]\frac{\partial^2 X}{\partial Z^2}= 2 \,a_2 \, [$]

[$]\frac{\partial^3 X}{\partial Z^3}= 6 \, a_3 \, [$]

[$]\frac{\partial^4 X}{\partial Z^4}= 24 \, a_4 \, [$]

So we have that

[$]X(Z)=a_0 \,+ a_1 \, Z + a_2 \, Z^2 + a_3 \, Z^3 + a_4 \, Z^4 +\, ... [$]

[$]X(Z)=a_0 \,+ \frac{\partial X}{\partial Z} \, Z + 1/2 \, \frac{\partial^2 X}{\partial Z^2} \, Z^2 +1/6 \, \frac{\partial^3 X}{\partial Z^3} \, Z^3 +1/24 \, \frac{\partial^4 X}{\partial Z^4} \, Z^4 +\, ... [$]

So basically our power series is the same as Taylor series where derivatives between the two random variables, the stochastic variable X and standard Gaussian Z, are calculated at the median. ( You could do these calculations at other points in the density as well but those calculations would be much more involved and if correct should yield the exact same final result if the density is based on continuous derivatives)

If we could just find the median of density of our random variable X and its derivatives [$]\frac{\partial X}{\partial Z}\, [$], [$]\frac{\partial^2 X}{\partial Z^2} [$] and other higher derivatives with respect to Z, we can find coefficients of our power series representation of our random variable X in terms of powers of standard Gaussian.

Again the above derivatives would have to be calculated at median of densities of both random variables, X and Z.

From Post 1492:

As we learnt in the previous post, In order to construct the Z-series of a particular stochastic random variable X, we have to find its various derivatives with respect to standard Gaussian at the median(of both densities). So our first step towards construction of a Z-series representation would be to find median of the stochastic random variable X in question. I suppose that we have constructed analytical density of X using one of the methods described in previous post. We would have to find median mostly through Newton-Raphson as there are usually no simple formulas for median of a density.

After finding median, we can fix first coefficient of Z_Series as

[$] c_0 \, = \, Median [$]

Both our density for random variable X and the standard gaussian density are related through change of variable formula for two densities as

[$]p_Z(Z) \, = \, p_X(X(Z)) \, \frac{dX}{dZ} \, [$] Eq(1)

So we can find [$]\, \frac{dX}{dZ} \, [$] at median from the ratio of densities of respective variables at median as

[$]\, \frac{dX}{dZ} \, = \, \frac{p_Z(Z=0)}{p_X(X(Z)=c_0)}= \, \frac{p_Z(0)}{p_X(c_0)}[$]

In order to find higher derivatives of X with respect to Z at median, we differentiate Eq(1) on both sides w.r.t Z as

[$]p'_Z(Z) \, = \, p'_X(X(Z)) \, {(\frac{dX}{dZ})}^2 + \, p_X(X(Z)) \, \frac{d^2X}{dZ^2}\, [$] Eq(2)

In above equation and similar subsequent equations, we know the derivatives of gaussian density at Z=0 analytically. And we can easily find all first few nth derivatives of density of random variable X at its median [$]\frac{dp^n_X}{dX^n}(X(Z)) = \, \frac{dp^n_X}{dX^n}(c_0) [$] from the analytic density as we have constructed in the previous post. So we know values of all variables in Eq(2) other than [$]\, \frac{d^2X}{dZ^2}\, [$] whose value we back out from Eq(2)

The third derivative equation would be

[$]p''_Z(Z) \, = \, p''_X(X(Z)) \, {(\frac{dX}{dZ})}^3 + \, 3\, p'_X(X(Z)) \, {\frac{dX}{dZ}} \, {\frac{d^2X}{dZ^2}} + \, p_X(X(Z)) \, \frac{d^3X}{dZ^3}\, [$] Eq(3)

which can be used to back out [$] \, \frac{d^3X}{dZ^3}\,[$]

Similarly we can continue to differentiate Eq(2) and keep finding value of next higher derivative of X w.r.t Z at median.

After finding first few derivatives of X w.r.t Z at median, we can construct the Z_series of X as

[$]X(Z)=a_0 \,+ \frac{\partial X}{\partial Z} \, Z + 1/2 \, \frac{\partial^2 X}{\partial Z^2} \, Z^2 +1/6 \, \frac{\partial^3 X}{\partial Z^3} \, Z^3 +1/24 \, \frac{\partial^4 X}{\partial Z^4} \, Z^4 +\, ... [$]

In a similar spirit, I think we can take the base density as a different density (could be gamma density) and represent some stochastic variable as a series in Gamma density variable by equating the equations of two densities through change of density derivative at their median.

Statistics: Posted by Amin — March 25th, 2023, 12:38 pm

]]>

is it just an (illocutionary) speech act?

Does ChatGPT understand this kind of stuff?

Statistics: Posted by Cuchulainn — March 25th, 2023, 12:16 pm

]]>

https://www.bbc.com/news/world-us-canada-65071989

Statistics: Posted by Cuchulainn — March 25th, 2023, 12:09 pm

]]>