SERVING THE QUANTITATIVE FINANCE COMMUNITY

Amin
Topic Author
Posts: 2046
Joined: July 14th, 2002, 3:00 am

Re: Breakthrough in the theory of stochastic differential equations and their simulation

I have definitively solved the simulation o stochastic differential equations using my method of Stochastic Calculus of Standard Deviations(SCSD). I am getting high accuracy to a few basis points with extreme volatility out to 5-10 years. The method is basically analogue of the method of iterated integrals that I used for solution of ordinary differential equations adapted to stochastic differential equations. Might possibly post the programs in a week.

tagoma
Posts: 18233
Joined: February 21st, 2010, 12:58 pm

Re: Breakthrough in the theory of stochastic differential equations and their simulation

That googling "good c++ books justin london" returns anything finished to convince me that Google search is untrustworthy.

Amin
Topic Author
Posts: 2046
Joined: July 14th, 2002, 3:00 am

Re: Breakthrough in the theory of stochastic differential equations and their simulation

Since this post in off-topic is relevant to this thread, I have copied it here for reference. Here is the link to the paper.https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2872598

Amin wrote:
Cuchulainn wrote:
I had a quick look at the SSRN article. It seems you did a lot of improvement compared to the initial try outs.

in theory it is important that is works to crunch numbers as well as doing symbolic maths.

Daniel, I have given three mathematica code listings in the paper each for first Order Odes, nth order ODEs and Systems of ODEs. Mathematica is used for different differentials and integrals that are messy to calculate by hand. You could for example  change the mathematica line that specifies the ODEs and run the rest of the code as such. Again all you need to do is to change the specification of ODE or system of ODEs and rest of the mathematica code will solve the problem to give the right answer. Solution of ODEs that follow from the method is series solution of the closed form answer.
Method of iterated integral that I have used for solution of ODEs has far larger applications. You can for example solve non-linear transforms using this method. It could be used for the solution of SDEs and more. This method is a cousin of Taylor series.

Cuchulainn
Posts: 56926
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Re: Breakthrough in the theory of stochastic differential equations and their simulation

Indeed, the second revision is more readable and structured, but there is some way to go (depending on what your requirements are) as the article is more of a engineering/methods/recipe type article (nothing wrong with that) without the mathematical foundations yet.

Some constructive remarks. To reduce the scope, I only examined first-order scalar IVP case:

1. The 'iterated integral trick' (your words) seems to have been plucked from thin air. In particular, I got stuck at equations (3) and (4). The maths is informal. In practice it might be OK but it has to be proved in theory.
2. The compelling reason for iterated integral is not clear to me  I examined equations (13) (BTW you can mention is $e^t$) and (14) $tan( t)$ and I get exactly the same answers as Picard iteration using a much simpler single integral.
3. For (14) the series explodes at multiples of $t = \pi / 2$ On a follow-on, what is needed is  a result stating the interval in which a solution of the IVP exists and is nice and smooth (for a counterexample, take a ODE with a discontinuous right hand side).  BTW how to prove existence and uniqueness results is well known.
4. page 23: How do we 'use' the formula for $Y(t)$? For number crunching at least, just use an ODE solver.
5. Have you had a shot at the Riccati ODE which crops up here and there?

Some of these remarks have already been aired by @amike and myself.

Amin
Topic Author
Posts: 2046
Joined: July 14th, 2002, 3:00 am

Re: Breakthrough in the theory of stochastic differential equations and their simulation

Very constructive comments. Sorry for not being prompt to answer. Will get back in a day or two and try to answer most of your points. Thanks again for constructive discussion.

Cuchulainn
Posts: 56926
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Re: Breakthrough in the theory of stochastic differential equations and their simulation

You're welcome, Mr. Amin.

Amin
Topic Author
Posts: 2046
Joined: July 14th, 2002, 3:00 am

Re: Breakthrough in the theory of stochastic differential equations and their simulation

Sorry again for coming back to you so late.
OK I explain equations 3-4 here that are the basics and foundation of of iterated integral.
For one variable, we can have the following equation which is very straightforward for one variable calculus.
$y(\tau) =y(t_0) +\int_{t_0}^{\tau } \frac{\partial y(s)}{\partial s} \, ds$  Equation(1)
In the following two variable function, we apply the logic of above equation(1) to only the second variable and keep the first variable exactly the same untouched.
$f(\tau , y(\tau))=f\left(\tau , y(t_0)\right)+\int_{t_0}^{\tau } \frac{\partial f(\tau , y(s))}{\partial s} \, ds$  Equation(2)
From the chain rule for partial derivatives, we have
$\frac{\partial f(\tau , y(s))}{\partial s}=\frac{\partial f(\tau ,y(s))}{\partial y}\frac{\partial y(s))}{\partial s}$ Equation (3)
Integrating both sides, we get
$\int_{t_0}^{\tau } \frac{\partial f(\tau , y(s))}{\partial s} \, ds=\int _{t_0}^{\tau }\frac{\partial f(\tau ,y(s))}{\partial s}\frac{\partial y(s))}{\partial s}ds$  Equation(4)
substituting Equation(4) in equation(2), we get
$f(\tau , y(\tau))=f\left(\tau , y(t_0)\right)+\int _{t_0}^{\tau }\frac{\partial f(\tau , y(s))}{\partial s}\frac{\partial y(s))}{\partial s}ds$  Equation(5)
However, we know from the original ODE
$\frac{\partial y(s))}{\partial s}=f(s,y(s))$ Equation(6)
We substitute Equation(6) in equation(5) to get
$f(\tau , y(\tau))=f\left(\tau ,y(t_0) \right)+\int _{t_0}^{\tau }\frac{\partial f(\tau , y(s))}{\partial s}f(s, y(s))ds$  Equation(7)
Above Equation(5) is equation(3) in my paper and Equation(7) is equation(4) in my paper. If something is unclear, please let me know and I will try to explain. The iterated integral logic is repeated application of the above logic to higher derivatives.
I will come back to other comments but please let me know if you understand the logic and if the above explanation is good enough for that.

Amin
Topic Author
Posts: 2046
Joined: July 14th, 2002, 3:00 am

Re: Breakthrough in the theory of stochastic differential equations and their simulation

Here I explain the complete logic behind Iterated Integrals solution to ODEs a bit more in detail. In the next post, I will extend the iterated Integrals logic for SDEs and monte carlo.
We are given a general first order ODE
$\frac{\text{dy}(t)}{\text{dt}}=f(t, y(t)),t\geq t_0,\ y(t_0)=y_0$   Equation(1)
$y(t)=y(t_0)+\int_{t_0}^t \frac{dy(\tau)}{d\tau} \, d\tau$   Equation(2)
Below is the formal solution of the ODE
$= y(t_0)+\int _{t_0}^t f(\tau, y(\tau))d\tau$    Equation(3)
For one variable, we can have the following equation which is very straightforward for one variable calculus.
$y(\tau) =y(t_0) +\int_{t_0}^{\tau } \frac{d y(s)}{d s} \, ds$  Equation(4)
In the following two variable function, we apply the logic of above equation(4) to only the second variable and keep the first variable exactly the same untouched. As we are finding change for only one of the variables, we switch to partial derivatives in this equation.
$f(\tau , y(\tau))=f\left(\tau , y(t_0)\right)+\int_{t_0}^{\tau } \frac{\partial f(\tau , y(s))}{\partial s} \, ds$  Equation(5)
From the chain rule for partial derivatives, we have
$\frac{\partial f(\tau , y(s))}{\partial s}=\frac{\partial f(\tau ,y(s))}{\partial y}\frac{d y(s))}{d s}$ Equation (6)
Integrating both sides, we get
$\int_{t_0}^{\tau } \frac{\partial f(\tau , y(s))}{\partial s} \, ds=\int _{t_0}^{\tau }\frac{\partial f(\tau ,y(s))}{\partial y}\frac{d y(s))}{d s}ds$  Equation(7)
substituting Equation(7) in equation(5), we get
$f(\tau , y(\tau))=f\left(\tau , y(t_0)\right)+\int _{t_0}^{\tau }\frac{\partial f(\tau , y(s))}{\partial y}\frac{d y(s))}{d s}ds$  Equation(8)
However, we know from the original ODE
$\frac{d y(s))}{d s}=f(s,y(s))$ Equation(9)
We substitute Equation(9) in equation(8) to get
$f(\tau , y(\tau))=f\left(\tau ,y(t_0) \right)+\int _{t_0}^{\tau }\frac{\partial f(\tau , y(s))}{\partial y}f(s, y(s))ds$  Equation(10)
Integrating  the above equation (10) on both sides to match the equation(3), we get first order solution to our ODE
$y(t_0)+\int _{t_0}^t f(\tau, y(\tau))d\tau$
$=y(t_0)+\int _{t_0}^t f(\tau, y(t_0))d\tau+\int_{t_0}^t \int_{t_0}^{\tau } \frac{\partial f(\tau , y(s))}{\partial y}f(s, y(s))ds d\tau$ Equation(11)

Similarly taking the expansion to next order applying the same logic again yields the new higher order solution equation
$=y(t_0)+\int _{t_0}^t f(\tau, y(t_0))d\tau+\int_{t_0}^t \int_{t_0}^{\tau } \frac{\partial f(\tau , y(t_0))}{\partial y}f(s, y(t_0))ds d\tau$
$+\left.\int _{t_0}^t\int _{t_0}^{\tau }\int _{t_0}^s\frac{\partial }{\partial y}\left[\frac{\partial f(\tau , y(v))}{\partial y}f(s, y(v)\right.\right]f(v, y(v))dvdsd\tau$    Equation(12)

In the next post, I will extend the logic of iterated integrals to monte carlo simulation of the SDEs so that highly non-linear stochastic differential equations could be tackled easily by higher order monte carlo with larger monte carlo step size.

Amin
Topic Author
Posts: 2046
Joined: July 14th, 2002, 3:00 am

Re: Breakthrough in the theory of stochastic differential equations and their simulation

Suppose we want to solve the following general SDE

$\text{dX} (\tau) = \mu (X(\tau)) \text{d\tau }+ \sigma(X(\tau)) \text{dz}( \tau)$  Equation(1)

The formal solution of the above SDE is given as
$X(t)=X(t_0)+\int_{t_0}^t \mu (X(\tau)) \, d\tau+\int _{t_0}^t\sigma(X(\tau)) dz(\tau )$  Equation(2)

We want to solve both integrand terms in above equation as
$\mu (X(\tau))=\mu (X(t_0))+\int_{t_0}^\tau d\mu (X(s))$   Equation(3)
and
$\sigma (X(\tau))=\sigma (X(t_0))+\int_{t_0}^\tau d\sigma (X(s))$  Equation(4)
When we substitute above equation(3) and equation(4) in equation(2), we get

$X(t)=X(t_0)+\int_{t_0}^t \mu (X(t_0)) \, d\tau+\int_{t_0}^t \int_{t_0}^\tau d\mu (X(s)) d\tau$
$+\int _{t_0}^t\sigma(X(t_0)) dz(\tau )+\int _{t_0}^t \int_{t_0}^\tau d\sigma (X(s)) dz(\tau )$  Equation(5)

From Ito expansion of $d\mu (X(s))$ and $d\sigma (X(s))$, we know that
$d\mu (X(s))=\frac{\partial }{\partial X}[\mu ( X(s))]\mu (X(s)) \text{ds}+\frac{\partial }{\partial X}[\mu (X(s))]\sigma (X(s)) \text{dz(s)}$
$+0.5 \frac{\partial ^2[\mu ( X(s))]}{\partial X^2} \sigma ( X(s))^2\text{ds}$ Equation(6)

and
$d\sigma ( X(s))=\frac{\partial }{\partial X}[\sigma ( X(s))]\mu (X(s)) \text{ds}+\frac{\partial }{\partial X}[\sigma (X(s))]\sigma (X(s)) \text{dz(s)}$
$+0.5 \frac{\partial ^2[\sigma ( X(s))]}{\partial X^2} \sigma ( X(s))^2\text{ds}$  Equation(7)

We substitute equation (6) and Equation(7) in equation(5)

$X(t)=X(t_0)+\int_{t_0}^t \mu (X(t_0)) \, d\tau$
$+\int_{t_0}^t \int_{t_0}^\tau \frac{\partial }{\partial X}[\mu ( X(s))]\mu (X(s)) \text{ds} d\tau+\int_{t_0}^t \int_{t_0}^\tau \frac{\partial }{\partial X}[\mu (X(s))]\sigma (X(s)) \text{dz(s)} d\tau$
$+\int_{t_0}^t \int_{t_0}^\tau 0.5 \frac{\partial ^2[\mu ( X(s))]}{\partial X^2} \sigma ( X(s))^2\text{ds} d\tau$
$+\int _{t_0}^t\sigma(X(t_0)) dz(\tau )$
$+\int _{t_0}^t \int_{t_0}^\tau \frac{\partial }{\partial X}[\sigma ( X(s))]\mu (X(s)) \text{ds} dz(\tau )$
$+\int _{t_0}^t \int_{t_0}^\tau \frac{\partial }{\partial X}[\sigma ( X(s))]\sigma (X(s)) dz(s ) dz(\tau )$
$+\int_{t_0}^t \int_{t_0}^\tau 0.5 \frac{\partial ^2[\sigma ( X(s))]}{\partial X^2} \sigma ( X(s))^2\text{ds} dz(\tau )$ Equation(8)
In a further order expansion, we evaluate terms at time 's' at $t_0$ and add a third order integral. I am not showing the third order integral but up to second order, we will have

$X(t)=X(t_0)+\int_{t_0}^t \mu (X(t_0)) \, d\tau$
$+\int_{t_0}^t \int_{t_0}^\tau \frac{\partial }{\partial X}[\mu ( X(t_0))]\mu (X(t_0)) \text{ds} d\tau+\int_{t_0}^t \int_{t_0}^\tau \frac{\partial }{\partial X}[\mu (X(t_0))]\sigma (X(t_0)) \text{dz(s)} d\tau$
$+\int_{t_0}^t \int_{t_0}^\tau 0.5 \frac{\partial ^2[\mu ( X(t_0))]}{\partial X^2} \sigma ( X(t_0))^2\text{ds} d\tau$
$+\int _{t_0}^t\sigma(X(t_0)) dz(\tau )$
$+\int _{t_0}^t \int_{t_0}^\tau \frac{\partial }{\partial X}[\sigma ( X(t_0))]\mu (X(t_0)) \text{ds} dz(\tau )$
$+\int _{t_0}^t \int_{t_0}^\tau \frac{\partial }{\partial X}[\sigma ( X(t_0))]\sigma (X(t_0)) dz(s ) dz(\tau )$
$+\int_{t_0}^t \int_{t_0}^\tau 0.5 \frac{\partial ^2[\sigma ( X(t_0))]}{\partial X^2} \sigma ( X(t_0))^2\text{ds} dz(\tau )$  Equation(9)

Since all the terms are evaluated at $t_0$, they are constants and we can write the above integrals as
$X(t)=X(t_0)+\mu (X(t_0)) \int_{t_0}^t d\tau$
$+\frac{\partial }{\partial X}[\mu ( X(t_0))]\mu (X(t_0)) \int_{t_0}^t \int_{t_0}^\tau \text{ds} d\tau$
$+ \frac{\partial }{\partial X}[\mu (X(t_0))]\sigma (X(t_0)) \int_{t_0}^t \int_{t_0}^\tau \text{dz(s)} d\tau$
$+0.5 \frac{\partial ^2[\mu ( X(t_0))]}{\partial X^2} \sigma ( X(t_0))^2 \int_{t_0}^t \int_{t_0}^\tau \text{ds} d\tau$
$+\sigma(X(t_0)) \int _{t_0}^t dz(\tau )$
$+\frac{\partial }{\partial X}[\sigma ( X(t_0))]\mu (X(t_0)) \int _{t_0}^t \int_{t_0}^\tau \text{ds} dz(\tau )$
$+\frac{\partial }{\partial X}[\sigma ( X(t_0))]\sigma (X(t_0))\int _{t_0}^t \int_{t_0}^\tau dz(s ) dz(\tau )$
$+0.5 \frac{\partial ^2[\sigma ( X(t_0))]}{\partial X^2} \sigma ( X(t_0))^2\int_{t_0}^t \int_{t_0}^\tau \text{ds} dz(\tau )$  Equation(10)

The integrals of the kind  $\int_{t_0}^t \int_{t_0}^\tau \text{ds} d\tau$,  $\int_{t_0}^t \int_{t_0}^\tau \text{dz(s)} d\tau$ and $\int _{t_0}^t \int_{t_0}^\tau dz(s ) dz(\tau )$ and other higher order integrals can be very easily analytically solved for monte carlo. These integrals also commute. I will post a monte carlo simulation code for CEV and lognormal process based on the above logic in a few days.

If you like my above posts, I am looking for consulting and contract work and you could contact me for that at my email anan2999(at)yahoo(dot)com

Amin
Topic Author
Posts: 2046
Joined: July 14th, 2002, 3:00 am

Re: Breakthrough in the theory of stochastic differential equations and their simulation

Very sorry for delay in posting the code for simulation based on above higher order monte carlo schemes. I will post full code in a few days but here is the solution to stochastic integrals that need to be specified for monte carlo. Here is the solution to those integrals.
In the equations below in the context of monte carlo t is the  step size in time.
$\int _0^t dz(\tau )=\sqrt{\text{t}} Z$   Equation(1)
$\int _0^t\int _0^{\tau } dz(s) dz(\tau) =\frac{\text{t} \left(Z^2-1\right)}{2}$    Equation(2)
$\int _0^t\int _0^{\tau } ds dz(\tau )=\frac{\text{t}^{1.5} Z}{\sqrt{3}}$    Equation(3)
$\int _0^t\int_0^{\tau } dz(s) d\tau =\frac{\text{t}^{1.5} Z}{\sqrt{3}}$   Equation(4)
$\int _0^t\int _0^{\tau } dsd\tau =\frac{\text{t}^2}{2}$     Equation(5)

All the stochastic integrals share the same standard normal Z that we use in Equation(1). The above stochastic integrals are given in terms of one standard normal draw for each monte carlo path. When they start from zero, as in monte carlo in which all increments start over as if from time zero, all these double and higher order integrals commute.
I will soon post a full fledged matlab program for simulation using higher order monte carlo. For all those who wants to do it themselves, you have to use solution of above stochastic integrals Eq(1)-Eq(5) in Equation(10) of previous post.

Amin
Topic Author
Posts: 2046
Joined: July 14th, 2002, 3:00 am

Re: Breakthrough in the theory of stochastic differential equations and their simulation

Here is a matlab program that simulates higher order iterated integral monte carlo of lognormal diffusion with drift and compares it with traditional Euler method and also with exact analytic solution.

function [] = SVMonteCarloHigherOrderInfiniti( )
%SDE being discretized is
%dX(t)=mu0*X(t)*dt+simga0*X(t)*dz(t)
T_index=8; %No. of Simulation time steps
dt=.125/1.0; % Discretization spacing
TT=T_index*dt;%%Terminal Time
mu0=.40; %drift
sigma0=1.0;%volatility
X_0=1.0; %Starting Value of the lognormal diffusion
paths=1000000; %No Of Paths.
X1(1:paths)=X_0;%Iterated Integral Monte Carlo Variable
for nn=1:T_index %Loop over time.
Random0=randn(size(X));
%Stochastic integrals follow
dZn=Random0*sqrt(dt);
dZndt(1:paths)=dt.^1.5/sqrt(3)*Random0;
dtdZn(1:paths)=dt.^1.5/sqrt(3)*Random0;
dZndZn(1:paths)=dt/2.0.*(Random0.^2-1);
dtdt=dt.^2.0/2.0;

X=X+mu0.*X*dt+sigma0*X.*dZn; %Discretization of traditional Monte Carlo
X1=X1+mu0.*X1*dt+sigma0*X1.*dZn ... %%Discretization of Iterated Integral
+mu0.*mu0.*X1.*dtdt ...         %% Monte Carlo
+mu0.*sigma0.*X1.*dZndt ...
+sigma0.*mu0.*X1.*dtdZn ...
+sigma0.*sigma0.*X1.*dZndZn;

end
%Following block is the analytic solution of lognormal SDE with drift.
n=0;
for n=1:2000
n=n+1
Z(n)=n*.005;
Logf(n)=1.0/sqrt(2*pi)/Z(n)./(sigma0*sqrt(TT)).* ...
exp(-.5*(log(Z(n))-log(X_0)-(mu0-.5*sigma0.^2)*TT).^2./(sigma0.^2*TT));
end
%The next block turns monte carlo diffusion into density.
MaxCutOff=50;
BinSize=.01;
[XDensity,IndexOutX,IndexMaxX] = MakeDensityFromSimulation_Infiniti(X,paths,BinSize,MaxCutOff );
[XDensity1,IndexOutX1,IndexMaxX1] = MakeDensityFromSimulation_Infiniti(X1,paths,BinSize,MaxCutOff );

plot(IndexOutX(1:IndexMaxX),XDensity(1:IndexMaxX),'r',IndexOutX1(1:IndexMaxX1),XDensity1(1:IndexMaxX1),'g', ...
Z(1:2000),Logf(1:2000),'k')
%In the graph red line is traditional monte carlo with Euler method.
%green line is iterated integral monte carlo
%black line is analytic solution
str=input('look at overlaid Graph comparison of Monte carlo density of SDE using iterated integral(green line) with density of SDE generated from analytic method(black line)>');

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%5
Below is another function that is used for graphing the monte carlo diffusion and is called in the above program.

function [XDensity,IndexOut,IndexMax] = MakeDensityFromSimulation_Infiniti(X,Paths,BinSize,MaxCutOff )
%Processes monte carlo paths to return a series Xdensity as a function of IndexOut. IndexMax is the %maximum value of index.
%

Xmin=0;
Xmax=0;
for p=1:Paths
if(X(p)>MaxCutOff)
X(p)=MaxCutOff;
end
if(Xmin>real(X(p)))
Xmin=real(X(p));
end
if(Xmax<real(X(p)))
Xmax=real(X(p));
end
end

IndexMax=floor((Xmax-Xmin)/BinSize+.5)+1
XDensity(1:IndexMax)=0.0;
for p=1:Paths
index=real(floor(real(X(p)-Xmin)/BinSize+.5)+1);
if(real(index)<1)
index=1;
end
if(real(index)>IndexMax)
index=IndexMax;
end
XDensity(index)=XDensity(index)+1.0/Paths/BinSize;
end
IndexOut(1:IndexMax)=Xmin+(0:(IndexMax-1))*BinSize;
end

Attachments
SVMonteCarloHigherOrderInfiniti.zip
MakeDensityFromSimulation_Infiniti.zip

Amin
Topic Author
Posts: 2046
Joined: July 14th, 2002, 3:00 am

Re: Breakthrough in the theory of stochastic differential equations and their simulation

Here is post 32 in this thread made on Sat Apr 30, 2016 1:12 pm in which I have given systematic solution to stochastic integrals required for higher order monte carlo.

Amin:I will take a few more days to complete the paper but here are a few formulas for friends. Here I give the general algorithm for the type of integrals where only dz and dt integrals follow each other in any order. Like many other possibilities, one four order repeated integral with two dz uncertainties could be$\int _0^t \text{\sigma dz}(s)\int _0^s\text{du}\int _0^u\text{dv}\int _0^v\sigma \text{dz}(w)$The value of the above kind of integrals, as long as there is no explicit t, does not depend upon the sequence order(whether dz integral comes first or dt integral comes first does not change the value of the total integrals) so variance can be easily calculated by Ito Isometry. The order of hermite polynomial depends upon the number of normal uncertainties so is equal to number of dz integrals. Here is the simple formula to calculate the hermite polynomial representation of the integrals which equals$H_n(N)*\frac{\sqrt{\text{Variance}}}{\sqrt{n!}}$here n, which is the order of hermite polynomials, equals the number of normal uncertainties.   I will give a few examples
$\int _0^t \text{\sigma dz}(s)\int _0^s\text{du}=H_1(N)\frac{\text{\sigma t}^{1.5}}{\sqrt{1!}\sqrt{3}}$
$\int _0^t \text{\sigma dz}(s)\int _0^s\text{\sigma dz}(u)=H_2(N)\frac{\sigma ^2t}{\sqrt{2!}\sqrt{2}}$
$\int _0^t \text{\sigma dz}(s)\int _0^s\text{du}\int _0^u\text{\sigma dz}(v)=\int _0^t \text{\sigma dz}(s)\int _0^s\text{\sigma dz}(u)\int _0^u\text{dv}=\int _0^t \text{ds}\int _0^s\text{\sigma dz}(u)\int _0^u\text{\sigma dz}(v)=H_2(N)\frac{\sigma ^2t^2}{\sqrt{2!}\sqrt{12}}$
$\int _0^t \text{\sigma dz}(s)\int _0^s\text{du}\int _0^u\text{dv}=\int _0^t \text{ds}\int _0^s\text{\sigma dz}(u)\int _0^u\text{dv}=\int _0^t \text{ds}\int _0^s\text{du}\int _0^u\text{\sigma dz}(v)=H_1(N)\frac{\sigma ^2t^{2.5}}{\sqrt{1!}\sqrt{20}}$
$\int _0^t \text{\sigma dz}(s)\int _0^s\text{\sigma dz}(u)\int _0^u\text{\sigma dz}(v)=H_3(N)\frac{\sigma ^3t^{1.5}}{\sqrt{3!}\sqrt{6}}$
Fourth order and higher integrals can be evaluated to perfect precision using this method so I am not giving examples for that.

Here is Post 37 in this thread where I have explained higher order monte carlo written on Mon May 02, 2016 1:43 pm

Amin:Cuch, here you could see why those integrals are helpful. I have tried to explain in simple terms.We take a general stochastic differential equation.$x(t)=x(0)+\int _0^t\mu (x(s)) \text{ds}+\int _0^t\sigma (x(s)) \text{dz}(s)$  Eq(1)Where $\text{\mu (x(s))}$ and $\text{ \sigma (x(s))}$ are x dependent functions and are stochastic due to stochastic nature of x. It is unfortunate that when we use the above notation, we always substitute   $\text{\mu (x(0))}$ for   $\text{\mu (x(s))}$ and $\text{ \sigma (x(0))}$ for $\text{ \sigma (x(s))}$ as is standard in monte carlo and other numerical analysis and little effort has been made to understand the time dependent nature and evolution of  $\text{\mu (x(s))}$ and $\text{ \sigma (x(s))}$ in between the simulation intervals. As I earlier said that \text{$\mu$(x(s))} and $\text{ \sigma (x(s))}$  are functions that depend on x and are stochastic due to stochastic nature of x. We define the evolution of these functions as$\mu (x(s))=\mu (x(0))+\int _0^s \frac{\partial \mu (x(v))}{\partial x}\mu (x(v))\text{dv}+\int _0^s \frac{\partial \mu (x(v))}{\partial x}\sigma (x(v))\text{dz}(v)+.5\int _0^s \frac{\partial ^2\mu (x(v))}{\partial x^2}\sigma ^2(x(v)) \text{dv}$ Eq(2)$\sigma (x(s))=\sigma (x(0))+\int _0^s \frac{\partial \sigma (x(v))}{\partial x}\mu (x(v))\text{dv}+\int _0^s \frac{\partial \sigma (x(v))}{\partial x}\sigma (x(v))\text{dz}(v)+.5\int _0^s \frac{\partial ^2\sigma (x(v))}{\partial x^2}\sigma ^2(x(v)) \text{dv}$ Eq(3)We can easily make the observation that x dependent terms in the above integrals from 0 to s could be expanded by application of Ito formula into a time zero term and several other time dependent terms. We substitute equations 2 and 3 in Eq(1) to get$x(t)=x(0)+\int _0^t\left(\mu (x(0))+\int _0^s \frac{\partial \mu (x(v))}{\partial x}\mu (x(v))\text{dv}+\int _0^s \frac{\partial \mu (x(v))}{\partial x}\sigma (x(v))\text{dz}(v)+.5\int _0^s \frac{\partial ^2\mu (x(v))}{\partial x^2}.\sigma ^2(x(v)) \text{dv}\right) \text{ds}+$$\int _0^t\left(\sigma (x(0))+\int _0^s \frac{\partial \sigma (x(v))}{\partial x}\mu (x(v))\text{dv}+\int _0^s \frac{\partial \sigma (x(v))}{\partial x}\sigma (x(v))\text{dz}(v)+.5\int _0^s \frac{\partial ^2\sigma (x(v))}{\partial x^2}\sigma ^2(x(v)) \text{dv}\right) \text{dz}(s)$ Eq(4)Simplifying, we get$x(t)=x(0)+\int _0^t\mu (x(0))\text{ds}+\int _0^t\int _0^s \frac{\partial \mu (x(v))}{\partial x}\mu (x(v))\text{dvds}+\int _0^t\int _0^s \frac{\partial \mu (x(v))}{\partial x}\sigma (x(v))\text{dz}(v)\text{ds}+.5\int _0^t\int _0^s \frac{\partial ^2\mu (x(v))}{\partial x^2}\left.\sigma ^2(x(v)) \text{dv}\right) \text{ds}+\int _0^t\sigma (x(0))\text{dz}(s)+\int _0^t\int _0^s \frac{\partial \sigma (x(v))}{\partial x}\mu (x(v))\text{dvdz}(s)$$+\int _0^t\int _0^s \frac{\partial \sigma (x(v))}{\partial x}\sigma (x(v))\text{dz}(v)\text{dz}(s)+.5\int _0^t\int _0^s \frac{\partial ^2\sigma (x(v))}{\partial x^2}\sigma ^2(x(v)) \text{dv}\text{dz}(s)$  Eq(5) In the expression above only two single integrals $\int _0^t\mu (x(0))\text{ds}$and $\int _0^t\sigma (x(0))\text{dz}(s)$ depend upon time zero value of x and the rest of the double integrals all depend upon forward time dependent values and have to be expanded again similarly and this process could continue until derivatives existed(though we would generally always stop at third or fourth level if we could atain desired precision). We expand just the first double integral in Eq(5) further as an example$\int _0^t\int _0^s \frac{\partial \mu (x(v))}{\partial x}\mu (x(v))\text{dvds}=\int _0^t\int _0^s \frac{\partial \mu (x(0))}{\partial x}\mu (x(0))\text{dvds}+\int _0^t\int _0^s\int _0^v\frac{\partial }{\partial x} \left[\frac{\partial \mu (x(u))}{\partial x}\mu (x(u))\right]\mu (x(u))\text{dudvds}$$+\int _0^t\int _0^s\int _0^v\frac{\partial }{\partial x} \left[\frac{\partial \mu (x(u))}{\partial x}\mu (x(u))\right]\sigma (x(u))\text{dz}(u)\text{dvds}+\int _0^t\int _0^s\int _0^v.5\frac{\partial ^2}{\partial x^2} \left[\frac{\partial \mu (x(u))}{\partial x}\mu (x(u))\right]\sigma ^2(x(u))\text{dudvds}$   Eq(6)We could continue further but if further derivatives continue to exist, we have to ultimately truncate the expansion at some level and we could simply substitute time zero values of drift and volatility for forward time dependent values of drift and volatility. If we decide to stop the expansion at second level, we could write from Eq(5)  (Please notice that 2nd equality in equation below is approximate equality)$x(t)=x(0)+\int _0^t\mu (x(0))\text{ds}+\int _0^t\int _0^s \frac{\partial \mu (x(v))}{\partial x}\mu (x(v))\text{dvds}+\int _0^t\int _0^s \frac{\partial \mu (x(v))}{\partial x}\sigma (x(v))\text{dz}(v)\text{ds}+.5\int _0^t\int _0^s \frac{\partial ^2\mu (x(v))}{\partial x^2}\left.\sigma ^2(x(v)) \text{dv}\right) \text{ds}$$+\int _0^t\sigma (x(0))\text{dz}(s)+\int _0^t\int _0^s \frac{\partial \sigma (x(v))}{\partial x}\mu (x(v))\text{dvdz}(s)+\int _0^t\int _0^s \frac{\partial \sigma (x(v))}{\partial x}\sigma (x(v))\text{dz}(v)\text{dz}(s)+.5\int _0^t\int _0^s \frac{\partial ^2\sigma (x(v))}{\partial x^2}\sigma ^2(x(v)) \text{dv}\text{dz}(s)$$=x(0)+\int _0^t\mu (x(0))\text{ds}+\int _0^t\int _0^s \frac{\partial \mu (x(0))}{\partial x}\mu (x(0))\text{dvds}+\int _0^t\int _0^s \frac{\partial \mu (x(0))}{\partial x}\sigma (x(0))\text{dz}(v)\text{ds}+.5\int _0^t\int _0^s \frac{\partial ^2\mu (x(0))}{\partial x^2}\left.\sigma ^2(x(0)) \text{dv}\right) \text{ds}$$+\int _0^t\sigma (x(0))\text{dz}(s)+\int _0^t\int _0^s \frac{\partial \sigma (x(0))}{\partial x}\mu (x(0))\text{dvdz}(s)+\int _0^t\int _0^s \frac{\partial \sigma (x(0))}{\partial x}\sigma (x(0))\text{dz}(v)\text{dz}(s)+.5\int _0^t\int _0^s \frac{\partial ^2\sigma (x(0))}{\partial x^2}\sigma ^2(x(0)) \text{dv}\text{dz}(s)$$=x(0)+\mu (x(0))\int _0^t\text{ds}+\frac{\partial \mu (x(0))}{\partial x}\mu (x(0))\int _0^t\int _0^s \text{dvds}+\frac{\partial \mu (x(0))}{\partial x}\sigma (x(0))\int _0^t\int _0^s \text{dz}(v)\text{ds}+.5\frac{\partial ^2\mu (x(0))}{\partial x^2}\sigma ^2(x(0)) \int _0^t\int _0^s \text{dv} \text{ds}$$+\sigma (x(0))\int _0^t\text{dz}(s)+\frac{\partial \sigma (x(0))}{\partial x}\mu (x(0))\int _0^t\int _0^s \text{dvdz}(s)+\frac{\partial \sigma (x(0))}{\partial x}\sigma (x(0))\int _0^t\int _0^s \text{dz}(v)\text{dz}(s)+.5\frac{\partial ^2\sigma (x(0))}{\partial x^2}\sigma ^2(x(0))\int _0^t\int _0^s\text{dv}\text{dz}(s)$In case we wanted to go to fourth or hififth or even higher level, we could have continued the repeated expansion and finally replaced forward values at their time zero values but If we only wanted to continue to third level, we could have truncated the further series and replaced forward values by their time zero values and could have included integrals of the kind from Eq(6). Please recall that when we wrote Eq(6), we expanded just one integral and other integral terms have to be similarly expanded.(Please notice that 2nd equality in equation below is approximate equality)$\int _0^t\int _0^s \frac{\partial \mu (x(v))}{\partial x}\mu (x(v))\text{dvds}$$=\int _0^t\int _0^s \frac{\partial \mu (x(0))}{\partial x}\mu (x(0))\text{dvds}+\int _0^t\int _0^s\int _0^v\frac{\partial }{\partial x} \left[\frac{\partial \mu (x(u))}{\partial x}\mu (x(u))\right]\mu (x(u))\text{dudvds}+\int _0^t\int _0^s\int _0^v\frac{\partial }{\partial x} \left[\frac{\partial \mu (x(u))}{\partial x}\mu (x(u))\right]\sigma (x(u))\text{dz}(u)\text{dvds}+\int _0^t\int _0^s\int _0^v.5\frac{\partial ^2}{\partial x^2} \left[\frac{\partial \mu (x(u))}{\partial x}\mu (x(u))\right]\sigma ^2(x(u))\text{dudvds}$$=\frac{\partial \mu (x(0))}{\partial x}\mu (x(0))\int _0^t\int _0^s \text{dvds}+\frac{\partial }{\partial x} \left[\frac{\partial \mu (x(0))}{\partial x}\mu (x(0))\right]\mu (x(0))\int _0^t\int _0^s\int _0^v\text{dudvds}+\frac{\partial }{\partial x} \left[\frac{\partial \mu (x(0))}{\partial x}\mu (x(0))\right]\sigma (x(0))\int _0^t\int _0^s\int _0^v\text{dz}(u)\text{dvds}+.5\frac{\partial ^2}{\partial x^2} \left[\frac{\partial \mu (x(0))}{\partial x}\mu (x(0))\right]\sigma ^2(x(0))\int _0^t\int _0^s\int _0^v\text{dudvds}$Cuch, here is the answer to your question in an earlier post why these integrals are helpful. As you can see in the above equation, once we have taken initial time zero values of x dependent coefficients, the integrals to be evaluated are of the kind  $\int _0^t\int _0^s\int _0^v\text{dudvds}$,$\int _0^t\int _0^s\int _0^v\text{dz}(u)\text{dvds}$, $\int _0^t\int _0^s\int _0^v\text{dz}(u)\text{dz}(v)\text{ds}$ and $\int _0^t\int _0^s\int _0^v\text{dz}(u)\text{dz}(v)\text{dz}(s)$ which I mentioned how to evaluate in a previous post using ito isometry and simple hermite polynomials.
Last edited by Amin on December 10th, 2017, 2:17 pm

Amin
Topic Author
Posts: 2046
Joined: July 14th, 2002, 3:00 am

Re: Breakthrough in the theory of stochastic differential equations and their simulation

In light of post 32 and post 37 of this thread copied in the last post 567, I want to mention that I had fully specified the method of iterated integrals solution to SDEs as early as May 2016. Though I have written a formal paper on iterated integrals solution to ODEs, I have not written a formal paper on iterated integrals solution to SDEs. If any friend wants to do further research and discover new things using method of iterated integrals beyond what I have done or include method of iterated integrals in a book, I would request them to please give reference to my research that I deserve. I would be very thankful to friends who would give me credit that I deserve due to my original research.

I just noticed that latex in the above copied posts 35 and 37 of this thread is problematic despite that I copied it from there. Please refer to original post 32 and 37 on this thread here https://forum.wilmott.com/viewtopic.php?f=4&t=99702&start=30

Amin
Topic Author
Posts: 2046
Joined: July 14th, 2002, 3:00 am

Re: Breakthrough in the theory of stochastic differential equations and their simulation

On 1st of next year, I will post matlab code for simulation with stochastic calculus of standard deviations. This is different from what I have been doing like monte carlo simulation of SDEs with method of iterated integrals but method of iterated integrals is still used for the solution of Fokker planck with stochastic calculus of standard deviations. We could later solve for two dimensional distributions of the kind that arise in stochastic volatility models. Very exciting stuff and very innovative research on probability distributions and solution of Fokker planck equations.

Amin
Topic Author
Posts: 2046
Joined: July 14th, 2002, 3:00 am

Re: Breakthrough in the theory of stochastic differential equations and their simulation

Sorry Friends. It will be another day or two before I could post everything. This new version is very close to something people could use in a commercial industrial setting. I want to explain everything so people could understand it and be able to extend it on their own and do more research. It will still take two to three days before I could post an easily accessible explanation of the method and its underlying principles. I want to link every equation in my explanation to the ones in the code so people could easily change it on their own and do more research. It is promised to be very different and exciting for people in stochastics. I am sure in a few months, people would start doing all sort of new things like calibrations of SV models and backward monte carlo of derivatives in this new setting I have proposed.