• 1
• 4
• 5
• 6
• 7
• 8 Cuchulainn
Topic Author
Posts: 64440
Joined: July 16th, 2004, 7:38 am
Location: Drosophila melanogaster
Contact:

### Re: Silly questions

It is BM, indeed. Maybe it is better to try to describe the problem (already discussed "ODE asymptotic .. thread (Aluffi, Gemam et al))

Recall

$d \vec{x}_t = -\nabla f(\vec{x_t}) dt + \sqrt{(2T)} d \vec{W}_t$ (1)

When there is no random fluctuation I can solve constrained optimisation as a gradient system. Now I want to do hlll-climbing (basin hopping) by solving (1).

My issues are

A) Solving (1) as a SDE is not ideal (?) we lose out on ODE solver
B) What I want is to use Strang  splitting into deterministiic and random legs, and which form the latter leg should be ie really the trigger for my question. So, what would be the 'best' approach for the stochastic part?

Here is idea for stochastic Burgers

https://arxiv.org/pdf/1907.12747.pdf
"Compatibility means deliberately repeating other people's mistakes."
David Wheeler

http://www.datasimfinancial.com
http://www.datasim.nl Alan
Posts: 10615
Joined: December 19th, 2001, 4:01 am
Location: California
Contact:

### Re: Silly questions

I may be missing something, but since the time-step $\Delta t$ is small, I would just do the usual simplest possible Monte Carlo for the 'random part':

At each step, draw a std-normally distributed n-vector $\vec{Z}$ and take
$\vec{x}_{t + \Delta t} = \vec{x}_t + c \vec{Z} \sqrt{\Delta t}$, where $c = \sqrt{2 T}$. Cuchulainn
Topic Author
Posts: 64440
Joined: July 16th, 2004, 7:38 am
Location: Drosophila melanogaster
Contact:

### Re: Silly questions

I may be missing something, but since the time-step $\Delta t$ is small, I would just do the usual simplest possible Monte Carlo for the 'random part':

At each step, draw a std-normally distributed n-vector $\vec{Z}$ and take
$\vec{x}_{t + \Delta t} = \vec{x}_t + c \vec{Z} \sqrt{\Delta t}$, where $c = \sqrt{2 T}$.
Yes, this would be one acceptable approach I reckon.

Let me explain the  full context using the equations here as baseline

https://arxiv.org/abs/1707.06618

The objective is to solve the SDE (1.2) by Algorithm 1 (GLD). No surprises really as it is essentially just Euler's method.

Now, here's the nub:

1. Euler for the deterministic part is shaky/brittle.
2.  What I want (heresy, it works in practice but does it work in theory) ?) is to split 2.1 into an ODE part and an SDE part.
3. Proposal; use Lie-Trotter/Strang operator splitting on the ODE/SDE in part 2.

Interested in the mathematical justification for this approach. If it works then we can find a global minimum (and we can send Gradient Descent out to pasture).
"Compatibility means deliberately repeating other people's mistakes."
David Wheeler

http://www.datasimfinancial.com
http://www.datasim.nl Alan
Posts: 10615
Joined: December 19th, 2001, 4:01 am
Location: California
Contact:

### Re: Silly questions

Can you show it works in practice" in an example? Cuchulainn
Topic Author
Posts: 64440
Joined: July 16th, 2004, 7:38 am
Location: Drosophila melanogaster
Contact:

### Re: Silly questions

Can you show it works in practice" in an example?
Yes! I even programmed the SGLD (Algorithm 2) which is what ML people do.
For the ODE part, that works very well, even with Lagrangians and KKT (equality, inequality constraints)

So, I need to resurrect and clean up my C++ code from last year (I build it from the ground up). The algorithm is easy

1. GLD (SDE)
2. Split 1) ODE, 2) SDE

I'll go back to bunker and get back soon! I think it will be a very useful exercise.

A test case is Rastrigin, many locals, 1 global.

https://en.wikipedia.org/wiki/Rastrigin_function
"Compatibility means deliberately repeating other people's mistakes."
David Wheeler

http://www.datasimfinancial.com
http://www.datasim.nl Cuchulainn
Topic Author
Posts: 64440
Joined: July 16th, 2004, 7:38 am
Location: Drosophila melanogaster
Contact:

### Re: Silly questions

The characteristic function of a probability distribution can be computed in closed form if we are lucky (Student t has no closed form). But CF solves an ODE (to be found) which is easier? For example, Student t CF $\phi_X$ satisfies:

$t\phi^{''}_X(t) - (\nu - 1)\phi^{'}_X(t) -\nu t\phi_X(t) = 0$ for $0 < t < \infty$

with "BC"

$\phi_X(0) = 1, |\phi_X(t)| < \infty$.

Q. Solve analytically or numerically (this part is a piece of cake)? Depending on the requirements(?)
"Compatibility means deliberately repeating other people's mistakes."
David Wheeler

http://www.datasimfinancial.com
http://www.datasim.nl Alan
Posts: 10615
Joined: December 19th, 2001, 4:01 am
Location: California
Contact:

### Re: Silly questions

The characteristic function of a probability distribution can be computed in closed form if we are lucky (Student t has no closed form). But CF solves an ODE (to be found) which is easier? For example, Student t CF $\phi_X$ satisfies:
wikipedia gives the closed-form cf. Cuchulainn
Topic Author
Posts: 64440
Joined: July 16th, 2004, 7:38 am
Location: Drosophila melanogaster
Contact:

### Re: Silly questions

The characteristic function of a probability distribution can be computed in closed form if we are lucky (Student t has no closed form). But CF solves an ODE (to be found) which is easier? For example, Student t CF $\phi_X$ satisfies:
wikipedia gives the closed-form cf.
This formula is from Hurst 1995/Joarder 1995 using weird and wonderful A&S style functions (must take ages to work it out??). It's the kind of thing you'd expect to be done in the 1930s.
I first looked in Cont and Tankov 2005 page 33 that there is no closed solution.

hence, the ODE solution which looks elegant, easy and generic.

https://arxiv.org/abs/1912.01245

More ways to skin a cat?

// This kind of maths has a charm all of its own.
"Compatibility means deliberately repeating other people's mistakes."
David Wheeler

http://www.datasimfinancial.com
http://www.datasim.nl Alan
Posts: 10615
Joined: December 19th, 2001, 4:01 am
Location: California
Contact:

### Re: Silly questions

Mathematica can do it: bessel.png (6.84 KiB) Viewed 798 times Cuchulainn
Topic Author
Posts: 64440
Joined: July 16th, 2004, 7:38 am
Location: Drosophila melanogaster
Contact:

### Re: Silly questions

Mathematica can do it:

bessel.png
Is that there an irregular mod cyl Bessel that I see before me, yonder?
https://en.cppreference.com/w/cpp/numer ... _functions
https://en.cppreference.com/w/cpp/numer ... l_bessel_k

Any good test values?
"Compatibility means deliberately repeating other people's mistakes."
David Wheeler

http://www.datasimfinancial.com
http://www.datasim.nl Alan
Posts: 10615
Joined: December 19th, 2001, 4:01 am
Location: California
Contact:

### Re: Silly questions

Mathematica can do it:

bessel.png
Is that there an irregular mod cyl Bessel that I see before me, yonder?
https://en.cppreference.com/w/cpp/numer ... _functions
https://en.cppreference.com/w/cpp/numer ... l_bessel_k

Any good test values?
yes, apparently cyl_bessel_k
For a test value, take nu=1, t = 1.2345, see answer at cyl_bessel_k link Cuchulainn
Topic Author
Posts: 64440
Joined: July 16th, 2004, 7:38 am
Location: Drosophila melanogaster
Contact:

### Re: Silly questions

For the 2nd question, I meant test value for the characteristic function.
"Compatibility means deliberately repeating other people's mistakes."
David Wheeler

http://www.datasimfinancial.com
http://www.datasim.nl Alan
Posts: 10615
Joined: December 19th, 2001, 4:01 am
Location: California
Contact:

### Re: Silly questions

For the 2nd question, I meant test value for the characteristic function.
But with $\nu=1$ the two differ only by trivial factors Cuchulainn
Topic Author
Posts: 64440
Joined: July 16th, 2004, 7:38 am
Location: Drosophila melanogaster
Contact:

### Re: Silly questions

Can $\frac{1}{2} \sigma^2 F^{2\beta} \frac{\partial^2 V}{\partial F^2}$ be transformed into sumping like

$\frac{1}{2} \sigma^2 \frac{\partial^2 V}{\partial y^2}$ , $y = f(F)$

without using Lamperti on the SDE?
"Compatibility means deliberately repeating other people's mistakes."
David Wheeler

http://www.datasimfinancial.com
http://www.datasim.nl