Serving the Quantitative Finance Community

  • 1
  • 2
  • 3
  • 4
  • 5
  • 8
 
User avatar
Cuchulainn
Topic Author
Posts: 20252
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Does Cauchy sequence ever occur in the real world or is it all the minds of mathematicians?

May 3rd, 2018, 8:39 am

Someone asked that question.

It happens all the time; Cauchy made the concept of convergence, norms and metric spaces rigorous. It allows you to give sharp error bound in certain numerical processes. Basically, you want to compute an approximation to an unknown function F. What we do is to construct a computable  approximation G(h) which is essentially a Cauchy sequence. 

Without support for this you don't know how 'good' a process is and it is not possible to compute an upper bound on the accuracy as a function of certain defining parameters. It makes the output of numerical processes predictable.

BTW There are so many 'real worlds to choose from. All models are wrong; some are useful.

The problem is that some processes give answers but you do not know how accurate they are beyond extensive numerical experimentation on test suites.

Image
 
User avatar
Traden4Alpha
Posts: 3300
Joined: September 20th, 2002, 8:30 pm

Re: Does Cauchy sequence ever occur in the real world or is it all the minds of mathematicians?

May 3rd, 2018, 12:07 pm

Interesting!

Yet I notice everything you said speaks in mathematical terms. For example, you speak of having a "sharp error bound in certain numerical processes" rather than one on "physical processes."

Every physical system that I know of never settles. The systems are constantly changing under internal thermal vibrations, radioactive decay, evaporation/condensation, chemical changes, and gross physical perturbation. They all violate the convergence requirement of a true Cauchy sequence. I'd also suspect (but perhaps I'm wrong) that quite a few numerical processes form Cauchy sequences at the analytical level but fail to always converge at the numerical digital level due to oscillation around the LSB.

That said, I totally agree that Cauchy sequences are a useful mathematical approximation in engineering (the artificial world more so that the natural world) where we are most concerned with avoiding excessive divergence or gaining some reasonable level of convergence within a reasonable time or terms even if true convergence never occurs.

P.S. I do wonder if belief in Cauchy sequences is one of the reasons for financial crises. Equilibrium economics is a crock of you know what. Are housing prices in Amsterdam (or Ireland) a Cauchy sequence?????
 
User avatar
Cuchulainn
Topic Author
Posts: 20252
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Does Cauchy sequence ever occur in the real world or is it all the minds of mathematicians?

May 3rd, 2018, 1:03 pm

Every physical system that I know of never settles. 

Doesn't make it true.

You don't believe in steady state solutions, like your training nets? in fact, they are a discretised ODE and dt = 1/number of layers. I bet you didn't know that?
 
User avatar
Traden4Alpha
Posts: 3300
Joined: September 20th, 2002, 8:30 pm

Re: Does Cauchy sequence ever occur in the real world or is it all the minds of mathematicians?

May 3rd, 2018, 1:34 pm

Steady state solutions are a useful approximation as long as you don't look too closely or expect them to be that way for too long. That is, in physical systems, epsilon never quite reaches zero and often jumps to large values in time.

I believe more in punctuated equilibria than in steady states.
 
User avatar
Traden4Alpha
Posts: 3300
Joined: September 20th, 2002, 8:30 pm

Re: Does Cauchy sequence ever occur in the real world or is it all the minds of mathematicians?

May 3rd, 2018, 1:40 pm

P.S. Steady-state solutions prove bumble bees cannot fly. But then bumble bees, like most living things including people and their economies, are NOT in a steady state.
 
User avatar
Cuchulainn
Topic Author
Posts: 20252
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Does Cauchy sequence ever occur in the real world or is it all the minds of mathematicians?

May 3rd, 2018, 3:29 pm

I am not familiar with bumble bees' organisation. But my question on NNs again: BP is essentially steady state, yes?
 
User avatar
Traden4Alpha
Posts: 3300
Joined: September 20th, 2002, 8:30 pm

Re: Does Cauchy sequence ever occur in the real world or is it all the minds of mathematicians?

May 3rd, 2018, 3:43 pm

I am not familiar with bumble bees' organisation. But my question on NNs again: BP is essentially steady state, yes?
That seems to depend on the learning parameters and may be a bit of black art.

Isn't there a trade-off in the SGD parameters between overly-rapid convergence to a local minimum versus unstable oscillations?
 
User avatar
ppauper
Posts: 11729
Joined: November 15th, 2001, 1:29 pm

Re: Does Cauchy sequence ever occur in the real world or is it all the minds of mathematicians?

May 3rd, 2018, 4:11 pm

P.S.  Steady-state solutions prove bumble bees cannot fly.  But then bumble bees, like most living things including people and their economies, are NOT in a steady state.
Urban legend: Scientists once proved that bumblebees can’t fly.
 
User avatar
Traden4Alpha
Posts: 3300
Joined: September 20th, 2002, 8:30 pm

Re: Does Cauchy sequence ever occur in the real world or is it all the minds of mathematicians?

May 3rd, 2018, 4:26 pm

P.S.  Steady-state solutions prove bumble bees cannot fly.  But then bumble bees, like most living things including people and their economies, are NOT in a steady state.
Urban legend: Scientists once proved that bumblebees can’t fly.
Nice! But If you actually read the whole article, it shows exactly what I said: steady-state solutions for aerodynamic lift show that bumble bees can't produce enough of it. But, of course, the problem lies in the selection of the steady-state model for this physical system (and many other physical and natural systems).
 
User avatar
ppauper
Posts: 11729
Joined: November 15th, 2001, 1:29 pm

Re: Does Cauchy sequence ever occur in the real world or is it all the minds of mathematicians?

May 4th, 2018, 5:51 am

P.S.  Steady-state solutions prove bumble bees cannot fly.  But then bumble bees, like most living things including people and their economies, are NOT in a steady state.
Urban legend: Scientists once proved that bumblebees can’t fly.
Nice!  But If you actually read the whole article, it shows exactly what I said: steady-state solutions for aerodynamic lift show that bumble bees can't produce enough of it.  But, of course, the problem lies in the selection of the steady-state model for this physical system (and many other physical and natural systems).
If you actually read the whole article, you'll see
But that doesn’t prove that bees cannot fly. All it proves is that bees with smooth, rigid wings cannot glide, which you can show for yourself with a few dead bees and a little lacquer.
 
User avatar
Cuchulainn
Topic Author
Posts: 20252
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Does Cauchy sequence ever occur in the real world or is it all the minds of mathematicians?

May 4th, 2018, 2:11 pm

Back to Cauchy and where I address some of your misconceptions(?, or are you trying to wind me up:)):

Yet I notice everything you said speaks in mathematical terms. For example, you speak of having a "sharp error bound in certain numerical processes" rather than one on "physical processes."

I am talking about numerical methods that approximate physical processes. That did not address the latter does not give you the right to shoot down mathematics. 

Isn't there a trade-off in the SGD parameters between overly-rapid convergence to a local minimum versus unstable oscillations?

Let's look at it from this angle: let's say you want to find minimum of Rastrigin function (many local minima) and (S)GD will (if it is lucky) converge to one of them. And for discontinuous functions SGD will choke. So,. what you are talking about is the wrong Solver for the wrong problem, nothing more. Use DE if you want a robust global solver.

BTW GD was not invented in  MIT AI Lab but by Cauchy hisself in 1847 when doing his calculation into astronomy. That was his real-life input. Hopefully another urban myth dispelled. It would seem CS folk are using these GD algos without really understanding when/why/how/what-if ..


https://www.math.uni-bielefeld.de/docum ... claude.pdf

P.S. I do wonder if belief in Cauchy sequences is one of the reasons for financial crises. Equilibrium economics is a crock of you know what. Are housing prices in Amsterdam (or Ireland) a Cauchy sequence?????

Noise++. The reasons are well known, and were not caused by Cauchy sequences. Sounds a bit disingenuous, to he honest.
Actually, house prices only changed 1% in the last 300 years. So, it be an alternating series?

Image
Last edited by Cuchulainn on May 4th, 2018, 2:34 pm, edited 4 times in total.
 
User avatar
Traden4Alpha
Posts: 3300
Joined: September 20th, 2002, 8:30 pm

Re: Does Cauchy sequence ever occur in the real world or is it all the minds of mathematicians?

May 4th, 2018, 2:24 pm

Nice!  But If you actually read the whole article, it shows exactly what I said: steady-state solutions for aerodynamic lift show that bumble bees can't produce enough of it.  But, of course, the problem lies in the selection of the steady-state model for this physical system (and many other physical and natural systems).
If you actually read the whole article, you'll see
But that doesn’t prove that bees cannot fly. All it proves is that bees with smooth, rigid wings cannot glide, which you can show for yourself with a few dead bees and a little lacquer.
The steady-state solution "proof" of flight impossibility is a mathematical one which is embedded in a reductio ad absurdum argument about the validity of the state-state model.

Using models involves a commutative diagram:
1) a physical system maps to a symbolic model via an encoding
2) the symbolic system maps to a second symbolic system via mathematical processes
3) the second symbolic system maps back to a second predicted physical system via the inverse of the encoding
4) the second predicted physical system can be compared to observations of the original physical system

Step #2 involves unassailable mathematic logic which is why it can be so very powerful. Steps 1 and 3 (encoding/decoding) are the weak link in the system.

My deeper point was that Cauchy sequences may be an approximation for some things in the real world (as long one does not look too far ahead in time or too closely at tiny epsilons) but Cauchy sequences don't actually occur in the real world. The pure proven properties of Cauchy sequences are only in the minds of mathematicians. Engineers and physicists might use Cauchy sequences as an approximation but must always be aware that they do not actually exist in the physical world.
 
User avatar
Cuchulainn
Topic Author
Posts: 20252
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Does Cauchy sequence ever occur in the real world or is it all the minds of mathematicians?

May 4th, 2018, 2:36 pm

What mental models do engineers have in their heads? I think they work of a single monolithic model, like the difficult some people have when lattice models break down and then whoopsie-daisy negative probability is born.
Same issue as with using SGD.

 I think what's confusing you is that Cauchy is not used in the problem domain but in the numerical process that approximate the former.
 
User avatar
Traden4Alpha
Posts: 3300
Joined: September 20th, 2002, 8:30 pm

Re: Does Cauchy sequence ever occur in the real world or is it all the minds of mathematicians?

May 4th, 2018, 3:35 pm

Back to Cauchy and where I address some of your misconceptions(?, or are you trying to wind me up:)):

Yet I notice everything you said speaks in mathematical terms. For example, you speak of having a "sharp error bound in certain numerical processes" rather than one on "physical processes."

I am talking about numerical methods that approximate physical processes. That did not address the latter does not give you the right to shoot down mathematics. 
First, I'm not trying to wind you up.

Second, I'm not shooting down the mathematics itself. The math is beautiful. It's whether the math actually really applies to the physical system that may be problematic.

Even in the case of digital systems, Cauchy may fail. I'm sure that you've seen it yourself: a mathematical solution that has analytic proof that it forms a Cauchy sequence but when you write the code and run it you discover that it does not always converge because it hits some cycle of flipping the LSB. Doesn't that happen?
Isn't there a trade-off in the SGD parameters between overly-rapid convergence to a local minimum versus unstable oscillations?

Let's look at it from this angle: let's say you want to find minimum of Rastrigin function (many local minima) and (S)GD will (if it is lucky) converge to one of them. And for discontinuous functions SGD will choke. So,. what you are talking about is the wrong Solver for the wrong problem, nothing more. Use DE if you want a robust global solver.
Agreed! If you've got a better global solver for machine learning, please implement it because lots of people really need it.
BTW GD was not invented in  MIT AI Lab but by Cauchy hisself in 1847 when doing his calculation into astronomy. That was his real-life input. Hopefully another urban myth dispelled. It would seem CS folk are using these GD algos without really understanding when/why/how/what-if ..


https://www.math.uni-bielefeld.de/docum ... claude.pdf
I've never claimed anything about who invented what because it honestly does not matter to me. The invention matters. The inventor does not.
P.S. I do wonder if belief in Cauchy sequences is one of the reasons for financial crises. Equilibrium economics is a crock of you know what. Are housing prices in Amsterdam (or Ireland) a Cauchy sequence?????

Noise++. The reasons are well known, and were not caused by Cauchy sequences. Sounds a bit disingenuous, to he honest.
Actually, house prices only changed 1% in the last 300 years. So, it be an alternating series?

Image
Well there we may have to disagree. If you look at the foundations of economics, they are based on expectations of convergence of supply, demand, prices, etc. What is "equilibrium" but a synonym for convergence?
 
User avatar
lovenatalya
Posts: 187
Joined: December 10th, 2013, 5:54 pm

Re: Does Cauchy sequence ever occur in the real world or is it all the minds of mathematicians?

May 6th, 2018, 7:29 am

The real issue here concerns infinity vs. finiteness. 

In any kind of "real world", there is a minimal positive [$]\epsilon[$]. In numerical analysis, it is the computer floating point limit. In quantum mechanics, it is the Plank length. In optics, it is the wavelength of the light you are using. There is also the Heisenberg uncertainty principle lower bound when you consider two non-commutative operators. Correspondingly there is also the upper bound. With a given minimal [$]\epsilon[$], a Cauchy sequence can be defined, and its length is finite, since there is a lower bound on the difference amongst the sequence after some index.

The mathematical version of the Cauchy sequence is taking the aforementioned Cauchy sequence to infinity. The prototype of  Infinity is the natural number. We can understand the mathematical version or infinite version of the Cauchy sequence as the (countably) infinite intersection of all the physical versions or all the finite versions of Cauchy sequence with all kinds resolution/precisions.