Deterministic and Stochastic Applied Physics

Pierre-Simon Laplace was one of the first mathematicians who took an interest in problems of probability and determinism.  It's surprising how much of the math and applied physics that Laplace developed gets used in day-to-day analysis. For example, while working on the ENSO and QBO analysis, I have invoked the following topics at some point:

  1. Laplace's tidal equations
  2. Laplace's equation
  3. Laplacian differential operator
  4. Laplace transform
  5. Difference equation
  6. Planetary and lunar orbital perturbations
  7. Probability methods and problems
    1. Inductive probability
    2. Bayesian analysis, e.g. the Sunrise problem
  8. Statistical methods and applications
    1. Central limit theorem
    2. Least squares
  9. Filling in holes of Newton's differential calculus
  10. Others here

Apparently he did so much and was so comprehensive that in some of his longer treatises he often didn't cite the work of others, making it difficult to pin down everything he was responsible for (evidently he did have character flaws).

In any case, I recall applying each of the above in working out some aspect of a problem. Missing was that Laplace didn't invent Fourier analysis but the Laplace transform is close in approach and utility.

When Laplace did all this research, he must have possessed insight into what constituted deterministic processes:

We may regard the present state of the universe as the effect of its past and the cause of its future. An intellect which at a certain moment would know all forces that set nature in motion, and all positions of all items of which nature is composed, if this intellect were also vast enough to submit these data to analysis, it would embrace in a single formula the movements of the greatest bodies of the universe and those of the tiniest atom; for such an intellect nothing would be uncertain and the future just like the past would be present before its eyes.

— Pierre Simon Laplace,
A Philosophical Essay on Probabilities[wikipedia]
This is summed up as:

He also seemed to be a very applied mathematician, as per a quote I have used before  “Probability theory is nothing but common sense reduced to calculation.”  Really nothing the least bit esoteric about any of Laplace's math, as it seemed always motivated by solving some physics problem or scientific observation. It appears that he wanted to explain all these astronomic and tidal problems in as simple a form as possible. Back then it may have been esoteric, but not today as his techniques have become part of the essential engineering toolbox. I have to wonder if Laplace were alive now whether he would agree that geophysical processes such as ENSO and QBO were equally as deterministic as the sun rising every morning or of the steady cyclic nature of the planetary and lunar orbits. And it wasn't as if Laplace possessed confirmation bias that behaviors were immediately deterministic; as otherwise he wouldn't have spent so much effort in devising the rules of probability and statistics that are still in use today, such as the central limit theorem and least squares.

Perhaps he would have glanced at the ENSO problem for a few moments, noticed that in no way that it was random, and then casually remarked with one his frequent idiomatic phrases:

"Il est aisé à voir que..."  ... or ..  ("It is easy to see that...").

It may have been so obvious that it wasn't important to give the details at the moment, only to fill in the chain of reasoning later.  Much like the contextEarth model for QBO, deriving from Laplace's tidal equations.

Where are the Laplace's of today that are willing to push the basic math and physics of climate variability as far as it will take them? It has seemingly jumped from Laplace to Lorenz and then to chaotic uncertainty ala Tsonis or mystifying complexity ala Lindzen. Probably can do much better than to punt like that ... on first down even !

Scaling El Nino

Recently, the rock climber Alex Honnold took a route up El Capitan without ropes.There's no room to fail at that. I prefer a challenge that one can fail at, and then keep trying.  This is the ascent to conquering El Nino:

The Free-thought Route*

Χ  Base camp:  ENSO (El Nino/Southern Oscillation) is a sloshing behavior, mainly in the thermocline where the effective gravity makes it sensitive to angular momentum changes.
Χ  Faster forcing cycles reinforce against the yearly cycle, creating aliased periods. How?
Χ  Monthly lunar tidal cycles provide the aliased factors: Numbers match up perfectly.
This aliasing also works for QBO, an atmospheric analog of ENSO.
Χ  A biennial meta-stability appears to be active. Cycles reinforce on alternating years.
Χ  The well-known Mathieu modulation used for sloshing simulations also shows a biennial character.
Machine learning experiments help ferret out these patterns.
Χ  The delay differential equation formulation matches up with the biennial Mathieu modulation with a delay of one-year.  That's the intuitive yearly see-saw that is often suggested to occur.
  The Chandler wobble also shows a tidal forcing tendency, as does clearly the earth's LOD (length-of-day) variations.
Χ  Integrating the DiffEq model provides a good fit, including long-term coral proxy records
Χ  Solving the Laplace tidal equation via a Sturm-Liouville expression along the equator helps explain details of QBO and ENSO
  Close inspection of sea-level height (SLH) tidal records show evidence of both biennial and ENSO characteristics
Δ Summit: Final validation of the geophysics comparing ENSO forcing against LOD forcing.

Model fits to ENSO using a training interval

The route encountered several dead-ends with no toe-holds or hand-holds along the way (e.g. the slippery biennial phase reversal, the early attempts at applying Mathieu equation). In retrospect many of these excursions were misguided or overly complex, but eventually other observations pointed to the obvious route.

This is a magnification of the fitting contour around the best forcing period values for ENSO. These pair of peak values are each found to be less than a minute apart from the known values of the Draconic cycle (27.2122 days) and Anomalistic cycle (27.5545 days).

The forcing comes directly from the angular momentum variations in the Earth's rotation. The comparison between what the ENSO model uses and what is measured via monitoring the length-of-day (LOD) is shown below:

 

 

*  This is not the precise route I took, but how I wish it was in hindsight.

The Lunar Geophysical Connection

The conjecture out of NASA JPL is that the moon has an impact on the climate greater than is currently understood:

Claire Perigaud (Caltech/JPL)
and
Has this research gone anywhere?  Looks as if has gone to this spin-off.
According to the current consensus, variability in wind is what contributes to forcing for behaviors such as the El Nino/Southern Oscillation (ENSO).
OK, but what forces the wind? No one can answer that apart from saying wind variability is just a part of the dynamic climate system.  And so we are lead to believe that a wind burst will cause an ENSO and then the ENSO event will create a significant disruptive transient to the climate much larger than the original wind stimulus. And that's all due to positive feedback of some sort.
I am only paraphrasing the current consensus.
A much more plausible and parsimonious explanation lies with external lunar forcing reinforced by seasonal cycles.

Continue reading

ENSO Proxy Validation

This is a straightforward validation of the ENSO model presented at last December's AGU.

What I did was use the modern instrumental record of ENSO — the NINO34 data set — as a training interval, and then tested across the historical coral proxy record — the UEP data set.

The correlation coefficient in the out-of-band region of 1650 to 1880 is excellent, considering that only two RHS lunar periods (draconic and anomalistic month) are used for forcing. As a matter of fact, trying to get any kind of agreement with the UEP using an arbitrary set of sine waves is problematic as the time-series appears nearly chaotic and thus requires may Fourier components to fit. With the ENSO model in place, the alignment with the data is automatic. It predicts the strong El Nino in 1877-1878 and then nearly everything before that.

Continue reading

Canonical Solution of Mathieu Equation for ENSO

From a previous post, we were exploring possible solutions to the Mathieu equation given a pulsed stimulus.  This is a more straightforward decomposition of the differential equation using a spreadsheet.

The Mathieu equation:

f''(t) + \omega_0^2 (1 + \alpha \cos(\nu t)) f(t) = F(t)


can be approximated as a difference equation, where the second derivative f''(t) is ~ (f(t)-2f(t-dt)+f(t-2dt))/dt. But perhaps what we really want is a difference to the previous year and determine if that is enough to reinforce the biennial modulation that we are seeing in the ENSO behavior.

Setting up a spreadsheet with a lag term and a 1-year-prior feedback term, we apply both the biennial impulse-modulated lunar forcing stimulus and a yearly-modulated Mathieu term.

Fig. 1: Training (in shaded blue) and test for different intervals.

I was surprised by how remarkable the approximate fit was in the recent post, but this more canonical analysis is even more telling. The number of degrees of freedom in the dozen lunar amplitude terms apparently has no impact on over-fitting, even on the shortest interval in the third chart. There is noise in the ENSO data no doubt, but that noise seems to be secondary considering how the fit seems to mostly capture the real signal. The first two charts are complementary in that regard — the fit is arguably better in each of the training intervals yet the test interval results aren't really that much different from the direct fit looking at it by eye.

Just like in ocean tidal analysis, the strongest tidal cycles dominate;  in this case the Draconic and Anomalistic monthly, the Draconic and Anomalistic fortnightly, and a Draconic monthly+Anomalistic fortnightly cross term are the strongest (described here). Even though there is much room for weighting these factors differently on orthogonal intervals, the Excel Solver fit hones in on nearly the same weighted set on each interval.  As I said in a previous post, the number of degrees of freedom apparently do not lead to over-fitting issues.

One other feature of this fit was an application of a sin() function applied to the result. This is derived from the Sturm-Liouville solution to Laplace's tidal equation used in the QBO analysis  — which works effectively to normalize the model to the data, since the correlation coefficient optimizing metric does not scale the result automatically.

Pondering for a moment, perhaps the calculus is not so different to work out after all:

C93Q9eVWsAA8N4Q[1]

(from @rabaath on Twitter)

A bottom-line finding is that there is really not much complexity to this unique tidal formulation model of ENSO.  But because of the uniqueness of the seasonal modulation that we apply, it just doesn't look like the more-or-less regular cycles contributing to sea-level height tidal data.  Essentially similar algorithms are applied to find the right weighting of tidal factors, but whereas the SLH tidal data shows up in daily readings, the ENSO data is year-to-year.

Further, the algorithm does not take more than a minute or two to finish fitting the model to the data. Below is a time lapse of one such trial. Although this isn't an optimal fit, one can see how the training interval solver adjustment (in the shaded region) pulls the rest of the modeled time-series into alignment with the out-of-band test interval data.

Shortest Training Fit for ENSO

This is remarkable. Using the spreadsheet linked in the last post, the figure below is a model of ENSO derived completely by a training fit over the interval 1900 to 1920, using the Nino3.4 data series and applying the precisely phased Draconic and Anomalistic long-period tidal cycles.

Fig. 1 : The ENSO model in red. The blue BG region is used for training of the lunar tidal amplitudes against the Nino3.4 data in green. That data is square root compacted to convert it to an equivalent velocity.

Not much more to say. There is a major disturbance starting in the mid-1980's, but that is known from a Takens embedding analysis described in the first paper in this post.

Tidal Model of ENSO

The input forcing to the ENSO model includes combinations of the three major lunar months modulated by the seasonal solar cycle. This makes it conceptually similar to an ocean tidal analysis, but for ENSO we are more concerned about the long-period tides rather than the diurnal and semi-diurnal cycles used in conventional tidal analysis.

The three constituent lunar month factors are:

Month type Length in days
anomalistic 27.554549
tropical 27.321582
draconic 27.212220

So the essential cyclic terms are the following phased sinusoids

Continue reading

Solution to Pulsed Mathieu Equation

Here is a bit of applied math that I have never seen described before.  It considers solving a variant of the Mathieu differential equation, an unwieldy beast that finds application in models of fluid sloshing (among others). Normally the Mathieu equation is only solvable as a stimulus forcing function convolved against the transcendental Mathieu function. Tools such as Mathematica include the Mathieu function in their library, but any other solution would require a full numerical DiffEq integration.

This is the typical Mathieu equation formulation:

f''(t) + ( \omega_0^2 + B cos (\nu_0 t) ) f(t) = 0


But instead of this nonlinear time-varying DiffEq, consider that we replace the sinusoidal modulation with a delta pulse. This is precisely our model of ENSO that we are trying to create from first principles. The pulse represents a forcing impulse from the alignment of a lunar and solar tidal cycle.

f''(t) + ( \omega_0^2 + B \delta (t) ) f(t) = 0


Taking the Laplace transform, we quickly arrive at what looks like an ordinary 2nd-order differential equation solution, albeit with an interesting initial condition:

s^2 F(s) + \omega_0^2 F(s) + B f(0) = 0


After taking the inverse Laplace transform, we have:

f(t) = I(t) \ast B f(0) \delta(t)


where I(t) represents the impulse response of the first two terms, which is then convolved with a delta function evaluated at a value of f(t) for t=0. This is the only remnant of the nonlinear nature of the classical Mathieu function, in that the initial condition has a scaling proportional to the value of the function at that time. On the other hand, the solution to an ordinary DiffEq would not be dependent on the value of the function.

Thereafter we can extend this to a general solution; by creating a pulse train of delta functions we get this final convolution:

f(t) = I(t) \ast B \sum_{n=0}^{\infty} f(t - n T) \delta(t - n T)


where T is the pulse train period.

This is straightforward to evaluate for any pulse train -- all we have to do is keep track of the changing value of f(t) as we come across each pulse.

Fig 1: A pulse train with both annual and biannual contributions.

In the above figure, why does the annual pulse have a two-year periodicity? That's due to the alignment of a non-congruent tidal period with a seasonal pulse in terms of a Fourier series as described here and mathematically refined here.

If that is not intuitive, we can still consider it more of an anzat and see where it takes us.

For an ENSO time series, we an convolve the above delta pulse train with a set of known tidal periods of arbitrary amplitude and phase. For a fit trained up to 1980, this is the extrapolation post-1980:

Fig 2: Fit of pulsed Mathieu equation using a set of tidal periods as a forcing function. The impulse response here is a simple year-long rectangular window, i.e. the response has a memory of only a calendar year. This can be further refined if necessary. Yet, the projected time-series evaluated out-of-band from the fitting interval does a good job of capturing the ENSO profile.

The projected waveform matches the last 16 years very effectively considering how noisy the ENSO series is, and is very close to the fit over the entire interval. This is apart from the possible perturbation around the 1982 El Nino. The fact that the monthly tidal periods differ by slight amounts dictates that long multi-decadal intervals should be used for fitting. (In contrast, the model of QBO is primarily Draconic so the fitting interval can be much shorter)

Fig 3: Fit over the entire interval. The set of tidal periods applied was limited to the 3 major months (draconic, anomalistic, and sidereal/tropical) and the multiplicative combinations creating the fortnightly tides. This may not be enough to get the details right but erred on the side of under-fitting to establish the physical mechanism.

This approach is the logical follow-on to the wave transformed fitting approach that I had been using, most recently here and here. Both of these approaches are equally clever, which is a necessary ingredient when one is dealing with the unwieldy Mathieu equations. Moreover they both pull out the obvious stationary aspects of the ergodic ENSO time series.

QBO Split Training

As with ENSO, we can train QBO on separate intervals and compare the fit on each interval.  The QBO 30 hPa data runs from 1953 to the present.  So we take a pair of intervals — one from 1953-1983 (i.e. lower) and one from 1983-2013 (i.e. higher) — and compare the two.

The primary forcing factor is the seasonally aliased nodal or Draconic tide which is shown in the upper left on the figure.  The lower interval fit in BLUE matches extremely well to the higher interval fit in RED, with a correlation coefficient above 0.8.

These two intervals have no inherent correlation other than what can be deduced from the physical behavior generating the time-series.  The other factors are the most common long-period tidal cycles, along with the seasonal factor.  All have good correlations — even the aliased anomalistic tide (lower left), which features a pair of closely separated harmonics, clearly shows strong phase coherence over the two intervals.

That's what my AGU presentation was about — demonstrating how QBO and ENSO are simply derived from known geophysical forcing mechanisms applied to the fundamental mathematical geophysical fluid dynamics models. Anybody can reproduce the model fit with nothing more than an Excel spreadsheet and a Solver plugin.

Here are the PowerPoint slides from the presentation.

Short Training Intervals for ENSO

Given the fact that very short training intervals will reveal the underlying fundamental frequencies of QBO, could we do the same for ENSO? Not nearly as short, but about 40 years is the interval required to uncover the fundamental frequencies. Again none of this is possible unless we make the assumption of a phase reversal for ENSO between the years 1980 and 1996.

Continue reading