Scaling El Nino

Recently, the rock climber Alex Honnold took a route up El Capitan without ropes.There's no room to fail at that. I prefer a challenge that one can fail at, and then keep trying.  This is the ascent to conquering El Nino:

The Free-thought Route*

Χ  Base camp:  ENSO (El Nino/Southern Oscillation) is a sloshing behavior, mainly in the thermocline where the effective gravity makes it sensitive to angular momentum changes.
Χ  Faster forcing cycles reinforce against the yearly cycle, creating aliased periods. How?
Χ  Monthly lunar tidal cycles provide the aliased factors: Numbers match up perfectly.
This aliasing also works for QBO, an atmospheric analog of ENSO.
Χ  A biennial meta-stability appears to be active. Cycles reinforce on alternating years.
Χ  The well-known Mathieu modulation used for sloshing simulations also shows a biennial character.
Machine learning experiments help ferret out these patterns.
Χ  The delay differential equation formulation matches up with the biennial Mathieu modulation with a delay of one-year.  That's the intuitive yearly see-saw that is often suggested to occur.
  The Chandler wobble also shows a tidal forcing tendency, as does clearly the earth's LOD (length-of-day) variations.
Χ  Integrating the DiffEq model provides a good fit, including long-term coral proxy records
Χ  Solving the Laplace tidal equation via a Sturm-Liouville expression along the equator helps explain details of QBO and ENSO
  Close inspection of sea-level height (SLH) tidal records show evidence of both biennial and ENSO characteristics
Δ Summit: Final validation of the geophysics comparing ENSO forcing against LOD forcing.

Model fits to ENSO using a training interval

The route encountered several dead-ends with no toe-holds or hand-holds along the way (e.g. the slippery biennial phase reversal, the early attempts at applying Mathieu equation). In retrospect many of these excursions were misguided or overly complex, but eventually other observations pointed to the obvious route.

This is a magnification of the fitting contour around the best forcing period values for ENSO. These pair of peak values are each found to be less than a minute apart from the known values of the Draconic cycle (27.2122 days) and Anomalistic cycle (27.5545 days).

The forcing comes directly from the angular momentum variations in the Earth's rotation. The comparison between what the ENSO model uses and what is measured via monitoring the length-of-day (LOD) is shown below:

 

 

*  This is not the precise route I took, but how I wish it was in hindsight.

The Lunar Geophysical Connection

The conjecture out of NASA JPL is that the moon has an impact on the climate greater than is currently understood:

Claire Perigaud (Caltech/JPL)
and
Has this research gone anywhere?  Looks as if has gone to this spin-off.
According to the current consensus, variability in wind is what contributes to forcing for behaviors such as the El Nino/Southern Oscillation (ENSO).
OK, but what forces the wind? No one can answer that apart from saying wind variability is just a part of the dynamic climate system.  And so we are lead to believe that a wind burst will cause an ENSO and then the ENSO event will create a significant disruptive transient to the climate much larger than the original wind stimulus. And that's all due to positive feedback of some sort.
I am only paraphrasing the current consensus.
A much more plausible and parsimonious explanation lies with external lunar forcing reinforced by seasonal cycles.

Continue reading

ENSO Proxy Validation

This is a straightforward validation of the ENSO model presented at last December's AGU.

What I did was use the modern instrumental record of ENSO — the NINO34 data set — as a training interval, and then tested across the historical coral proxy record — the UEP data set.

The correlation coefficient in the out-of-band region of 1650 to 1880 is excellent, considering that only two RHS lunar periods (draconic and anomalistic month) are used for forcing. As a matter of fact, trying to get any kind of agreement with the UEP using an arbitrary set of sine waves is problematic as the time-series appears nearly chaotic and thus requires may Fourier components to fit. With the ENSO model in place, the alignment with the data is automatic. It predicts the strong El Nino in 1877-1878 and then nearly everything before that.

Continue reading

Canonical Solution of Mathieu Equation for ENSO

From a previous post, we were exploring possible solutions to the Mathieu equation given a pulsed stimulus.  This is a more straightforward decomposition of the differential equation using a spreadsheet.

The Mathieu equation:

f''(t) + \omega_0^2 (1 + \alpha \cos(\nu t)) f(t) = F(t)

can be approximated as a difference equation, where the second derivative f''(t) is ~ (f(t)-2f(t-dt)+f(t-2dt))/dt. But perhaps what we really want is a difference to the previous year and determine if that is enough to reinforce the biennial modulation that we are seeing in the ENSO behavior.

Setting up a spreadsheet with a lag term and a 1-year-prior feedback term, we apply both the biennial impulse-modulated lunar forcing stimulus and a yearly-modulated Mathieu term.

Fig. 1: Training (in shaded blue) and test for different intervals.

I was surprised by how remarkable the approximate fit was in the recent post, but this more canonical analysis is even more telling. The number of degrees of freedom in the dozen lunar amplitude terms apparently has no impact on over-fitting, even on the shortest interval in the third chart. There is noise in the ENSO data no doubt, but that noise seems to be secondary considering how the fit seems to mostly capture the real signal. The first two charts are complementary in that regard — the fit is arguably better in each of the training intervals yet the test interval results aren't really that much different from the direct fit looking at it by eye.

Just like in ocean tidal analysis, the strongest tidal cycles dominate;  in this case the Draconic and Anomalistic monthly, the Draconic and Anomalistic fortnightly, and a Draconic monthly+Anomalistic fortnightly cross term are the strongest (described here). Even though there is much room for weighting these factors differently on orthogonal intervals, the Excel Solver fit hones in on nearly the same weighted set on each interval.  As I said in a previous post, the number of degrees of freedom apparently do not lead to over-fitting issues.

One other feature of this fit was an application of a sin() function applied to the result. This is derived from the Sturm-Liouville solution to Laplace's tidal equation used in the QBO analysis  — which works effectively to normalize the model to the data, since the correlation coefficient optimizing metric does not scale the result automatically.

Pondering for a moment, perhaps the calculus is not so different to work out after all:

C93Q9eVWsAA8N4Q[1]

(from @rabaath on Twitter)

A bottom-line finding is that there is really not much complexity to this unique tidal formulation model of ENSO.  But because of the uniqueness of the seasonal modulation that we apply, it just doesn't look like the more-or-less regular cycles contributing to sea-level height tidal data.  Essentially similar algorithms are applied to find the right weighting of tidal factors, but whereas the SLH tidal data shows up in daily readings, the ENSO data is year-to-year.

Further, the algorithm does not take more than a minute or two to finish fitting the model to the data. Below is a time lapse of one such trial. Although this isn't an optimal fit, one can see how the training interval solver adjustment (in the shaded region) pulls the rest of the modeled time-series into alignment with the out-of-band test interval data.

Shortest Training Fit for ENSO

This is remarkable. Using the spreadsheet linked in the last post, the figure below is a model of ENSO derived completely by a training fit over the interval 1900 to 1920, using the Nino3.4 data series and applying the precisely phased Draconic and Anomalistic long-period tidal cycles.

Fig. 1 : The ENSO model in red. The blue BG region is used for training of the lunar tidal amplitudes against the Nino3.4 data in green. That data is square root compacted to convert it to an equivalent velocity.

Not much more to say. There is a major disturbance starting in the mid-1980's, but that is known from a Takens embedding analysis described in the first paper in this post.

Tidal Model of ENSO

The input forcing to the ENSO model includes combinations of the three major lunar months modulated by the seasonal solar cycle. This makes it conceptually similar to an ocean tidal analysis, but for ENSO we are more concerned about the long-period tides rather than the diurnal and semi-diurnal cycles used in conventional tidal analysis.

The three constituent lunar month factors are:

Month type Length in days
anomalistic 27.554549
tropical 27.321582
draconic 27.212220

So the essential cyclic terms are the following phased sinusoids

Continue reading

Solution to Pulsed Mathieu Equation

Here is a bit of applied math that I have never seen described before.  It considers solving a variant of the Mathieu differential equation, an unwieldy beast that finds application in models of fluid sloshing (among others). Normally the Mathieu equation is only solvable as a stimulus forcing function convolved against the transcendental Mathieu function. Tools such as Mathematica include the Mathieu function in their library, but any other solution would require a full numerical DiffEq integration.

This is the typical Mathieu equation formulation:

f''(t) + ( \omega_0^2 + B cos (\nu_0 t) ) f(t) = 0

But instead of this nonlinear time-varying DiffEq, consider that we replace the sinusoidal modulation with a delta pulse. This is precisely our model of ENSO that we are trying to create from first principles. The pulse represents a forcing impulse from the alignment of a lunar and solar tidal cycle.

f''(t) + ( \omega_0^2 + B \delta (t) ) f(t) = 0

Taking the Laplace transform, we quickly arrive at what looks like an ordinary 2nd-order differential equation solution, albeit with an interesting initial condition:

s^2 F(s) + \omega_0^2 F(s) + B f(0) = 0

After taking the inverse Laplace transform, we have:

f(t) = I(t) \ast B f(0) \delta(t)

where I(t) represents the impulse response of the first two terms, which is then convolved with a delta function evaluated at a value of f(t) for t=0. This is the only remnant of the nonlinear nature of the classical Mathieu function, in that the initial condition has a scaling proportional to the value of the function at that time. On the other hand, the solution to an ordinary DiffEq would not be dependent on the value of the function.

Thereafter we can extend this to a general solution; by creating a pulse train of delta functions we get this final convolution:

f(t) = I(t) \ast B \sum_{n=0}^{\infty} f(t - n T) \delta(t - n T)

where T is the pulse train period.

This is straightforward to evaluate for any pulse train -- all we have to do is keep track of the changing value of f(t) as we come across each pulse.

Fig 1: A pulse train with both annual and biannual contributions.

In the above figure, why does the annual pulse have a two-year periodicity? That's due to the alignment of a non-congruent tidal period with a seasonal pulse in terms of a Fourier series as described here and mathematically refined here.

If that is not intuitive, we can still consider it more of an anzat and see where it takes us.

For an ENSO time series, we an convolve the above delta pulse train with a set of known tidal periods of arbitrary amplitude and phase. For a fit trained up to 1980, this is the extrapolation post-1980:

Fig 2: Fit of pulsed Mathieu equation using a set of tidal periods as a forcing function. The impulse response here is a simple year-long rectangular window, i.e. the response has a memory of only a calendar year. This can be further refined if necessary. Yet, the projected time-series evaluated out-of-band from the fitting interval does a good job of capturing the ENSO profile.

The projected waveform matches the last 16 years very effectively considering how noisy the ENSO series is, and is very close to the fit over the entire interval. This is apart from the possible perturbation around the 1982 El Nino. The fact that the monthly tidal periods differ by slight amounts dictates that long multi-decadal intervals should be used for fitting. (In contrast, the model of QBO is primarily Draconic so the fitting interval can be much shorter)

Fig 3: Fit over the entire interval. The set of tidal periods applied was limited to the 3 major months (draconic, anomalistic, and sidereal/tropical) and the multiplicative combinations creating the fortnightly tides. This may not be enough to get the details right but erred on the side of under-fitting to establish the physical mechanism.

This approach is the logical follow-on to the wave transformed fitting approach that I had been using, most recently here and here. Both of these approaches are equally clever, which is a necessary ingredient when one is dealing with the unwieldy Mathieu equations. Moreover they both pull out the obvious stationary aspects of the ergodic ENSO time series.

QBO Split Training

As with ENSO, we can train QBO on separate intervals and compare the fit on each interval.  The QBO 30 hPa data runs from 1953 to the present.  So we take a pair of intervals — one from 1953-1983 (i.e. lower) and one from 1983-2013 (i.e. higher) — and compare the two.

The primary forcing factor is the seasonally aliased nodal or Draconic tide which is shown in the upper left on the figure.  The lower interval fit in BLUE matches extremely well to the higher interval fit in RED, with a correlation coefficient above 0.8.

These two intervals have no inherent correlation other than what can be deduced from the physical behavior generating the time-series.  The other factors are the most common long-period tidal cycles, along with the seasonal factor.  All have good correlations — even the aliased anomalistic tide (lower left), which features a pair of closely separated harmonics, clearly shows strong phase coherence over the two intervals.

That's what my AGU presentation was about — demonstrating how QBO and ENSO are simply derived from known geophysical forcing mechanisms applied to the fundamental mathematical geophysical fluid dynamics models. Anybody can reproduce the model fit with nothing more than an Excel spreadsheet and a Solver plugin.

Here are the PowerPoint slides from the presentation.

Short Training Intervals for ENSO

Given the fact that very short training intervals will reveal the underlying fundamental frequencies of QBO, could we do the same for ENSO? Not nearly as short, but about 40 years is the interval required to uncover the fundamental frequencies. Again none of this is possible unless we make the assumption of a phase reversal for ENSO between the years 1980 and 1996.

Continue reading

Solver vs Multiple Linear Regression for ENSO

In the previous ENSO post I referenced the Rajchenbach article on Faraday waves.

There is a telling assertion within that article:

"For instance, to the best of our knowledge, the dispersion relation (relating angular frequency ω and wavenumber k) of parametrically forced water waves has astonishingly not been explicitly established hitherto. Indeed, this relation is often improperly identified with that of free unforced surface waves, despite experimental evidence showing significant deviations"

What they are suggesting is that too much focus has been placed on natural resonances and the dispersion relationships within a free fluid volume. Whereas the forced response is clearly as important — if not more — and that the forcing will show through in the solution of the equations. I have been pursuing this strategy for a while, having started down the Mathieu equation right away and then eventually realizing the importance of the forced response, yet the Rajchenbach article is the first case that I have found made of what I always thought should be a rather obvious assumption. The fact that the peer-reviewers allowed the "astonishingly" adjective in the paper is what makes it telling. It's astonishing in the equivalent sense that Rajchenbach & Clamond are pointing out that a pendulum's motion will be impacted by a periodic push. In other words, astonishing in the sense that this premise should be obvious!

Continue reading