# Last post on ENSO

The last of the ENSO charts.

This is how conventional tidal prediction is done:

Note how well it does in extrapolating a projection from a training interval.

This is an ENSO model fit to SOI data using an analytical solution to Navier-Stokes. The same algorithm is used to solve for the optimal forcing as in the tidal analysis solution above, but applying the annual solar cycle and monthly/fortnightly lunar cycles instead of the diurnal and semi-diurnal cycle.

The time scale transitions from a daily modulation to a much longer modulation due to the long-period tidal factors being invoked.

Next is an expanded view, with the correlation coefficient of 0.73:

This is a fit trained on the 1880-1950 interval (CC=0.76) and cross-validated on the post-1950 data

This is a fit trained on the post-1950 interval (CC=0.77) and cross-validated on the 1880-1950 data

Like conventional tidal prediction, very little over-fitting is observed. Most of what is considered noise in the SOI data is actually the tidal forcing signal. Not much more to say, except for others to refine.

Thanks to Kevin and Keith for all their help, which will be remembered.

click to enlarge

# GC41B-1022: Biennial-Aligned Lunisolar-Forcing of ENSO: Implications for Simplified Climate Models

In the last month, two of the great citizen scientists that I will be forever personally grateful for have passed away. If anyone has followed climate science discussions on blogs and social media, you probably have seen their contributions.

Keith Pickering was an expert on computer science, astrophysics, energy, and history from my neck of the woods in Minnesota. He helped me so much in working out orbital calculations when I was first looking at lunar correlations. He provided source code that he developed and it was a great help to get up to speed. He was always there to tweet any progress made. Thanks Keith

Kevin O'Neill was a metrologist and an analysis whiz from Wisconsin. In the weeks before he passed, he told me that he had extra free time to help out with ENSO analysis. He wanted to use his remaining time to help out with the solver computations. I could not believe the effort he put in to his spreadsheet, and it really motivated me to spending more time in validating the model. He was up all the time working on it because he was unable to lay down. Kevin was also there to promote the research on other blogs, right to the end. Thanks Kevin.

There really aren't too many people willing to spend time working analysis on a scientific forum, and these two exemplified what it takes to really contribute to the advancement of ideas. Like us, they were not climate science insiders and so will only get credit if we remember them.

# Machine Learning and the Climate Sciences

I've been applying equal doses of machine learning (and knowledge based artificial intelligence in general) and physics in my climate research since day one. Next month on December 12, I will be presenting Knowledge-Based Environmental Context Modeling at the AGU meeting which will cover these topics within the earth sciences realm :

Table 1: Technical approach to knowledge-based model building for the earth sciences

In my opinion, machine learning likely will eventually find all the patterns that appear in climate time-series but with various degrees of human assistance.

"Vipin Kumar, a computer scientist at the University of Minnesota in Minneapolis, has used machine learning to create algorithms for monitoring forest fires and assessing deforestation. When his team tasked a computer with learning to identify air-pressure patterns called teleconnections, such as the El Niño weather pattern, the algorithm found a previously unrecognized example over the Tasman Sea."

In terms of the ENSO pattern, I believe that machine learning through tools such as Eureqa could have found the underlying lunisolar forcing pattern, but would have struggled mightily to break through the complexity barrier. In this case, the complexity barrier is in (1) discovering a biennial modulation which splits all the spectral components and (2) discovering the modifications to the lunar cycles from a strictly sinusoidal pattern.

The way that Eureqa would have found this pattern would be through it's symbolic regression algorithm (which falls under the first row in Table 1 shown above). It essentially would start it's machine learning search by testing various combinations of sines and cosines and capturing the most highly correlated combinations for further expansion.   As it expands the combinations, the algorithm would try to reduce complexity by applying trigonometric identities such as this

${\displaystyle \sin(\alpha \pm \beta )=\sin \alpha \cos \beta \pm \cos \alpha \sin \beta }$

After a while, the algorithm will slow down under the weight of the combinatorial complexity of the search, and then the analyst would need to choose promising candidates from the complexity versus best-fit Pareto front. At this point one would need to apply knowledge of physical laws or mathematical heuristics which would lead to a potentially valid model.

So, in the case of the ENSO model, Eureqa could have discovered the (1) biennial modulation by reducing sets of trigonometric identities, and perhaps by applying a sin(A sin()) frequency modulation (which it is capable of) to discover the (2) second-order modifications to the sinusoidal functions, or (3) it could have been fed a differential equation structure to provide a hint to a solution  .... but, a human got there first by applying prior knowledge of signal processing and of the details in the orbital lunar cycles.

Yet as the Scientific America article suggests, that will likely not be the case in the future when the algorithms continue to improve and update their knowledge base with laws of physics.

This more sophisticated kind of reasoning involves the refined use of the other elements of Table 1.  For example, a more elaborate algorithm could have lifted an entire abstraction level out of a symbolic grouping and thus reduced its complexity. Or it could try to determine whether a behavior was stochastic or deterministic.  The next generation of these tools will be linked to knowledge-bases filled with physics patterns that are organized for searching and reasoning tasks. These will relate the problem under study to potential solutions automatically.

# The ENSO Forcing Potential - Cheaper, Faster, and Better

Following up on the last post on the ENSO forcing, this note elaborates on the math.  The tidal gravitational forcing function used follows an inverse power-law dependence, where a(t) is the anomalistic lunar distance and d(t) is the draconic or nodal perturbation to the distance.

$F(t) \propto \frac{1}{(R_0 + a(t) + d(t))^2}'$

Note the prime indicating that the forcing applied is the derivative of the conventional inverse squared Newtonian attraction. This generates an inverse cubic formulation corresponding to the consensus analysis describing a differential tidal force:

$F(t) \propto -\frac{a'(t)+d'(t)}{(R_0 + a(t) + d(t))^3}$

For a combination of monthly and fortnightly sinusoidal terms for a(t) and d(t) (suitably modified for nonlinear nodal and perigean corrections due to the synodic/tropical cycle)   the search routine rapidly converges to an optimal ENSO fit.  It does this more quickly than the harmonic analysis, which requires at least double the unknowns for the additional higher-order factors needed to capture the tidally forced response waveform. One of the keys is to collect the chain rule terms a'(t) and d'(t) in the numerator; without these, the necessary mixed terms which multiply the anomalistic and draconic signals do not emerge strongly.

As before, a strictly biennial modulation needs to be applied to this forcing to capture the measured ENSO dynamics — this is a period-doubling pattern observed in hydrodynamic systems with a strong fundamental (in this case annual) and is climatologically explained by a persistent year-to-year regenerative feedback in the SLP and SST anomalies.

Here is the model fit for training from 1880-1980, with the extrapolated test region post-1980 showing a good correlation.

The geophysics is now canonically formulated, providing (1) a simpler and more concise expression, leading to (2) a more efficient computational solution, (3) less possibility of over-fitting, and (4) ultimately generating a much better correlation. Alternatively, stated in modeling terms, the resultant information metric is improved by reducing the complexity and improving the correlation -- the vaunted  cheaper, faster, and better solution. Or, in other words: get the physics right, and all else follows.

# Reverse Engineering the Moon's Orbit from ENSO Behavior

With an ideal tidal analysis, one should be able to apply the gravitational forcing of the lunar orbit1 and use that as input to solve Laplace's tidal equations. This would generate tidal heights directly. But due to aleatory uncertainty with respect to other factors, it becomes much more practical to perform a harmonic analysis on the constituent tidal frequencies. This essentially allows an empirical fit to measured tidal heights over a training interval, which is then used to extrapolate the behavior over other intervals.  This works very well for conventional tidal analysis.

For ENSO, we need to make the same decision: Do we attempt to work the detailed lunar forcing into the formulation or do we resort to an empirical bottoms-up harmonic analysis? What we have being do so far is a variation of a harmonic analysis that we verified here. This is an expansion of the lunar long-period tidal periods into their harmonic factors. So that works well. But could a geophysical model work too?

# Interface-Inflection Geophysics

This paper that a couple of people alerted me to is likely one of the most radical research findings that has been published in the climate science field for quite a while:

Topological origin of equatorial waves
Delplace, Pierre, J. B. Marston, and Antoine Venaille. Science (2017): eaan8819.

An earlier version on ARXIV was titled Topological Origin of Geophysical Waves, which is less targeted to the equator.

The scientific press releases are all interesting

What the science writers make of the research is clearly subjective and filtered through what they understand.

# CW

Now that we have strong evidence that AMO and PDO follows the biennial modulated lunar forcing found for ENSO, we can try modeling the Chandler wobble in detail. Most geophysicists argue that the Chandler wobble frequency is a resonant mode with a high-Q factor, and that random perturbations drive the wobble into its characteristic oscillation. This then interferes against the yearly wobble, generating the CW beat pattern.

But it has really not been clearly established that the measure CW period is a resonant frequency.  I have a detailed rationale for a lunar forcing of CW in this post, and Robert Grumbine of NASA has a related view here.

The key to applying a lunar forcing is to multiply it by a extremely regular seasonal pulse, which introduces enough of a non-linearity to create a physically-aliased modulation of the lunar monthly signal (similar as what is done for ENSO, QBO, AMO, and PDO).

# PDO

After spending several years on formulating a model of ENSO (then and now) and then spending a day or two on the AMO model, it's obvious to try the other well-known standing wave oscillation — specifically, the Pacific Decadal Oscillation (PDO). Again, all the optimization infrastructure was in place, with the tidal factors fully parameterized for automated model fitting.

This fit is for the entire PDO interval:

What's interesting about the PDO fit is that I used the AMO forcing directly as a seeding input. I didn't expect this to work very well since the AMO waveform is not similar to the PDO shape except for a vague sense with respect to a decadal fluctuation (whereas ENSO has no decadal variation to speak of).

Yet, by applying the AMO seed, the convergence to a more-than-adequate fit was rapid. And when we look at the primary lunar tidal parameters, they all match up closely. In fact, only a few of the secondary parameters don't align and these are related to the synodic/tropical/nodal related 18.6 year modulation and the Ms* series indexed tidal factors, in particular the Msf factor (the long-period lunisolar synodic fortnightly). This is rationalized by the fact that the Pacific and Atlantic will experience maximum nodal declination at different times in the 18.6 year cycle.

# Second-Order Effects in the ENSO Model

For ocean tidal predictions, once an agreement is reached on the essential lunisolar terms, then the second-order terms are refined. Early in the last century Doodson catalogued most of these terms:

"Since the mid-twentieth century further analysis has generated many more terms than Doodson's 388. About 62 constituents are of sufficient size to be considered for possible use in marine tide prediction, but sometimes many fewer can predict tides to useful accuracy."

That's possibly the stage we have reached in the ENSO model.  There are two primary terms for lunar forcing (the Draconic and Anomalistic) cycles, that when mixed with the annual and biannual cycles, will reproduce the essential ENSO behavior.  The second-order effects are the  modulation of these two lunar cycles with the Tropical/Synodic cycle.  This is most apparent in the modification of the Anomalistic cycle. Although not as important as in the calculation of the Total Solar Eclipse times, the perturbation is critical to validating the ENSO model and to eventually using it to make predictions.

The variation in the Anomalistic period is described at the NASA Goddard eclipse page. They provide two views of the variation, a time-domain view and a histogram view.

 Time domain view Histogram view

Since NASA Goddard doesn't provide an analytical form for this variation, we can see if the ENSO Model solver can effectively match it via a best-fit search to the ENSO data. This is truly an indirect method.

First we start with a parametric approximation to the variation, described by a pair of successive frequency modulated (and full-wave rectified) terms that incorporate the Tropical-modified term, wm. The Anomalistic term is wa.

COS(wa*t+pa+c_1*ABS(SIN(wm*t+k_1*ABS(SIN(wm*t+k_2))+c_2)))

$\cos(\omega_a t+\phi_a+c_1 \cdot |\sin(\omega_m t+k_1 \cdot |\sin(\omega_m t+k_2)|+c_2)|)$

This can generate the cusped behavior observed, but the terms pa, c_1, c_2, k_1, and k_2 need to be adjusted to align to the NASA model. The solver will try to do this indirectly by fitting to the 1880-1950 ENSO interval.

Plotting in RED the Anomalistic time series and the histogram of frequencies embedded in the ENSO waveform, we get:

 Time domain view of model Histogram view of model

This captures the histogram view quite well, and the time-domain view roughly (in other cases it gives a better cusped fit).  The histogram view is arguably more important as it describes the frequency variation over a much wider interval than the 3-year interval shown.

What would be even more effective is to find the correct analytical representation of the Anomalistic frequency variation and then plug that directly into the ENSO model. That would provide another constraint to the solver, as it wouldn't need to spend time optimizing for a known effect.

Yet as a validation step, the fact that the solver detects the shape required to match the variation is remarkable in itself. The solver is obviously searching for the forcing needed to produce the ENSO waveform observed, and happens to use the precise parameters that also describe the second-order Anomalistic behavior.  That could happen by accident but in that case there have been too many happy accidents already, i.e. period match, LOD match, Eclipse match, QBO match, etc.

# Millennium Prize Problem: Navier-Stokes

Watched the hokey movie Gifted on a plane ride. Turns out that the Millennium Prize for mathematically solving the Navier-Stokes problem plays into the plot.

I am interested in variations of the Navier–Stokes equations that describe hydrodynamical flow on the surface of a sphere.  The premise is that such a formulation can be used to perhaps model ENSO and QBO.

The so-called primitive equations are the starting point, as these create constraints for the volume geometry (i.e. vertical motion much smaller than horizontal motion and fluid layer depth small compared to Earth's radius). From that, we go to Laplace's tidal equations, which are a linearization of the primitive equations.

I give a solution here, which was originally motivated by QBO.

Of course the equations are under-determined, so the only hope I had of solving them is to provide this simplifying assumption:

${\frac{\partial\zeta}{\partial\varphi} = \frac{\partial\zeta}{\partial t}\frac{\partial t}{\partial\varphi}}$

If you don't believe that this partial differential coupling of a latitudinal forcing to a tidal response occurs, then don't go further. But if you do, then: