The ENSO Forcing Potential - Cheaper, Faster, and Better

Following up on the last post on the ENSO forcing, this note elaborates on the math.  The tidal gravitational forcing function used follows an inverse power-law dependence, where a(t) is the anomalistic lunar distance and d(t) is the draconic or nodal perturbation to the distance.

F(t) \propto \frac{1}{(R_0 + a(t) + d(t))^2}'

Note the prime indicating that the forcing applied is the derivative of the conventional inverse squared Newtonian attraction. This generates an inverse cubic formulation corresponding to the consensus analysis describing a differential tidal force:

F(t) \propto -\frac{a'(t)+d'(t)}{(R_0 + a(t) + d(t))^3}

For a combination of monthly and fortnightly sinusoidal terms for a(t) and d(t) (suitably modified for nonlinear nodal and perigean corrections due to the synodic/tropical cycle)   the search routine rapidly converges to an optimal ENSO fit.  It does this more quickly than the harmonic analysis, which requires at least double the unknowns for the additional higher-order factors needed to capture the tidally forced response waveform. One of the keys is to collect the chain rule terms a'(t) and d'(t) in the numerator; without these, the necessary mixed terms which multiply the anomalistic and draconic signals do not emerge strongly.

As before, a strictly biennial modulation needs to be applied to this forcing to capture the measured ENSO dynamics — this is a period-doubling pattern observed in hydrodynamic systems with a strong fundamental (in this case annual) and is climatologically explained by a persistent year-to-year regenerative feedback in the SLP and SST anomalies.

Here is the model fit for training from 1880-1980, with the extrapolated test region post-1980 showing a good correlation.

The geophysics is now canonically formulated, providing (1) a simpler and more concise expression, leading to (2) a more efficient computational solution, (3) less possibility of over-fitting, and (4) ultimately generating a much better correlation. Alternatively, stated in modeling terms, the resultant information metric is improved by reducing the complexity and improving the correlation -- the vaunted  cheaper, faster, and better solution. Or, in other words: get the physics right, and all else follows.

 

 

 

 

 

 

 

 

 

 

 

 

 

Is the SOI noisy or is it signal?

Applying the analytical solution to Laplace's tidal equations, we can isolate the parts of the Southern Oscillation Index signal that appear quite noisy (i.e. 1880-1885, 1900-1905, etc).

For this 3-month averaged SOI fit, it's a sin(sin(f(t))) function in the ENSO model that generates the folded signal which appears as a rapidly fluctuating and noisy signal. Although my simplification of Laplace's equation was originally applied to QBO, it is applicable to other equatorial standing wave phenomenon such as ENSO, of which the SOI is a measure. The SOI signal has always been considered noisy — especially in contrast to other ENSO measures such as NINO34 — but perhaps this needs to be rethought, as the higher frequency components may be real signal.

These results will be presented at next month's AGU meeting:

Approximating the ENSO Forcing Potential

From the last post, we tried to estimate the lunar tidal forcing potential from the fitted harmonics of the ENSO model. Two observations resulted from that exercise: (1) the possibility of over-fitting to the expanded Taylor series, and (2) the potential of fitting to the ENSO data directly from the inverse power law.

The Taylor's series of the forcing potential is a power-law polynomial corresponding to the lunar harmonic terms. The chief characteristic of the polynomial is the alternating sign for each successive power (see here), which has implications for convergence under certain regimes. What happens with the alternating sign is that each of the added harmonics will highly compensate the previous underlying harmonics, giving the impression that pulling one signal out will scramble the fit. This is conceptually no different than eliminating any one term from a sine or cosine Taylor's series, which are also all compensating with alternating sign.

The specific conditions that we need to be concerned with respect to series convergence is when r (perturbations to the lunar orbit) is a substantial fraction of R (distance from earth to moon) :

F(r) = \frac{1}{(R+r)^3}

Because we need to keep those terms for high precision modeling, we also need to be wary of possible over-fitting of these terms — as the solver does not realize that the values for those terms have the constraint that they derive from the original Taylor's series. It's not really a problem for conventional tidal analysis, as the signals are so clean, but for the noisy ENSO time-series, this is an issue.

Of course the solution to this predicament is not to do the Taylor series harmonic fitting at all, but leave it in the form of the inverse power law. That makes a lot of sense — and the only reason for not doing this until now is probably due to the inertia of conventional wisdom, in that it wasn't necessary for tidal analysis where harmonics work adequately.

So this alternate and more fundamental formulation is what we show here.

Continue reading

Reverse Engineering the Moon's Orbit from ENSO Behavior

With an ideal tidal analysis, one should be able to apply the gravitational forcing of the lunar orbit1 and use that as input to solve Laplace's tidal equations. This would generate tidal heights directly. But due to aleatory uncertainty with respect to other factors, it becomes much more practical to perform a harmonic analysis on the constituent tidal frequencies. This essentially allows an empirical fit to measured tidal heights over a training interval, which is then used to extrapolate the behavior over other intervals.  This works very well for conventional tidal analysis.

For ENSO, we need to make the same decision: Do we attempt to work the detailed lunar forcing into the formulation or do we resort to an empirical bottoms-up harmonic analysis? What we have being do so far is a variation of a harmonic analysis that we verified here. This is an expansion of the lunar long-period tidal periods into their harmonic factors. So that works well. But could a geophysical model work too?

Continue reading

Improved Solver Target Error Metric

In addition to the algorithm used for solving optimization problems, an important criteria is the form of the metric used to minimize the error or maximize the similarity between data and model.

The commonly used forms such as error variance (i.e. mean squared error) have issues related to how well they can navigate the search space. Other forms such as correlation coefficient (CC) often work better, but at the expense of losing track of scale.  This indicates that CC is better at matching the general characteristics of a specific shape than a pure error criteria. And if weighted, it can deal with noisy intervals.

In fact, the symbolic reasoner Eureqa features a proprietary metric referred to as a hybrid correlation coefficient.  From my experiences with the tool, hybrid version does qualitatively work better.

So in my quest to find an alternative metric, I came up with something related to the Cosine Similarity (CS) measure. As defined, CS is not that different from Pearson's correlation coefficient as it does not subtract the mean. But with a slight modification it's an excellent "starter" metric for initial exploration.

The new metric is essentially a +/- excursion matching criteria (EMC), which is important for a behavior as cyclically erratic about the origin as ENSO.

The algorithm for the EMC can be described as a ratio of two factors. The numerator is the sum of the multiplications of the model and data values. The denominator is the normalizing factor, which is the sum of the multiplication of the absolute values of each value.

EMC =  \frac{\sum x_i \cdot y_i}{\sum |x_i| \cdot |y_i|}

The resulting metric ranges from -1 to 1, with 1 being a perfect sign excursion matching, and -1 if all excursions had the sane magnitude but were reversed in sign.

This of course is not a perfect criteria as it will tend to force the minimal excursions to zero while maximizing the maximum excursions, instead of first normalizing them as the true CS does.

The evidence to how well it works is mainly based on observations in massive reductions in search time. For ENSO model optimization search, the EMC reduces the time it takes to get in the ballpark by 100×, so what could take an hour reduces to about a minute of computational time. It is important not to let it overfit, so wait until the metric starts to slow in its improvement before stopping the search and switching to the CC metric for the final stages optimization.

As it is so fast I have been using it for minimally filtered ENSO time series, where I can start from minimally seed sets of parameters. This gives more confidence that results are not correlated from one search optimization run to the next.

The EMC is therefore a great metric for randomizing searches. I can imagine using it in a scenario with different initialized seed values and then waiting a fixed time to return an interim solution, and then using the best of these in a more refined CC search.

Why it works so well is something I am still trying to explain. It is a more efficient computation than CC, but that is not enough to explain 100x.

 

 

 

Interface-Inflection Geophysics

This paper that a couple of people alerted me to is likely one of the most radical research findings that has been published in the climate science field for quite a while:

Topological origin of equatorial waves
Delplace, Pierre, J. B. Marston, and Antoine Venaille. Science (2017): eaan8819.

An earlier version on ARXIV was titled Topological Origin of Geophysical Waves, which is less targeted to the equator.

The scientific press releases are all interesting

  1. Science Magazine: Waves that drive global weather patterns finally explained, thanks to inspiration from bagel-shaped quantum matter
  2. Science Daily: What Earth's climate system and topological insulators have in common
  3. Physics World: Do topological waves occur in the oceans?

What the science writers make of the research is clearly subjective and filtered through what they understand.

Continue reading

The Earth's invisible "Saturn ring": The QBO

This is puzzling:

“This temperature and zonal wind structure resembles those of Earth’s quasi-biennial oscillation (QBO) and Jupiter’s quasiquadrennial oscillation (QQO), in which temperature anomalies and eastward/westward winds alternate in altitude”
Fouchet, T., et al. “An equatorial oscillation in Saturn’s middle atmosphere.” Nature 453.7192 (2008): 200.

And recently the final results of the Cassini spacecraft mission were in the news:

“The density wave is generated by the gravitational pull of Saturn’s moon Janus.”
Wild! Cassini Probe Spots Weird Waves in Saturn's Rings September 11, 2017 https://www.space.com/38114-weird-waves-saturn-rings-cassini-photo.html

But no one in the research literature has made the connection of the moon's orbit to the dynamics of the QBO.

From Fouchet et al, again

On Earth, the alternating wind regimes repeat at intervals that vary from 22 to 34 months, with an average period of about 28 months. On Jupiter, the equatorial stratospheric temperature exhibits a 4.4-year period and the equatorial zonal winds in the upper troposphere oscillate with a 4.5-year period. Long-term ground-based monitoring reveals a period of 14.7±0.9 terrestrial years on Saturn. The observational similarities between Saturn’s oscillation and the QBO and QQO are the strong equatorial confinement of temperature minima and maxima and associated shear layers, a stronger eastward than westward shear layer, and the bounding of the equatorial oscillation at latitudes 15–20° north and south. Temperatures near these latitudes are relatively high when equatorial temperatures are relatively low, and vice versa.
On Earth and Jupiter, the quasi-periodic oscillations are triggered by the interaction between upwardly propagating waves and the mean zonal flow.

Both Jupiter and Saturn have 4 significant moons,making the collective lunar orbit difficult to describe. It's possible that Saturn's and Jupiter's "QBO" are more like the Earth's upper stratosphere oscillations, which align to the semiannual period (0.5 year period). This is suggestive as the values for Jupiter and Saturn's "QBO" period are closer to 1/2 the planet's full calendar year period, as show below

Planet "year" length "QBO" period "QBO upper" period
Earth 1 year 2.37 years 0.5 year
Saturn 29.46 years
14.7±0.9 years
Jupiter 11.86 years
4.5 years

 

CW

Now that we have strong evidence that AMO and PDO follows the biennial modulated lunar forcing found for ENSO, we can try modeling the Chandler wobble in detail. Most geophysicists argue that the Chandler wobble frequency is a resonant mode with a high-Q factor, and that random perturbations drive the wobble into its characteristic oscillation. This then interferes against the yearly wobble, generating the CW beat pattern.

But it has really not been clearly established that the measure CW period is a resonant frequency.  I have a detailed rationale for a lunar forcing of CW in this post, and Robert Grumbine of NASA has a related view here.

The key to applying a lunar forcing is to multiply it by a extremely regular seasonal pulse, which introduces enough of a non-linearity to create a physically-aliased modulation of the lunar monthly signal (similar as what is done for ENSO, QBO, AMO, and PDO).

Continue reading

PDO

After spending several years on formulating a model of ENSO (then and now) and then spending a day or two on the AMO model, it's obvious to try the other well-known standing wave oscillation — specifically, the Pacific Decadal Oscillation (PDO). Again, all the optimization infrastructure was in place, with the tidal factors fully parameterized for automated model fitting.

This fit is for the entire PDO interval:

What's interesting about the PDO fit is that I used the AMO forcing directly as a seeding input. I didn't expect this to work very well since the AMO waveform is not similar to the PDO shape except for a vague sense with respect to a decadal fluctuation (whereas ENSO has no decadal variation to speak of).

Yet, by applying the AMO seed, the convergence to a more-than-adequate fit was rapid. And when we look at the primary lunar tidal parameters, they all match up closely. In fact, only a few of the secondary parameters don't align and these are related to the synodic/tropical/nodal related 18.6 year modulation and the Ms* series indexed tidal factors, in particular the Msf factor (the long-period lunisolar synodic fortnightly). This is rationalized by the fact that the Pacific and Atlantic will experience maximum nodal declination at different times in the 18.6 year cycle.

Continue reading