After several detours and dead-ends, it looks as if I have locked on a plausible ENSO model, parsimonious with recent research. The sticky widget almost from day 1 was the odd behavior in the ENSO time-series that occurred starting around 1980 and lasting for 16 years. This has turned out to be a good news/bad news/good news opportunity. Good in the fact that applying a phase inversion during that interval allowed a continuous fit across the entire span. Yet it's bad in that it gives the impression of applying a patch, without a full justification for that patch. But then again good in that it explains why other researchers never found the deterministic behavior underlying ENSO -- applying conventional tools such as the Fourier transform aren't much help in isolating the phase shift (accepting Astudillo's approach).
Having success with the QBO model, I wasn't completely satisfied with the ENSO model as it stood at the beginning of this year. It didn't quite click in place like the QBO model did. I had been attributing my difficulties to the greater amount of noise in the ENSO data, but I realize now that's just a convenient excuse -- the signal is still in there. By battling through the inversion interval issue, the model has improved significantly. And once the correct forcing and Mathieu modulation is applied, the model locks in place to the data with the potential to work as well as a deterministic tidal prediction algorithm.
As a rewind, there were several essential ingredients that I originally thought would play into the ENSO model. From the ENSO's vague similarity to a Bloch wave, I expected a Mathieu modulation to the wave equation to play a role. And this was reinforced by my discovering that all current sloshing models feature a Mathieu modulating factor. That insight occurred about two years ago in the timeline.
Around that time, I also postulated that the Chandler wobble, the QBO of atmospheric winds, and TSI variations would provide a first-order set of forcing inputs. The Chandler wobble has continued to be an important factor, but the QBO caused some frustrations as it was tantalizing close in providing the right frequency, but not quite. The TSI was likely the one factor that turned out to be a bust. I needed a period of over 10 years to adequately model a required forcing frequency, and while TSI variations were the only factor close to that value, I never could reconcile how that slight of solar heating modulation could play that significant a role.
The use of long-period tidal factors came in to the picture when I started having good success with modeling the QBO. The caveat to this QBO/ENSO relationship is that QBO is really forced by a gravitational forcing, while ENSO is mainly the result of angular momentum changes -- with the understanding that a lunar gravitational pull can modulate the angular momentum.
As I indicated at the beginning, the phase transition of 1980 was both a curse and a blessing. When I realized that a biennial modulation could provide just the metastable initial conditions required to cause a phase inversion, that's when the model started to click in place. The Wang paper describing the triaxial wobble terms solidified the idea that there was a 14 year forcing term that could replace the place-holder TSI term. By modulating the 14 year term with a biennial factor, a period of around 2.33 years effectively mimicked the QBO forcing as well. This clearly showed up in the frequency spectrum as biennial modulated side-bands as shown below, and which confirmed the work of Kim.
So at this point, more than 2 years after I started looking at modeling ENSO, the ingredients are ( listed according to greater to lesser importance):
- A strict biennial modulation in the forcing and in the Mathieu modulation factor of the wave equation.
- A Chandler wobble envelope of period ~6.5 years to provide angular momentum changes to force ENSO
- Another triaxial wobble term of ~14 years.
- A long period lunar nodal tidal factor corresponding to 18.6 years
- A pair of aliased lunar anomalistic tidal factors corresponding to 3.91 and 1.34 years (these are aliased from the 27.5545 day lunar period)
- Second-order harmonics and cross-terms of the above.
I apply either a DiffEq solver or a wave-equation transform for fitting the models to data. The latter has a faster turnaround and can fit models such as the following Figure 2. That's essentially what took two years in the making, following all the detours and circling back from dead-ends, and working the largely quiet-zone known as the Azimuth Project trying to drum up some interest in the project (no peep on the latest posts 🙁 at least not yet) .
But what's still exciting is now that the model has kind of locked in place, all the second order factors can be applied. I have noticed that the correlation coefficient is not even close to a maximum. That often happens with models that get close to the truth. Below is a scatter plot of the +/- excursions for a fit after 1940, data with the minimal amount of noise.
Below is the training interval, showing that even though the fit was optimized after 1940, the model works remarkably well in the years prior.
There are a few cases where the fit produces phantom peaks which do not show strongly in the data, such as a phantom La Nina (cold) excursion of 1936. The reason for this is definitely worth looking into.
As a comparison, the fit via a free tidal analysis programs is shown below. It uses a similar multiple regression fitting technique allowing between 5 to 35 tidal periods in the fit. Note below that there are also excursions beyond what is predicted -- in this case, due to the occasional intense storm surges. Similar to tide measurements being impacted by sporadic events, could it be that the ENSO of 1936 was an anomalous year, starting out with record cold in the Dakotas and continuing with one of the hottest summers on record in the Midwest? We may be able to figure that out with the help of this approach.
World Tides is a general purpose program for the analysis and prediction of tides. Using least squares harmonic analysis, it allows the user to decompose a water level record into its tidal and non-tidal components by fitting between 5 and 35 user-selectable tidal frequencies (tidal harmonic constituents). The constituents can then be saved to allow future prediction of tides. Non-tidal water level displayed in graphs provide a complete depiction of storm surge and storm tide (storm surge plus astronomic tide) during tropical and extratropical events.
As the ENSO model gets cleaned up, I will place it on the ContextEarth as an interactive chart. This will feature both the wave equation transform approach, and a DiffEq solution approach -- shown below with a hand optimization of the factors, and a 180 degree flipping of the biennial forcing starting at 1981. Although not optimal, it does exactly what it is supposed to do with a forcing phase inversion -- the response is phase inverted as well.
As it stands, this ENSO model is very close to the QBO model in terms of my having confidence that it they are reflecting the real geophysical processes that are occurring in the ocean and the atmosphere.
In the last post I mentioned the research of Sonechkin, which is actually quite intriguing. I will place a couple of Sonechkin's papers in the comments. Although Sonechkin's research is spotty and not very comprehensive on this topic, he has been suggesting over the last few decades that something just like what I have described is actually the fundamental driver behind ENSO. Sonechkin is also not a crank, with over 1000 citations to one of his co-authored papers (in Nature).