Skip to main content

Details on the October ENSO Diagnostic Discussion: Trust, but verify.

Do we sound like a broken record? The CPC/IRI El Niño-Southern Oscillation forecast released today is essentially unchanged from last month, with around 60-65% chance of El Niño, starting in October-November. Sea surface temperatures in the Niño3.4 region are +0.3°C over the last week, a downwelling Kelvin wave continues to transport warm water toward the eastern equatorial Pacific, and global climate models continue to call for the development of a weak El Niño.

Just how good are these models, though? In our last post, Tony discussed ENSO forecasts over the last few years, including prediction for El Niño in the fall of 2012 that never materialized. Here, I’ll take a look at one forecast system to see how it performed compared to the past 33 years of observed ENSO episodes.

While dynamical climate modeling has been around for decades, many of the current models have only been in existence for a few years (for details on dynamical prediction models, see the footnotes from Tony’s post). For example, NOAA’s operational climate model, the NCEP Climate Forecast System version 2 (CFSv2), started producing forecasts in 2011.  It is difficult to evaluate model performance with only a few years of forecasts. So, in order to get more years to study, we create forecasts based on historical records.

For example, we start with the observed state of the atmosphere and ocean in September 1997, and then let the computer model make a forecast out to 9 months (for the rest of 1997 and into 1998).  Since we already know what happened in 1997-1998, we can compare the observations against this “retrospective forecast” (also known as a “hindcast”) to determine how well the model performed.

One of the tools we use in making the ENSO forecast is the North American Multi-Model Ensemble (NMME) forecasting system, a project that incorporates several global climate models, including the CFSv2. I looked at the September forecasts for October-November-December (OND) from 1982 – 2013, to see how the forecast tracked with the observations (Fig. 1). This system has been active since 2011, so I used 29 years of hindcasts (1982 – 2010) and three years of archived real-time forecasts (2011 – 2013). The exact same versions of the models are used for both the hindcasts and the forecasts, allowing for a continuous data set.

The forecast I looked at was the area-averaged sea surface temperature in the Niño3.4 region, averaged over a three-month period (this is also called the Oceanic Niño Index (ONI), NOAA’s indicator of ENSO). In the 1982 – 2013 period, we had 10 El Niños winters and 12 La Niña winters; the rest were neutral years. The NMME forecast I used is the combination of all the models (“ensemble mean,” the red line in Fig. 2).

To get a “hit”, the system had to correctly forecast an ENSO event, based on five consecutive seasons with an ONI greater than +0.5°C (El Niño) or less than-0.5°C (La Niña.) The other possible outcomes are “miss”, when an ENSO event happened, but wasn’t forecast, and “false alarm”, when the forecast system called for an ENSO event, but it didn’t occur.

In the past 33 years, grouping all ENSO events together, the September forecast for October-November-December had 18 hits, 4 misses, and one false alarm (2012). The rest of the forecasts were “correct negatives” – forecasts for neutral conditions. The forecasts correlate with the observed ONI at a 0.95 coefficient. While this is a simplistic way of taking stock of the forecasts, it does give an idea of why we tend to trust the model forecasts made this time of the year, especially at such a short lead time.  At longer lead times, for example, the June forecasts for October-November-December, there are 13 hits, 8 misses, and 3 false alarms.

A few other elements are giving forecasters the confidence to stick with the 60-65% chance of El Niño. One, the CFSv2 appears to have forecast many of the changes in subsurface ocean temperatures since May (i.e. oceanic Kelvin waves). Also, there are currently some westerly wind anomalies in the western Pacific, which may encourage more eastward heat transport across the tropical Pacific.

Finally, atmospheric conditions, as represented by the Southern Oscillation Index, remained in a borderline El-Niño-friendly state throughout September (SOI = -0.7 at the end of September.) So, while this year’s ENSO forecasts may sound like a broken record as we wait to see if the forecast hits, the track record of model forecasts for past El Niños and the current state of the atmosphere and ocean tell us that the odds are still for an appearance of El Niño this year.

**Editor's note: A draft version of this post was accidentally posted at 11 a.m. EST. The post was updated with the final version at 12:23 p.m.

Comments

looking at this weighted assessment does the patterns in Portugal have any effect on this outcome as they just received heavy rains after a drought and their biome is similar to California?

Although Portugal and California are similar in their basic climate (e.g. both have a Mediterranean climate), what happens to one in response to an El Nino does not at all necessarily have bearing on what happens in to the other. Further, recent rains in Portugal have little influence on the ENSO situation. ENSO may affect rainfall, but the rainfall does not feedback onto ENSO in any substantial manner. Of course, Portugal has much less of a late winter impact from ENSO than does southern California.

In reply to by nathaniel taylor

To what extent are the models in questioned developed using historical data? The models could get good results on hindcasting because they make good predictions, or they could get good results on hindcasting because they have been tuned for that data-set already, meaning that the good hindcasting doesn't tell us much. Is it possible to distinguish these?

This is an excellent question. In developing dynamical  models, historical data is indeed used to validate/verify/evaluate model performance, and also to tune some of the parameters used in the models. Therefore, dynamical models are expected to do slightly better in hindcast model than in real-time mode. But the difference between their performance during the two periods (hindcast vs. real-time) is normally smaller than that found in statistical models, which train completely using hindcast data. A good portion of a dynamical model has explicit and completely represented physics, and that part is not tuned. Only where the physics needs to be abridged, as for example in representing tropical convection using grid points larger than the spatial scale of the convection, is there wiggle room for some tuning. But the question is very good. We note in the graph that the errors during the last 2 to 3 years, the real-time period, look a bit larger than those during the hindcast period. That should make us a little bit more worried about how well we can trust the model for the current prediction of weak El Nino coming in the next several months. If there is an error similar to that of the last 2 years (where the outcome was cooler than the forecast), we could end up getting only a marginal El Nino condition instead of a full-blown weak El Nino as predicted. On the other hand, assuming the error will be as that found during the last 2 years is also questionable. A sample of two cases is highly inadequate for coming to a conclusion or decision. The bottom line is that we must wait and see, and should trust the model to at least a moderate extent.

Can you talk about the three "missed" forecasts graphed above from the NMME that were not for an El Nino? Is the graph showing that the prediction was for a neutral condition, neither warm nor cold, and it turned out to be on the cold side? Also, re: the "missed" forecasts, do you happen to know if those somewhat chilly winter conditions led to an unexpectedly dry winter in CA?

The three missed forecasts for La Nina were those in the autumns of 2000, 2005 and 2008, as shown on the graph. All 3 cases ended up having weak La Nina. As for the ensuing winters in southern California (the part of the state most strongly affected by ENSO, especially in late winter), during Jan-Feb-Mar season we did observe drier than average conditions in 2001 (a few months after the missed forecasts were made), drier than average conditions in 2009, and in 2006 it was wetter than normal in central and northern California but near to slightly below normal in southern California. So we see that the observations did tend to follow the expected impacts, but not unfailingly. That is a typical rate of observing ENSO teleconnections--they show a tendency but not a guaranteed outcome. 

So, you are not prediticting, as of mid October 2014, a full blown El Nino?

Is there any satellite tracking of data markers, i.e. heat, water vapor, sea hgt, etc and recorded to yield a story/data used in forecasting El Nino/La Nina?

Yes, there are satellites which are relied upon to give us a complete picture of many tropical Pacific Ocean variables, such as sea surface temperatures, sea level height (which is an indicator of heat content below the surface of the ocean), and rainfall.  I'm not familiar *which* satellites are relied upon, but suffice it to say without them, it would be difficult to measure what is happening with ENSO.

In reply to by John Harper

As the drought in CA continues, an El Nino winter could offer huge relief. I have lived in the Sierra-Nevada region for 20+ years and have seem the ups and downs through these cycles. While climate change can be attributed to increasing drought conditions and lowered precipitation, does there seem to be any correlation with El Nino and climate change over say...the last 50 years? Do the models go back that far?

Emily, Tony and all, Excellent graphic Fig.1 and great case to be made for the skill for ENSO fcst from NMME. From the Fig.1 caption, >> The four "missed" forecasts, and the one false alarm (2012) are highlighted with asterisks. << I notice that of these 5 cases (4 missed + 1 false), 4 happened since 2000 or so in the last 15 years(compared to only one in the previous 18 years). This means we have more of these model forecast 'failures' happening lately? I am inclined to think that this has this got to to do with the 'warming background state' (aka rapid global warming) during which the models are having difficulty with in predicting ENSO! and this is a sign of things to come? As far as the latest fcst for the upcoming ElNino, let us keep our fingers crossed!! :-) Thanks Muthuvel Chelliah

Good observation.  Since 2000, ENSO has shifted to lower variability, meaning we don't see the bigger amplitude events that we saw in the 80s/90s.  The models in general have more difficulty forecasting weaker events and capturing the correct onset time.  All those cases of misses/false alarms during OND since 2000, occured with the weaker events (< +/-1degC).  The jury is out on whether this weaker ENSO variability is associated with climate change and it is very difficult to determine with the short lenght of the historical record.  There are certainly papers that project weaker variability but there are also papers that suggest strong variability. 

I wonder if the forecasts of 5 cases (4 missed + 1 false), with 4 happening since 2000 -- may be due to the slowing of the Jet Stream. As the Jet Stream is powered by the opposing forces of cold Arctic air and warm air from the south -- and is now slowing down and wobbling more because the pressure from the Arctic is waning -- this weather engine is perhaps affecting our other weather engine, our ocean currents. What do you think?

As Michelle said above, there are many indications of a climate regime shift, particularly affecting ENSO, around 1999-2000. Recently, several ideas about possible weakening/slowing/changing of the Jet Stream have been proposed; while these ideas have solid physical bases, it is difficult to identify the cause-and-effect relationships in the very short observed record. However, this is a topic of extremely active research. 

I live on the Coast of Southern Baja - coordinates near 26.238858 N 112.479075 W . The ocean temps in our region have been over 80 degrees since early July. Several weeks in August were near 90 degrees. We rarely have water temps over 80 degrees for more than a few weeks in August. We've also had an unusual amount of hurricanes and south wind days, and to me what seems like an unusual amount of high pressure cells over Mainland Mexico this summer. Even as I write in October, the water temp is well above 80 degrees, which is a record in my experience. The only other time I've seen conditions similar to this was during the El Niño around 1997. Are these conditions some local weather phenomenon unrelated to El Niño, or is it actually related to the possible upcoming El Niño?

Here is a nice article from the NOAA Northwest Fisheries Science Center that discusses the warm temperatures seen nearly along the entire eastern North Pacific Ocean:

http://www.nwfsc.noaa.gov/news/features/food_chain/index.cfm

I agree with their asessment that the cause is not straightforward at this time.  However, while we have not officially entered El Nino conditions at this point (ENSO depends on physics in/over the Pacific Ocean near the equator), it is not unusual for certain atmospheric and oceanic variables to behave "El Nino-like" prior to onset.  It is possible that warmer waters is consistent with the evolution *toward* El Nino, though the exceptionally above-average temperatures are certainly unusual and do not accompany every El Nino (or evolution toward El Nino).   

There seems to be a preference for dynamical models (over statistical) presumably because the dynamical models are ‘more advanced’ and better performing. I’m wondering, do the dynamical and statistical models have a similar amount of ‘real time’ observational data input? For example, it has been suggested that warm water volume can lead Nino 3.4 by up to ~6 months, so clearly having this data maybe an advantage.

Good question.  The statistical and dynamical models shown in the popular IRI/CPC plume of models do not all have the same "initial condtions," meaning they do not all have the exact same input data.  Dynamical models often have a huge amount of atmospheric and ocean data, while the statistical models only receive input from less than a handful of variables (most often sea surface temperature).   Also, the input data into these models can be based upon different time averages.  In particular the statistical models often rely on monthly or seasonal (3-month) averaged inputs, while the dynamical models generally rely on daily or shorter averages.  So we tend to believe the dynamical models are at an advantage simply because they see the most recent observational evolution better than many of the statistical models.  However, there are certainly cases where statistical models performed better, so clearly this is only part of the story.  But overall, in the last 10 years, we have generally seen better model performance from the dyanamical models over the statistical.  Here is one paper that explores that:

http://journals.ametsoc.org/doi/full/10.1175/BAMS-D-11-00111.1 (is free/open access)

 

I'm going to stick my neck out and ask a crazy question. In the Pacific Northwest, a lot of weather talk is revolving around "the blob," which is a nickname someone gave to the area of the northern Pacific that has seen quite warm temperatures even into this northern hemisphere's autumn season. I'm reading news stories about fishing crews near the gulf of Alaska pulling in fish such as tuna that would normally be found much farther to the south. Meanwhile ENSO discussions are saying "Yeah the models say El Nino coming but the facts don't completely show it." Meanwhile where I live the talk is all about "the blob." I don't know if N. Pacific oscillations are something you deal with, but: (1) Could what is going on in the N. Pacific be causing ENSO conditions to be somewhat contradictory in comparison to ENSO models and (2) Even if El Nino continues to play this game of hide and seek, could "the blob" create El Nino-like conditions in the U.S. by itself during the coming winter months? Meanwhile on the southern front, Australia's Bureau of Meteorology seems to be backing off (somewhat) on El Nino: http://www.abc.net.au/news/2014-10-22/nrn-el-nino-reprieve/5832818

I wonder if fish aren't coming north also due to the warming of the Arctic and the slowing of the Jet Stream? Along with our ocean currents, our other "big weather engine" is the Jet Stream -- which is powered by the opposing forces of cold Arctic air and warm air from the south. The Jet Stream is now slowing down and wobbling more because the pressure from the Arctic is waning -- I'd like to see more weather models including both of these forces -- I think it will explain some of the anomalies.

Stock of anchovy in peruvian coast has dramatically descend to the lowest in 16 years, that is the time of the great episode of El Nino in 1997-8, that affect very much Peruvian Coast with floods in the northen part and dryness in the south. This situation is ocurring even with a very light El Nino during the past months, so ther factors should be helping to cause this lack of fish. Could this episode of dramatic decline of anchovy stock be a sign of El Niño coming with more force that actually predicted. ???

Add new comment