Was there a significant bias between the ensemble of climate models and the long-term temperature trends?

What about spatial patterns?

  • 6
    $\begingroup$ Climate is the statistics of the atmosphere over a period of ~30 years. So it's sort of too early to tell. $\endgroup$ – gerrit Jun 25 '14 at 14:15
  • 4
    $\begingroup$ In addition to @gerrit's observations, the models of the 1990s were not intended to be able to predict climate on a decadal scale, as this requires predicting sources of internal variability, such as ENSO. As I understand it, the current generation of models is just beginning to reach the point where decadal projections are worth evaluating (CMIP5?). $\endgroup$ – Dikran Marsupial Jun 26 '14 at 12:01
  • 1
    $\begingroup$ Even the 2014 models, which are much better than the 1990s models, aren't very good at predicting 10-20 years in the future because the climate change in 10-20 years is only marginally bigger than the natural variation. Statistical noise makes near term predictions difficult. That said, my understanding of the 90s models is that they were too aggressive, not deliberately, but it was a new science. They didn't know how good the oceans would be at absorbing heat trapped by Greenhouse gas. I don't believe there was bias, though in some individuals maybe. it was new and complicated science. $\endgroup$ – userLTK Sep 24 '15 at 16:50
  • 1
    $\begingroup$ If interested, this article is related and goes into some detail. skepticalscience.com/ipcc-overestimate-global-warming.htm $\endgroup$ – userLTK Oct 6 '15 at 7:26

You might take a look at the Technical Summary of the most recent IPCC Assessment Report (5).

Thematic Focus Element #3 is "Comparing Projections from Previous IPCC Assessments with Observations"

It says:

Global Mean Temperature Anomaly

Relative to the 1961–1990 mean, the GMST anomaly has been positive and larger than 0.25°C since 2001. Observations are generally well within the range of the extent of the earlier IPCC projections (TFE.3, Figure1, middle left) This is also true for the Coupled Model Intercomparison Project Phase 5 (CMIP5) results (TFE.3, Figure 1; middle right) in the sense that the observed record lies within the range of the model projections, but on the lower end of the plume. Mt Pinatubo erupted in 1991 (see FAQ 11.2 for discussion of how volcanoes impact the climate system), leading to a brief period of relative global mean cooling during the early 1990s. The IPCC First, Second and Third Assessment Reports (FAR, SAR and TAR) did not include the effects of volcanic eruptions and thus failed to include the cooling associated with the Pinatubo eruption. AR4 and AR5, however, did include the effects from volcanoes and did simulate successfully the associated cooling. During 1995–2000 the global mean temperature anomaly was quite variable—a significant fraction of this variability was due to the large El Niño in 1997–1998 and the strong back-to-back La Niñas in 1999–2001. The projections associated with these assessment reports do not attempt to capture the actual evolution of these El Niño and La Niña events, but include them as a source of uncertainty due to natural variability as encompassed by, for example, the range given by the individual CMIP3 and CMIP5 simulations and projection (TFE.3, Figure 1). The grey wedge in TFE.3, Figure 1 (middle right) corresponds to the indicative likely range for annual temperatures, which is determined from the Representative Concentration Pathways (RCPs) assessed value for the 20-year mean 2016–2035 (see discussion of Figure TS.14 and Section 11.3.6 for details). From 1998 to 2012 the observational estimates have largely been on the low end of the range given by the scenarios alone in previous assessment reports and CMIP3 and CMIP5 projections. {2.4; Box 9.2}

I've attached an image as well from that summary:

previous IPCC temperature performance

Estimated changes in the observed globally and annually averaged surface temperature anomaly relative to 1961–1990 (in °C) since 1950 compared with the range of projections from the previous IPCC assessments. Values are harmonized to start from the same value at 1990. Observed global annual temperature anomaly, relative to 1961–1990, from three data sets is shown as squares and smoothed time series as solid lines from the Hadley Centre/Climatic Research Unit gridded surface temperature data set 4 (HadCRUT4; bright green), Merged Land–Ocean Surface Temperature Analysis (MLOST; warm mustard) and Goddard Institute for Space Studies Surface Temperature Analysis (GISTEMP; dark blue) data sets. The coloured shading shows the projected range of global annual mean near surface temperature change from 1990 to 2035 for models used in FAR (Figure 6.11), SAR (Figure 19 in the TS of IPCC 1996), TAR (full range of TAR, Figure 9.13(b)). TAR results are based on the simple climate model analyses presented in this assessment and not on the individual full three-dimensional climate model simulations. For the AR4 results are presented as single model runs of the CMIP3 ensemble for the historical period from 1950 to 2000 (light grey lines) and for three SRES scenarios (A2, A1B and B1) from 2001 to 2035. For the three SRES scenarios the bars show the CMIP3 ensemble mean and the likely range given by –40 % to +60% of the mean as assessed in Chapter 10 of AR4.

  • $\begingroup$ Was that a yes or a no? $\endgroup$ – matt_black Oct 9 '15 at 21:46
  • 2
    $\begingroup$ The long term temperature trends were within the range of the ensembles, as can be seen in the graph. $\endgroup$ – farrenthorpe Oct 9 '15 at 22:21

Climate projections shouldn't be seen as deterministic projection of climate. Since the earth system is non-linear (i.e. chaotic), a lot of path are possible for the next decades. In other words, the statistics of climate (i.e. trend, inter-annual variability, spatial patterns) of the next 3 decades could be closer to the 2060-2090 climate projections than the climate projection of 2020-2050.

This kind of comparison lies in the decadal prediction framework. This is an emerging science in itself. One that is far from being ready for prime time. The simulations done in the 1990s were not aimed at such timescale (decadal) prediction.

I doesn't answer directly the question, but I'm merely pointing out that such comparisons might be misleading.

  • 2
    $\begingroup$ While many ways of doing the comparisons will be misleading, surely we should expect some testing of the model's predictive power? Otherwise how do we know the models are not concealing some serious flaw? $\endgroup$ – matt_black Sep 21 '15 at 10:16
  • $\begingroup$ @matt_black, I think Balinus is pointing out that a projection that is accurate on a centennial scale may not be very accurate on a decadal scale, as sources of internal climate variability (e.g. ENSO) still have a large effect at a decadal scale relative to the expected warming, but this averages out to a much greater extent on a centennial scale. The point is, "how accurate can we reasonably expect the models to be on a decadal scale?". A hint is given by the spread of the individual model runs forming the ensemble. $\endgroup$ – Dikran Marsupial Sep 22 '15 at 11:00

Some comparisons do suggest that historic models do a poor job of simulating observed temperatures

This picture from a Nature article is the clearest analysis that avoids the various obfuscations possible when looking at noisy time series data:

nature chart

The choice made by these authors is to compare just the average extent of warming over a period rather than the noisy time series. This appears to avoid some of the choices that make the time series comparison so obfuscating and controversial (in fact their results don't disagree much with the ones shown in the previous answer form the IPCC, they just choose a presentation method that emphasises the comparison between models and observations). Their result seems to show that most models really do overestimate the actual amount of warming over recent years. In their words:


In somewhat more detail the authors talk about the statistics like this (my highlights):

The evidence, therefore, indicates that the current generation of climate models (when run as a group, with the CMIP5 prescribed forcings) do not reproduce the observed global warming over the past 20 years, or the slowdown in global warming over the past fifteen years. This interpretation is supported by statistical tests of the null hypothesis that the observed and model mean trends are equal, assuming that either: (1) the models are exchangeable with each other (that is, the ‘truth plus error’ view); or (2) the models are exchangeable with each other and with the observations (see Supplementary Information). Differences between observed and simulated 20-year trends have p values (Supplementary Information) that drop to close to zero by 1993–2012 under assumption (1) and to 0.04 under assumption (2) (Fig. 2c). Here we note that the smaller the p value is, the stronger the evidence against the null hypothesis. On this basis, the rarity of the 1993–2012 trend difference under assumption (1) is obvious. Under assumption (2), this implies that such an inconsistency is only expected to occur by chance once in 500 years, if 20-year periods are considered statistically independent. Similar results apply to trends for 1998–2012. In conclusion, we reject the null hypothesis that the observed and model mean trends are equal.

The conclusion is that most models (or, perhaps, most results from the ensemble of models) predict more warming that has actually happened. Whether this is a significant problems for models will become more obvious in the next few years as observational data accumulates. The authors include some caveats:

Other factors that contribute to the discrepancy could include a missing decrease in stratospheric water vapour (whose processes are not well represented in current climate models), errors in aerosol forcing in the CMIP5 models, a bias in the prescribed solar irradiance trend, the possibility that the transient climate sensitivity of the CMIP5 models could be on average too high or a possible unusual episode of internal climate variability not considered above. Ultimately the causes of this inconsistency will only be understood after careful comparison of simulated internal climate variability and climate model forcings with observations from the past two decades, and by waiting to see how global temperature responds over the coming decades.

Other authors have addressed the question using a slightly different and less direct approach. They tend to compare the implied climate sensitivity (crudely: how much warming we expect from a doubling of CO2 levels in the short or long term) from observations and models. Several recent papers have shown lower estimates (especially for transient climate response) than the values in climate models (see Lewis & Curry; Aldrin et. al.; Otto et. al.; Skeie et. al..

Summary The question was whether models do a good job of simulating the actual future climate. These results suggest they haven't. But there is plenty of scientific uncertainty about whether that failure is significant.

  • 2
    $\begingroup$ A key problem with this paper is that it uses the model variability to estimate the expected variability of the observations (effectively), which means you can't claim that the models are running too warm, only that they are either running too warm, or they under-simulate natural variability or a bit of both. My opinion it is a bit of both. The paper isn't that controversial, the problem is the overstating of the significance on climate blogs (e.g. the first line of your answer, which is written in bold). $\endgroup$ – Dikran Marsupial Sep 22 '15 at 11:06
  • 2
    $\begingroup$ " In conclusion, we reject the null hypothesis that the observed and model mean trends are equal." which we wouldn't expect anyway. The ensemble mean is an estimate of the forced response of the climate system, whereas the observed trend is the result of the forced response AND a realization of the unforced response (i.e. weather noise), and so the two will only be the same if the weather noise is very small. Proper test is to see if the observed trend falls within the spread of the modeled trends, which it apparently does. $\endgroup$ – Dikran Marsupial Sep 22 '15 at 11:12
  • 2
    $\begingroup$ "it was reasonable that the noise in the system would lead to actual weather being a long way from forecast weather over an extended period. " climate projections are not based on forecasting weather, but simulating weather. There is a big difference between the two. $\endgroup$ – Dikran Marsupial Sep 24 '15 at 7:36
  • 2
    $\begingroup$ Note however, the spread of the ensemble is likely to under-represent the true uncertainty, see this rather timely blog post by variable-variability.blogspot.co.uk/2015/09/… (n.b. the author is a climatologist). $\endgroup$ – Dikran Marsupial Sep 24 '15 at 8:19
  • 2
    $\begingroup$ there are very big differences between those two pairs, it would be a good idea to make sure you understand the difference if you are interested in comparing models and observations. Having said which, it is notable that your answer doesn't include the many caveats that were included in the Fyfe paper and overstate its conclusions. Note the paper ends "Ultimately the causes of this inconsistency will only be understood after careful comparison of simulated internal climate variability and climate model forcings with observations from the past two decades, ... $\endgroup$ – Dikran Marsupial Sep 25 '15 at 12:16

Your Answer

