Severe analytical problems in dendroclimatology, part seven

This is a long post with many graphical results, and is the most important one in this series to date.  The gist of the types, and magnitudes, of the problems introduced by the detrending step are presented here.  Given the general lack of interest this may also have to be the final installment, even though I probably have as much or more to say as I already have.  It’s just not worth the time when so few are interested in what I consider to be an extremely important and yet overlooked issue,  and still others simply want to aggravate.

In part four of this series I discussed the kinds of trend estimation errors that RCS would produce when standard practices are followed. In this post I’m going to elaborate on that, and also on the topic raised in parts five and six regarding the importance of the ring response variable used, and then discuss a couple of other important points. These evaluations require using the full model, not the abbreviated one linked to in part five, because the former is capable of testing far more complex situations.  It also performs calibration and prediction of the climate state itself, from the tree ring chronology. The difference between the two doesn’t matter there, but it can in more complex situations and so it’s generally best to present the actual reconstruction estimates as things get more complicated.

In part four, I standardized the chronologies to the climate variable so I could easily graph them together on the same scale. I used the ratio of the means of the two variables to do that, but there are other ways to do it, and these could potentially make a difference in the results. One way would be by using the difference in means; another (and very common) way would be to compute Z scores for each, which automatically puts them on the same scale.

When ring width is the response variable (and hence the predictor of the climatic state), with no biological trend in the rings (but with a geometric effect) here’s what results, using the same “best case” conditions described in part four, and standardization by the ratio of the means, as used there:

If I instead standardize using the difference of the means, I get:

If standardization is instead by Z scores (subtraction of mean followed by division by the standard deviation) we get this:

Thus, the method used to relate the chronologies to the climate variable has almost zero effect on the results. In part four I showed results when three other variations from ideal conditions were instituted, but without really drastic deviations (such as for example, nonlinear relationships between climate and ring response).

I’ll address the important issue raised in parts five and six: what happens if we remove the purely geometric effect induced by the use of ring widths as the response variable, by instead using ring areas? Does that solve the problem? Here’s a repeat of the first three graphs above, using ring widths areas as the response variable:

Once again the three standardizations give nearly identical results. These results are clearly better than those obtained using ring widths, especially when climatic trends are negative, so this is definite progress: eliminating the purely geometric effect does indeed provide a clearly better estimate of the long term trend, at least under these conditions. It’s still not perfect however, especially for negative trends, where a mis-estimate of almost one unit over the 460 year time-span still exists, and remember, this is the best case condition being evaluated here, so results are unlikely to get better as we deviate from them.

I’ll  do that by first changing the age structure, by forcing all trees to begin their lives before the 2/3 point of the full timespan spanned by the sampled trees, that is, before the 334th year, thereby shifting the sample some towards the more even aged end of the spectrum.  Using standardization by ratio of the means, this gives:
A definite decline in accuracy is seen, relative to the previous graph.  Now I instead institute growth differences between trees, up to +/- 33% of the mean (but with optimal age structure restored).  This results in:

If I combine these last two conditions together, it gives:

Now imposing a mildly unimodal biological growth trend, such that trees’ growth rates increase rapidly  gradually from year zero, plateau at a high point in the middle decades, and then gradually decline afterward (but with the age structure and inter-tree growth rate variation restored to their “best case” states), we get:

Now combining deviations from the ideal state in all three of these variables gives:

This result could reasonably be described as complete failure–there is clearly almost no ability to predict the actual climatic trend of the past.

A couple of important points here.  First, none of these three non-ideal conditions just added are particularly extreme: the unimodality of the biological effect is very broad and shallow-sloped, and forcing trees to originate in the first 2/3 of the analysis period is a condition far exceeded in many ITRDB tree ring data sets that are much more even aged than this.  The inter-tree growth rate variation is difficult to place in context, but certainly trees growing a maximum of 33% above, or below, the mean growth rate is not unreasonable (and since 33% represents the maximum value, as defined by a uniform distribution, the average departure of the trees from the mean growth rate will be only half that, or +/- 16.7%).  As before, there is no difference if one of the two other ways of standardizing the chronologies to the climate is instead used (results not shown).  The point is that none of these conditions can be considered extreme or outside the realm of reasonable expectation for actual data.  Indeed, they are on the mild side, conservative.

The second point is that this last graph has no geometric component to it–it uses ring areas as the dependent variable.  But that of course is not standard practice, which is to use ring widths.  Here’s what standard practice would result in:

Now we see that the negative trends are actually estimated so poorly that even the direction is wrong–the method actually predicts a positive trend even though the real trend is very strongly negative.

There are also at least two other very important things to note here, when ring widths are used.  The first is that there is a universal tendency for (1) trend magnitudes to be under-estimated, and (2) the negative trends to be mis-estimated much more severely than are positive trends.  The implications of this are fairly serious.  Clearly, if these tested conditions held over some large area, and there had been a negative climatic trend therein, the RCS method would give a completely misleading trend estimate.  But more generally, the method fails to capture the full magnitude of long term variation present regardless of the trend direction.  This is just the latest in a series of such findings, using highly varied analytical approaches, that come to this same conclusion: tree ring methods badly under-estimate the true amount of variation in past climates at medium to long time scales.  Although the RCS method is certainly an improvement upon the ICS method in this regard under good analytical conditions, it is not much better, if any, when conditions are worse.

The second point is that RCS introduces error to the high frequency and medium frequency variation estimates as well.  The following two graphs show the course of the actual climatic state variable, and then the reconstructed climate estimate, both taken from the previous graph but shown separately for clarity:

Notice the high frequency (year to year) variation over time in the first graph, the true climatic state.  It’s constant over time, for both trend directions.  This is as expected, because this variation is defined in the simulations to be so, i.e. drawn from a distribution with a constant standard deviation.  Now look at the variance in the second graph by comparison.  Clearly, the variance itself trends (technically referred to as being “heteroscedastic”, or non-constant), and in a direction that is opposite to that the climate trend itself–when the climate state value is relatively high, the variance is relatively low, and vice-versa.  Thus, existing methods also distort the estimates of year to year (and/or decade to decade) variation, along with the trend.  When ring areas are instead used, this problem is generally reduced, but not eliminated.  As an example, the following result obtains when there is no inter-tree growth variation or biological growth trend, but where all trees originate in the first one-third of the time period:

Thus, in essence, what the RCS detrending method is doing, especially when ring widths (common practice) are used, is shifting some of the variation that should actually be ascribed to long time scales, into shorter time scales, thereby introducing error into both time scales. That is, estimated trends are diminished relative to actual, while estimated high frequency variations are exaggerated, relative to actual.

What?  By this finding we have taken an already very serious analytical flaw and made it even worse, thereby topping out on the failure meter.  If you can’t use ring widths to estimate long term trends, nor high frequency variations, then what can you use them for?  Fortunately, the high frequency variations can be estimated in other ways that do not distort them badly, and so be salvaged, and so this is what should be done whenever year to year variations are the focus.  Year to year variations are certainly interesting and important, but they tell us nothing about longer term trends of course.

One last, but important, point on the errors in high frequency variations.  All of the results presented to date have been defined by a strictly linear, deterministic relationship between climate and ring response.  The high frequency variations in all cases, even when the estimated magnitude of that variation is distorted by the RCS procedure, as just shown, still always track reality at that scale, directionally: the true values, and the corresponding estimates, go up and down in synchrony.  This is important because it shows that the rings are in fact responding to the climate, and that the RCS algorithm is capable of recognizing this.  Therefore, long term trend mis-estimates do not require a departure from linearity in the relationship between climate and ring response, as per the issues raised by Loehle (2009; described in parts one and two of this series), and can therefore be due to other (analytical)  causes.  This fact has important potential uses when analyzing and diagnosing trends estimated from real data.  Hopefully, more on that point in the future.