In part two, I discussed problems with the second of the three problem issues being addressed, problems introduced in the “detrending” step, which is supposed to remove the long-term tree age/size effect (noise) and retain the climate signal. I’ll continue that here, and begin to bring the hammer down on why estimates of long-term climatic trends must frequently be wrong, at least when ring size is the analyzed variable (as it most frequently is in tree ring studies). I include some graphs this time to help bring the take-home message, well, home.
Update, 12-19-12: I replaced the four graphs with those that give a correct representation of the differences between the two types of RCS splines (stiff and flexible) tested.
To summarize briefly, the essential problem is that changes in tree size and long-term (“low frequency”) climate changes (~ century scale and longer) both affect ring response and can and do occur concurrently. Therefore, there is no obvious way to determine how much of each year’s ring response is due to tree size vs climate–a classic example of statistical confounding. The RCS method was developed to address this issue, by estimating the “expected” (~ mean) ring response for any given tree age (but size is actually the better predictor and my analyses use it, rather than age). This expected response is assumed to apply to every tree, and the deviations of each tree ring from it are therefore assumed to represent the effects of climate. For this concept to work however, there must be a good mixture of tree ages in the sample, so that (hopefully) each calendar year is sampled by rings from many different tree sizes (and conversely, each tree size occurs across a large part of the range of climate states experienced over time). I mentioned before that the method also requires that trees have as similar responses to climate as possible, and that they also experience similar non-climatic environments (e.g. soils, topography, competition etc.).
This seems at first glance like a reasonable solution to a potentially thorny problem. There is one small hitch though: it doesn’t work. A second small hitch is that there are no other known solutions to the problem. Only in certain highly optimal situations, completely unrealistic for most real tree populations, does it approximately remove the tree size effect and return an adequate long term trend estimate. Otherwise it will fail badly, usually with a definite directional bias. Big time problem. As in, leading to an uncertainty that is fatal to confidence in resulting chronologies and climate reconstructions. I’m not exaggerating; keep reading.
So why do people use it? The first answer is that a lot of times they actually don’t, they still use the older ICS method instead (described briefly in part two), with its known severe weaknesses in recovering long term trends. When they do use it, it’s presumably because it’s thought to be better than the ICS method, which it is, and because there are no other methods available (other than variants of RCS which have the same set of problems).
How well-known is this problem and has it been described before? There is some limited understanding of the method’s weaknesses in the literature, but a full understanding requires a systematic evaluation via simulation, which nobody appears ever to have done, until now. Why they haven’t done so, I have no idea, just as I have no idea why the issues raised by Loehle (2009) haven’t been recognized and embraced, or why people still estimate long-term trends using methods (ICS) known to be invalid for the purpose, without any attempt to quantify, even roughly, how much error might be introduced thereby.
One of the beauties of simulation analysis, among others, is that you can definitively test the limits of validity/accuracy of various analytical methods under various conditions. This for example is the basis for “pseudoproxy” tree ring analyses designed to determine which large-scale reconstruction methods are best under which conditions. You can test when methods do and do not fail, and how badly they fail when they do. So why hasn’t this same approach been applied to other serious issues of concern in the field? Again, I don’t know, but this in fact is exactly what I have done and was the basis of my recent submission to PNAS on this whole set of issues. I built a tree growth model, one that includes all the essential characteristics needed to evaluate the problem, such as how rings respond to climate and tree size, and how slightly different versions of the ICS and RCS methods affect the results. For this purpose, detailed physiological mechanisms of tree growth are not required. One simply defines various relationships between climate, tree size, and ring response, sets growth conditions and grows trees accordingly, and then evaluates whether commonly used detrending methods accurately return the known, long-term climate trend. [I didn’t have to build the tree growth model but there were some real advantages to doing so, a very big one being that I have something I know inside and out, another being that I learned a lot in the process.]
Relative to evaluating the limits of accuracy, the most stringent test is one in which all potentially influential variables are made to have no influence. Specifically, in this case I set conditions such that (1) there is a very good sample of trees/cores (50 trees with two cores each, whereas a typical tree ring site is often sampled by 10 to 20 trees, two cores each), (2) the start dates (first ring) of the trees are perfectly spaced throughout the 500 years of the chronology, that is, every ten years, such that the age mixture of the site is maximal, given the number of years and number of trees, (3) there is no variation in the inherent growth rate between trees, corresponding to no site or genetic differences between trees (4) there is no biological effect on ring response (ring area, or basal area increment (BAI)), meaning, the climatic variable fully determines the variation in ring response, (5) the effect of climate on ring response is perfectly linear, with no noise (i.e. y = a + bx, where y = ring area, x = climate variable (e.g. T), and no other factors influence y), and (6) a minimum sample size of six rings in any year, up to a maximum of 100.**
I then impose a steady linear increase, and then a decrease, of arbitrary magnitude, on the climate variable (e.g. T) over the full 500 years, but with a substantial amount of year to year variation, grow the trees, and then apply a number of different ICS and RCS detrendings. Each of these has slightly different parameters, following the conventions used in the literature and/or in the two most highly used software packages, ARSTAN and dplR.
So…under this perfectly optimal set of conditions for the RCS method, which together constitute a wildly unrealistic best-case scenario under which it must not fail if it is to have any general validity whatsoever, we get the following result…
Figure 1. RCS-based estimates under increasing and decreasing climatic trends. True climatic trend is the straight black line. Two RCS-based trend estimates are given by the straight blue and gray lines, as derived from the annual values (thinner blue and gray). The two RCS methods differ only in the flexibility of the spline-based fit used in constructing the regional curve; the nearly perfectly overlay of the blue and gray lines shows their similarity. The year to year variation of the climatic variable is similar to that of the RCS estimates, but is omitted to make the graph less cluttered. The RCS detrending was done using Andy Bunn’s dplR package in R.
Not good. There is a very clear under-estimation of the total trend magnitude in both directions, slightly greater for negative trends.
Suppose I impose the more realistic condition that all trees must have their first measured ring within the first 1/3 of the chronology length (first 167 years):
This returns a slightly worse result, especially when trends are positive.
Suppose I restore that condition and instead relax the restriction that all trees grow at the same inherent growth rate, such that some trees may grow up to 33% faster or slower than the average growth rate (at a maximum; the average deviation from the mean will be about half that, of 16.7%):
Figure 3. As in Figure 1, except that inherent growth rates of individual trees can vary by 33% above, or below, the mean value. [The magnitude of variation from the mean in each tree is random, so, in contrast with the other graphs, this results will change more from run to run. This is just one instance.]
This condition gives a better result under positive trends, but a little worse one under negative trends, thereby imparting a definite bias towards increasing trend.
Suppose I restore that condition and instead impose a definite (but mild) biological trend, mimicking the very common forestry observation that trees typically display unimodal growth in ring area (BAI), with an optimum at mid-life medium diameters:
Not much worse, but certainly no better, similar to the second condition in Fig 2.
Clearly, in all of these cases, the magnitude of the estimated long-term trend variation is muted, usually greatly, relative to the actual value. These are not exceptions to rules; except for Figure 3 all of these results are extremely stable, even though the inter-annual climate variation (again, not shown for graph clarity), is fairly large and stochastic (random). That random variation has little impact however, when multi-century trends are the focus, as they are here.
**Note: There is one important point relative to steps (4) and (5) described above, which is that the linear climate effect always acts on ring area, not ring width. The reason for this is that ring width has a purely geometric noise component to it. That is, the same amount of wood added each year to a growing tree bole will necessarily produce a gradually declining ring width, due strictly to geometric considerations, which biases the estimate of the total wood produced, and hence the predicted climatic variable. Such variation has nothing to do with either the climate, nor with tree physiological “decisions” (e.g. changing carbon allocation), and so it must be removed; the use of ring area (= BAI) accomplishes this automatically. However, since almost all existing studies analyze ring widths, and not ring areas, I convert the areas to widths once the tree is grown, before doing the various tests.
Still more to come.