Note: This is the first of a series of 14 posts on this topic that I wrote back in 2012/13. Science-wise, it’s the most important thing I’ve written here. I’m sticking this post to the front because I don’t know if I’ll ever get time to submit it for publication (again), and people need to know about these issues, which are completely ignored and/or misunderstood in the scientific literature. The issues involve how to–and not to–separate signal from noise estimating a climatic signal from tree rings. It’s technical. The story of the original submission to PNAS is part nine, here.
Dendroclimatology comprises the science of estimating past temperature and/or precipitation dynamics from tree ring measurements. The ring response variable is most often ring size or wood density, the former being much easier to measure and therefore the much more common metric. The field has about a 100 year history and has generated hundreds of papers during that time. Unfortunately, it also has several very serious analytical problems in the estimation of long term (roughly, century scale and up) trends. Some of these issues have been described in the literature, but others not fully. Here, I’ll describe what I see as the three most critical issues. It is not an overstatement to say that all long term climatic estimates are suspect due to these (and other) issues. It’s that serious.
As climatic change has become a more societally important issue over the last three decades, the number of “reconstruction” studies estimating long term change at large spatial scales has grown. This includes studies published in the highest level scientific journals, e.g. PNAS, Nature, and Science. This is a strong indication that either (1) these problems are not being understood by authors and/or reviewers, or (2) they are being glossed over to various degrees. Neither situation is acceptable in science, but the latter would certainly be the more troubling. I believe that both may in fact be occurring, based on the general lack of response to existing studies pointing out various problems, and from an utterly faulty review of a paper I submitted to PNAS this year, which demonstrated both a lack of understanding of the technicalities involved, and a lack of effort to even try to understand them.
It’s important to state here that these problems do not affect the confidence that scientists have in the reality of the greenhouse gas-induced climate change that is now occurring. Such confidence is based largely on radiation physics of the atmosphere, on estimates of climate sensitivity, and on quantitative estimates of expected thermodynamic feedbacks. These estimates are determined from a number of different empirical measurements and model-based insights, but none of them are derived from tree ring reconstructions of climate over the last millennium (that I am aware of). However, the issues raised here do certainly do affect our confidence in how unique the climatic conditions of the last few decades are, relative to those of the last thousand years or so. This in turn has implications for the expected, near-future, terrestrial ecosystem response/resilience to the current climatic changes, for example, at least as a rough guide or first approximation.
The three issues are:
- (1) ring width, being the result of a biological growth process, almost certainly responds in a unimodal way to temperature (i.e. gradually rising, then rather abruptly falling), and therefore predicting temperature from ring width cannot, by mathematical definition, give a unique solution,
- (2) the methods used to account for, and remove (“detrend”) that part of the long term trend in ring widths due to changes in tree age/size are ad-hoc curve fitting procedures that cannot reliably discriminate such trends from actual climatic trends, and
- (3) the methods and metrics used in many studies to calibrate and validate the relationship between temperature and ring response during the instrumental record period, are also frequently faulty.
Each of these issues by itself would be a serious problem, but collectively they render unreliable all long-term estimates of climate change from ring widths. Worse, there are also several other issues that I don’t have time to get into, at least for now. These involve (1) the several issues dealing with which seasonal/monthly periods are the most important to the ring response, and how that determination should be made, and (2) subsequent steps in the climate reconstruction process, by which single sites are combined in various ways to give larger scale estimates. The well known, but still unresolved divergence phenomenon between multi-decadal variation in climate and ring response during the instrumental record period is potentially affected by all three of the issues discussed here, but especially the first one.
The first issue–the effect of unimodal ring responses to climate–has been most clearly described by Loehle (2009). [See also Kingsolver’s (2009) paper from his American Naturalist Presidential Address for an interesting and more general discussion of the biological basis and importance of the underlying issue]. Loehle details the mathematical implications of inverting a statistical relationship between two variables, such that the independent variable is predicted from the dependent, to create “transfer” functions, as must occur. If the dependent variable (e.g. ring width) is a unimodal function of the independent variable (e.g. temperature), then there is no unique solution when one inverts the relationship in order to predict the latter from the former. The practice will result in a bimodal distribution of possible climatic states at any point in time, with the difference in the two modal estimates being a function of how far from the optimum ring size the actual (measured) ring size is.
The most important aspect to this issue is that the direction of the relationship changes at the optimum: to the left of the optimum, ring size increases with increasing temperature, while to the right of it, this relationship is reversed. This issue is neither difficult nor novel, and you can hardly have a more serious type of error or bias in the estimation of an unknown, from a known, variable. For this reason, these two papers (and perhaps others) should be absolutely required reading for all dendroclimatologists. Although most high profile, large scale reconstructions pre-date both papers, there is nevertheless no real evidence that the seriousness of this issue has been recognized and accepted in the field, generally since Loehle described it three years ago. There is also really no excuse for the problem not being recognized long, long before 2009, because the general idea that many biological functions of temperature are more or less unimodal in nature is not particularly new or recent. [Sometimes these relationships are asymptotic, which is only somewhat less of a problem–asymptotic relationships still lead to large temperature ranges across which the response is very insensitive and prediction therefore highly problematic]. These realities are worrisome indeed, and it is difficult to impossible to find a reasonable explanation for why this is the current state of affairs.
Of the three issues, it’s quite possible that this one is also the most critical, because unlike the others it (1) applies at any and all time scales of interest, from the inter-annual to the multi-centennial, and (2) it potentially applies to wood density measures as well as ring widths (though more work seems needed there). The issue also raises, indirectly, the more general and serious problem with the field, i.e. that it is based almost entirely on observational evidence and thus by nature cannot detect certain types of analytical problems. Such detection requires controlled experimentation, be it on actual trees (logistically difficult and expensive to be sure), or in model experiments that explore the limits of what is possible and/or likely. It’s largely an observational science without a strong theoretical foundation, which is always a recipe for serious potential trouble and confusion.
More detail on these issues will be presented in subsequent posts.
Ecologically Oriented 
Jim –
The major implication I’m seeing here is that tree rings are pretty much useless for determining temperature (due to the impossibility of pulling out a reliable signal), and hence would make any reconstruction requiring them equally useless. Is that reasonably close?
Also, could you unpack this sentence: “it is based almost entirely on observational evidence and thus by nature cannot detect certain types of analytical problems.” Is it the subjective nature of observational evidence (in the absence of a strong theoretical foundation) that causes this problem?
Lastly (because it’s getting late here and I have to be up early), would this sort of problem apply to other biological proxies if the proxy is subject to complex inputs? As an example, shell thickness in marine organisms?
Thanks for your time.
Yes kch, that’s correct, and it is a very major implication indeed. Existing analytical methods cannot reliably pull out a long term trend if there is one. And it does not apply only to temperature, but to any environmental signal, such as precipitation or fertilization (CO2 or Nitrogen). Anything that is trending will be mis-estimated. But temperature is the one that gets most of the attention, for obvious reasons. However, if the methods are changed, these problems can be greatly reduced. But this will require +/- starting over with tree ring collections, because the changes involve the field sampling procedures. The existing set of public collections at the NOAA ITRDB are largely useless, because of their age/size structures.
Another thing to remember here is that I’m only dealing with the “detrending” step of the full set of analytical procedures with my work. Even if you get that right, correctly removing the non-climatic trends, you still have the issues of how to separate any potentially multiple environmental signals from each other. And you still have the potential problem detailed by Loehle as well. And you have still other calibration and verification issues also (potentially). It all needs to be resolved to have confidence in tree rings as proxies.
The fact that the evidence itself is subjectively chosen is indeed one potential problem, but is not what I’m referring to there. My statement is more epistemic and will apply generally to much observational evidence: you cannot reliably separate inputs (signals) from each other without either (1) controlled experiments or (2) simulations (i.e. controlled experiments on model systems). The first of these is very problematic, logistically, and we are therefore left with the second, which is the approach I took. In doing so, you just have to insure that you’ve covered all reasonable real-world scenarios.
Yes! It’s a general epistemic problem and will thus apply to many problems having multiple inputs. Corals are the other major biological proxy with annual resolution and they have their own set of issues. There are other biological proxies that don’t rely on growth rates (pollen abundance for example) but their resolution is low and uncertain so they are not useful for estimating short term changes. We need things that have resolutions on scales of a couple of decades at the worst, to separate natural variations from trend drivers (forcings).
Thanks. One follow-up question on this part, if you don’t mind.
You say that controlled experiments are ‘problematic, logistically’. Why? Is it mostly due to the length of time needed to grow trees to the ages needed? Could you not use data from tree farming operations? There should be a good deal of climactic data available on the areas that has taken place – or would that be a problem of wrong species and/or not old enough? What kind of experiment would you need?
I was being generous in using that phrase actually. Performing such experiments would be impossible for all practical purposes in most situations, because you’d have to impose the necessary treatments (say temperature variations) over such an absurdly long time (centuries), to inherently large objects (trees), which is never ever going to happen in the real world, even if you could figure out how to do it, technically, which itself would be a very major challenge. And so unbelievably expensive that it would never be funded, ever.
Tree farm (“plantation”) data would be useful only in the sense that they control for competition via a constant tree spacing and control of competitive vegetation, but they always involve commercially important (i.e. timber) species, which are very different from the subalpine and boreal species used in dendroclimatology. And moreover, there is still no control over other environmental variations such as temperature, precipitation, air chemistry, etc.
Therefore there’s really only one choice here: simulation experiments. Only with simulated data can you control everything that needs to be controlled, and it is not difficult to make sure that you’ve covered all reasonably possible effects of tree age/size on the results, because age/size effects can only take a limited number of possible forms and magnitudes. It is by far the best approach, no contest at all. The fact that it’s been almost completely unused is telling.
Right, that’s clear enough. Thanks.
So, the usual follow up questions:
1) If you experiment using simulated data, how do you verify the results are correct? would you not need to compare at some point with observational data? Where does that come from?
2) Are the response profiles* so species specific that it isn’t possible to develop a general profile based on plantation species that would have some use on sub-alpine/boreal species?
3) I really think you should expand – as a blog post, perhaps – on the final sentence in the light of your comments on Loehle2009 and your post “Are scientists “regular” people?”
Is the problem motivated science? Motivated funding? Careerism? Cronyism? A combination of everything? And all of it exacerbated by confirmation bias, wilful ignorance and personality? How can it be avoided?
* I know my terminology sucks. Hopefully you get the gist of what I’m asking, and I’ll try for the proper terms when alerted to them (eg. When I was helping plant the things (at six cents each for jack pine or 4.5 cents for black spruce) we called them ‘tree farms’ – but plantations does makes a good deal more sense in this context).
Whoops – skip all discussion of using plantations for anything. I see that in part two you deal with that in the comments. No reason to go into it again…
You have very good questions kch and in fact your earlier one partially inspired my newest post, put up just now.
Hang on.
1) Very big and important general question in science, hard to answer briefly. The short answer in this case is that no matter what type of age/size effect I impose, including no effect at all, mild effects or strong effects, the existing detrending algorithms are incapable of returning the correct climatic trend. This means that something is wrong, mathematically, with the procedures. See other posts in the series for what that something is.
2) Correct. You would be trying to generalize from timber species to sub-alpine species, and there are numerous potential problems there, mostly having to do with photosynthate production and allocation in those very different classes of trees. Not to say that it couldn’t be explored however, especially if you had no better options.
3) Briefly: any time you lack a theoretical foundation for your observations, you will sooner or later be limited in what you can say about cause and effect from them, usually sooner. Another huge topic in science. Many non-scientists think observations are superior to models: not the case, and not the right way to think about it. They are both necessary, always.
And no problem with the terminology btw.