The projected timing of climate departure from recent variability

…is the title of an open access paper just published in Nature by a group of researchers at the University of Hawaii that attempts to estimate when future climates will move beyond recent ranges, and what effects that will have on biodiversity of several major taxa. I have some thoughts on this paper, especially on its overall approach. I rarely care about specific results presented in any paper until I’ve been able to look closely at its conceptual approach, assumptions (often unstated), and specific methods; what is the point in knowing what some paper claims to have found, without knowing pretty exactly how they went about it (and why)? In that regard, this paper could readily be used as a springboard to discuss a number of highly important issues in scientific practice generally, and in earth system science and/or ecology specifically.

The need to examine papers this way (i.e., as if one were a reviewer) raises a very important issue by itself, that is, how can anybody possibly read even a small subset of existing papers in a field? The answer is, that unless you’re a genius with a very wide scope of knowledge and training, and are a very focused and fast reader, or greatly restrict your scope, you can’t. Personally, I know very few people who qualify as such, zero in fact. You can skim things, you can read abstracts and discussion/conclusions, look at the graphs and tables, so as to get the general gist of the arguments and conclusions, but rarely can you delve into deep methodological details, except at the expense of the breadth of papers read. Detailed examination takes time, serious time, and a sound understanding of the fundamentals of the subject matter itself, and also of statistics and numerical analysis more generally, and the ability to judge the validity of the application(s) thereof to the subject matter.

This stuff is complicated; it’s not simple, it’s not easy, it’s not quick, it’s not automatic. It’s not molecular genetics, where important theoretical bases (e.g. inheritance rules, gene expression processes) are tight and well understood, and laboratory and analytical methods are frequently standardized and well defined. It’s environmental science; the variability is large, the contingencies many, the analytical methods all over the map and often poorly evaluated before being extensively applied. It wouldn’t be quite so bad if the focus was always maintained on certain environmental science sub-disciplines, but it’s not. Instead, inherently uncertain predictions in say, climate science, are often used as the basis to predict changes in climate-dependent variables, which themselves are highly complex, potentially poorly defined/quantified, and affected by many possible drivers.

Scientific publication is now like a fire hose stream of information coming at you, as is quickly realized by perusing publication databases like the Web of Science for a couple of hours–there are far more publications coming out daily than anybody can keep up with, for a topic as broad as the earth and environmental sciences (and same for other large topics). You want information of a quantity and quality you can handle, and instead it comes at you with the mass and velocity of water from a fire hydrant, whereupon you realize that you need a well considered strategy as to how you’re going to make any progress. In short, there’s an enormous gap between the volume of research coming out daily, and the number of people evaluating it in depth, and this is without question a recipe for problems of various kinds, seen on an ongoing basis.

This situation leaves you, the curious, with a few options. One is to take published science at face value, trusting the publication process to get things right a high percentage of the time, going largely with the flow, i.e., the consensus. A second is to make decisions about which scientists and/or research groups you trust and restrict yourself to reading their papers. A third is to make your own judgements, but focusing on only those papers either appearing to be the most important, however judged (the filtering process itself still a significant time requirement itself), or on those you can actually understand well. A fourth is to throw up your hands and walk away from the whole thing as something you really can’t deal with effectively.

OK, on to the paper. To get right to it, I think there are several serious problems with the work, largely conceptual, involving the general approach(es) taken. The paper’s stated main goals are to: (1) provide evidence of when future climates will move outside the range of historical climates, and (2) estimate where, and how large, the effects of those changes will be on biodiversity, generally, a grand objective indeed. The general approach used goes thus: (1) use GCMs to estimate the ranges of variability from 1860 to 2005 for several important climate variables centered around region-specific energy and water budgets, (2) use those same models to estimate when the mean state of those variables will exceed the high ends of those ranges, given two hypothetical GHG atmospheric growth rates, and (3) make some inferences about what these dates, along with nation-level economic conditions, imply for future “biodiversity” dynamics/stability.

The paper starts with the now seemingly obligatory litany of what future climate change might or could do to society and ecosystems. This kind of stuff gets repeated over and over in papers, usually with qualifying “coulds”, “mays”, or “cans” in the phrasing. I pay no attention to such text except to (sometimes) look at the citation list for any surprises, because it’s almost never useful information. We don’t need to be told that species can either adapt, migrate or “go extinct” in response to environmental change–this is not a chapter in a high school biology text. Nor do we need references to possible mental illness effects, or potential changes in “geopolitical stability”; if the stated goal is to discuss climate change rates and resulting effects on biodiversity, then please, stick right to it. The earth could also get blasted by an intense solar wind tomorrow, Haiti or San Francisco could get destroyed again by an earthquake, but I don’t see those things mentioned. Given how little space one has to make complex arguments in the science glamor journals (Nature, Science, PNAS, etc.), due to word limits, you’d think such irrelevancies would get axed in short order. Alas, to the contrary, you see them over and over again. They seem to have no purpose except to create or reinforce a particular attitude of mind, the effect of which does nothing but make me suspicious (although not necessarily prejudiced). I’m not interested in sweeping generalizations, especially when they don’t relate specifically to the stated goal(s).

The last sentence of the introduction ends “our understanding of climate change still lacks a precise indication of the time at which the climate of a given location will shift wholly outside the range of historical precedents”. The paper then proposes to provide such precise dates, and relate them to biodiversity dynamics. OK; you set the bar at the height you propose you can clear…now lets’s see you clear it.

Note: edited for clarity.


13 thoughts on “The projected timing of climate departure from recent variability

  1. While I am quite interested in seeing where you go in discussing the Mora paper (having seen it both slagged and praised by the usual suspects), I’m actually far more interested in the first five paragraphs here.

    I know that you are very much talking about the approaches taken by scientists within the field, but this could easily serve as a model for a discussion of the difficulties faced by the interested layman in dealing with climate science. (This is usually dealt with by blathering on about the need for ‘better communication’, but that usually seems to be directed at politer ways of saying “shut up and accept what we’re saying.” Kind of a non-starter for me.) So if you don’t mind, I’d like to talk about that part of your post.

    If the scientists have difficulty dealing with the mass of information being thrown at them, what are the ignorati like myself to do? I cannot learn enough, fast enough to get at the roots of all issues – you intimate that, really, no one can. Given the vast complexity of the field, I’d have to agree. Still, for me, that route’s out.

    You give four other options for dealing with the issues. The last – walking away – I can’t do. The whole thing is too important on too many levels.

    The first – trusting publications on face value – I am reluctant to do. I’ve seen too much of human nature, and know all too well the allure of confirmation bias. Is it present everywhere, in all publications? Of course not, but it will always be present to some degree, especially when dealing with an established consensus. Throw in political/social/economic implications and trust in a published consensus disappears.

    The third option – making my own judgment on those I deem important – is a little more possible. Just not practical. Laymen like myself simply do not have the background to see what is important, let alone to fully grasp the depths of any good paper. It is usually necessary to have someone explain any paper that might seem important to me, which makes any exercise of my own judgment a moot point…

    Which leaves us with option #2 – deciding who to trust. For you, the scientist, this means trusting the authors and the science of a paper. For me, the layman, this comes to mean finding trust in those who critique and write about papers. The big question is how to determine where that trust should be placed. I know the process I’ve experienced, and the factors that have influenced that process. Given the strident rancour across the blogs, obviously not everyone has followed the same journey to the same place as me.

    So the question: would it not be better for the ‘consensus’ position and scientists to work on regaining lost trust than work on communication strategies?

    • Well, great comment and questions as usual kch.

      You’re exactly right that all my pontifications there apply just as much to the general public as it does to us scientists, even moreso in fact. One of my main points in writing that stuff is to dispel any notions that we scientists just cruise easily through all this mass of material and pretty much agree with each other on the validity of it. Some would like to have the public believe that, but I think they’re a small minority, and I’m most definitely not one of them.

      No rational, thinking, independent person will (or should) ever accept “shut up and trust us” as a response to questions. It’s simply not acceptable, and it will never work, or lead to any real progress. Unfortunately, since we live in a society of specialization, some are specialists in discovering things (i.e. academics), and so there’s likely going to be some difficulties of various kinds in transmitting such knowledge to those who are not specialists, and that’s the challenge we face, and it’s often not easy.

      Hang on, I’m going to add some more to this later.

      Continuing…Trust is always vital in any communication of course, and I don’t think we’ll ever be completely free from having to trust experts to some degree, probably a large degree. To answer your question at the end, I think it depends on what someone means if when they talk about better communication. If by that they mean reducing jargon and talking in simpler terms, then great, if that’s indeed really the problem. On the other hand, if there’s a problem in the validity of the science itself, then improved communication is not going to help. The only way to remedy that is for those trained to understand the material speak up and say so when something’s not right. But that raises exactly the problem I discussed–too much material being published relative to the in-depth examination thereof; fire hose effect. Then, on top of addressing such issues, we need IMO, a group of scientists whose sole job is to translate complex, primary literature into forms that different segments of the population can absorb. We already have such a thing for Congress–The Congressional Record Service–and there are science writers who do this in various places, like at Science and Nature, but it’s kind of hit or miss, with no coordinated effort. And the result is what we see–some scientists taking to blogs or other popular media to try to communicate things, with mixed results.

      A last thought is that we need to do a better job of communicating analytical approaches and modes of thought in solving and addressing problems. What we see in many pop media pieces is a very heavy focus on findings, at the exclusion of methods. And of course, what’s the point of any finding, without knowing the pathway to them? If those findings are wrong, than it’s < 0 (i.e. they make things worse than if you had reported no findings at all).

    • “…I don’t think we’ll ever be completely free from having to trust experts to some degree, probably a large degree.”

      I’ll certainly agree – my problem is always finding the right experts. Preferably ones I feel I can trust without without being in complete agreement.

      “…we need IMO, a group of scientists whose sole job is to translate complex, primary literature into forms that different segments of the population can absorb.”

      While I’d love to see this, I can’t really see it being successful on any organized scale, if only because any such group will sooner or later fall into the interpretive mode and subject matter favored by its funders. The current hit and miss system of interpretation – flawed though it is – might actually be the best system.

      ” And of course, what’s the point of any finding, without knowing the pathway to them?”

      None, really. This is one reason our society has tried hard to develop transparent processes for both jurisprudence and governance – the transparency engenders the trust in those institutions an open society needs. I see no reason why trust in science should follow a different path. I will never find myself being able to trust any organization or individual that decides to pull a veil over the process they have used to reach their conclusion.

    • Can you describe what sorts of things make you trust a particular scientist and what don’t? I’m curious as to how that occurs. Is it primarily an issue of openness with data and methods and other details of scientific practice or is there more to it than that, and if so, what?

    • Ouch – tough question. I’ve never really had to think about the process before, so if this gets a little obscure and disjointed please forgive me and ask for clarification.

      I am not really capable of judging the science on it’s own merits. My knowledge is too limited, and I know it. As a result, I try to start from the position of acceptance. At the same time, however, my cynical nature forces me to believe that the vagaries of human nature will toss up bad science and agenda-driven scientists. So then – if the subject is of enough importance to me – I need validation/verification, and for that I need to find others who do understand the science, but in whom I can trust.

      The single most important aspect of my trust in a scientist (or, really, anyone for that matter) would be my perception of their intellectual honesty: with other scientists, with the public, with themselves and most importantly with the science itself. It is an ongoing process, with trust levels constantly shifting as new information comes available.

      The sorts of things that colour my views of a person’s intellectual honesty:

      – How do they respond to a critique of their work? (Ad hominem attack; gatekeeping; appealing to their own authority: all bad, in my books.)

      – Do they acknowledge the limitations of their work? (Insisting on implications that are not justified by the work itself: bad.)

      – Do they acknowledge the limitations of their own knowledge? (Insistent on claiming authority outside of their own sub field: bad.)

      – Are they willing to go out on a limb, to test accepted beliefs? (Always defending a consensus: maybe not bad in and of itself, but probably intellectually lazy.)

      – Are they willing to admit error? (We all make mistakes – refusing to see that: bad.)

      – Have they been caught in lies or misrepresentations? (Bad, bad, bad.)

      Note that this failing on this list does not necessarily destroy my trust in their science, but only in their ability to usefully/honestly comment on science in general, including and especially their own work. Also note that I feel that people can change. In the years that I have followed the climate wars, I’ve mentally moved a number of people from ‘trust’ to ‘distrust’ and vice versa. Partially my views of them, but also to some extent their own journeys, I think.

      Other factors relating to trust? Personality counts, at least in first impressions. I do work hard to see beyond that, though. The same goes for shared world-views – first impressions do count, but fade into the background as more relevant information becomes available. Not much else comes to mind (at least for now).

      So I’d say that my trust is not primarily based on the openness of methods and data, but that those are inextricably bound up in shading my views of their intellectual honesty – if a scientist fights to keep anyone from replicating his work, I cannot bring myself to fully trust either the work or the scientist. And really, why would anyone refuse data or method except to prevent replication or verification?

    • Thanks. I also wouldn’t mind seeing others views on this.

      More importantly, though, I’d wish that this kind of statement would be taken note of by the ‘improved communication’ people. I believe that retaining/regaining trust is a vital first step in the process of communicating science, and one often missed – it can be very difficult to see oneself as untrustworthy.

    • …it can be very difficult to see oneself as untrustworthy.

      Indeed. More generally, seeing oneself objectively is one of the great challenges of human existence, IMO.

  2. On this topic, I just saw a notice from Gavin Schmidt on this, which seems to me to hold tremendous promise for educating both ourselves, and the general public simultaneously. I really really hope this goes somewhere–it’s a great idea IMO:

    “The public can benefit tremendously from being able to see how scientists think and reason scientifically. We ask them to trust our collective scientific wisdom without allowing them to see how we evaluate conflicting or flawed evidence and develop judgments. Presently, the only extensive example of this available to the public is the set of emails from Climategate.”

    • I would hope that the CCNF becomes a viable group. It will need to develop public trust, of course, to be truly useful. I will be interested to see who signs on and what form it eventually takes – in particular, who is allowed to speak.

      My worry is that it will be simply another variation on telling the public what they should be thinking.

      My hope is that it will be a good picture of the process used by scientists to reach findings. This we laymen could use…

  3. I haven’t checked in in awhile, I have some catching up to do! I wanted to respond here because my views on assessing scientific veracity barely overlap with what is described above. The key to the vast majority of papers is how correlation is tied to causality. From this engineer’s perspective, there is only one may to handle the gray space in between – exhaustively. I find I can understand papers in most fields well enough to follow the argument, and what I am looking for is a logic structure that covers all the variables. [I’m going to stop now because I can no longer see what I am typing after 8 lines. Do others have this problem?]

Have at it

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s