I have refrained from reading CA comments that followed by friday-evening post (221), to stay on point. Comments to the answers to the comments to the questions will come in due time (i have a busy week coming up, and have unchained quite a firestorm that i cannot tackle tonight…).
Before we start, however, I have a few disclaimers to make :
- I am conducting this review alone. This is unrequested work, i am only doing it for the defense of my own scientific conviction and to share it with other readers, some of which have little background on the subject. Needless to say, the form would be vastly different if this were a review carried out for a scientific journal. However, my angle on it is : does Loehle’s reconstruction, meet the basic criteria that i would apply to a review, were i asked to be a referee ? Anyone is free to disagree – and i am looking forward to insightful comments from statisticians.
- Since the issue has started to appear on this CA thread, let me be perfectly clear. I am not here to defend previous work : the sole focus of this review is Loehle’s article. I am a newcormer to this game, having entered the wonderfully charged world of paleoclimate reconstructions less than a year ago. As such, i do not pretend to have inoxydizable expertise on all recontrusctions to date, much less on all of ClimateAudit’s blogposts. Hence, please do not ask me to justify so-and-so’s methods. Loehle comes as a challenger in this game, and i believe it is the job of the underdog to do at least as well as the Establishment he criticizes. I will accept no excuse of the type :”this had not be done in the usual reconstruction” if they are demonstratedly doable in the case of this simple method (cf error bars). McIntyre is well-founded when he says that much of my criticisms would apply to some previous work, none of which have i ever pretended to defend.
- I am not emphatically not here on behalf of RealClimate, the IPCC, GaTech and certainly not here to defend Dr Mann. I am entitled to the independence of my views and refer to none but my own education, judgment, and bibliography. Dr Mann, on the other hand, is entitled to the defense of his own work, has done so with relentless (and arguably, successful) effort over the past 9 years, and is an equally independent scientist who can fend for himself. My own opinion on the MBH99 reconstruction ? I wouldn’t bet the family farm that “1998 was the warmest year in at least a millennium”, but i agree with the NAS panel when they find that , “with a high level of confidence […] Global mean surface temperature was higher during the last few decades of the 20th century than during any comparable period during the preceding four centuries”. Ditto that is it is plausible that “The Northern Hemisphere was warmer during the last few decades of the 20th century than during any comparable period over the preceding millennium”.
Mann’s publication record speaks for itself, he and collaborators like Scott Rutherford have undertaken very extensive re-workings of the MBH98/99 algorithm, albeit not necessarily gracefully – in CA’s eyes. I wholeheartedly agree that climate science is lagging behind other fields in terms of providing open access to data and methods, and Steve McIntyre’s efforts are much needed in improving these standards. Nonetheless, i find CA-dwellers a little antiquated in their constant re-hashing of MBH98 criticisms. They would be well inspired to direct them at his most recent work if they so wish, rather than beating horses dead in 1998 or 1999. While it is true that MBH98 is not the panacea of climate reconstructions, it would be reductionist to corner all of “Team” into one article and ignore almost a decade of subsequent work. Since CA readers get understandably upset when i lump them together with climate obscurantists, they should lend a sympathetic hear to the statement that not every climate scientist is guilty of pensée unique.
Review of Loehle (2007).
We start by an extensive introduction that argues for the inability of tree-rings to record any temperature signal on the period of interest, and is decently informed. While it would be excessive to throw out just ANY tree-ring series (though they had issues with strip-bark trees, i don’t believe the NAS panel recommended the incineration of the entire dendroclimatological literature), the point is well taken : in light of the many factors that influence tree-ring growth over long periods, it is certainly a valid “What if ?” experiment to produce a reconstruction free of such data. Since the author acknowledges that it is not the first (Moberg  and Viau et al  have already done similar things) , we are eagerly awaiting what original statistical techniques will be presented here if the author is to arrive at a substantially different conclusion.
Eyebrows start rising when the only references given to support the existence of a “Medieval Warm Period” (MWP) are from the author himself (Loehle 2006), and Soon and Baliunas (2003), which has been thoroughly debunked by RealClimate colleagues (see Myth #2). For those not so versed in blog wars, The MWP is a favorite of global-warming denialists, because if you coax the data into showing a warm enough early millennium, you can negate all of the IPCC’s work : it’s super duper cool.
Unfortunately, there is ample evidence that the so-called Medieval Warm Period was far from global : see for instance the NAS report (North et al, 2006, chapter 11 and references therein). There is, however, great interest in understanding why it was warm in Europe or dry in the US South West and central America, and seemingly cool in the eastern tropical Pacific (cf Graham, 2006), so let’s read on.
METHODS : The section is startlingly short. 18 proxy timeseries are selected on the grounds that : a) they are available for 2000 years b) they are not tree-rings. A decent case is made that one should not worry too much about dating uncertainties. Fair enough. What next ? Data are smoothed, standardized, and… averaged. Period.
I had to read the article 3 times (re-downlading it just in case i had gotten a faulty version the first time) to convince myself that i wasn’t dreaming. All subsequent figures show the arithmetic mean of 17 or 18 series, as if they all recorded local temperature with the same accuracy, and were representative of the same geographical area.
Proxy temperature relationship
Since the article’s title mentions a “global temperature reconstruction”, one would expect two fundamental aspects to be discussed . 1) these are reliable temperature proxies ; 2) they are representative of a global average. In that light, i expected the merits of said proxies as paleothermometers to be at least brushed upon. There are only 18 of them (unlike most paleoproxy studies, which can compile >200 ), so it wouldn’t break the author’s back to at least READ the papers he is citing, dig up the error estimate, and put this all into a pretty table. It’s not so clear he does *read*, however, otherwise he wouldn’t have retrieved a “Borehole 18O temperature” from Dahl-Jensen et al. , just a borehole temperature inversion.
This is a very different situation from usual multiproxy studies which use sophisticated methods to ensure that a proxy’s weight in the final result reflects its ability to record some variance in the temperature field (whether local or not). While there is merit in exploring a bare bones approach (arithmetic mean), it then becomes indispensable to demonstrate that each proxy is : a) a temperature proxy (not a salinity one…). b) a good one at that.
At first glance, a) is dubious for the Holmgren and Keigwin timeseries (that’s at first glance… i need to dig more), and b) can usually be assessed from the authors’ error analysis. If it is not, then the author should conduct his own, or drop the use of such proxy.
Once again, such care would not be required when a climate-field-reconstruction or “composite-plus-scale” approach is employed, as the proxy’s ability to record temperature is implicit in the calibration therein. Since the author effectively treats the proxies are perfect thermometers (which is conceptually acceptable as long as it is explicitly justified), the lack of this discussion is unforgivable, and in my book, constitutes grounds for rejection any day of the week.
Error propagation analysis
This part is usually immensely complicated in the case of non-linear estimators, but it should be a piece of cake here. Since the proxies are so concisely described (i agree that a map would have been nice – apparently it didn’t take S. McIntyre very long to make one), it comes as no surprise that the final result is utterly devoid of error bars.
Instead, the author chooses a jacknife analysis, which in my view only speaks to the relative importance of each proxy in the final results. Imagine all proxies had a standard error of 2.5 degrees with the actual temperature : this analysis would say nothing about the accuracy of the mean.
Let’s be more constructive than in my original CA post : there are several possible approaches. For instance, one could assemble a vector of proxy errors, and assume the temperature proxies are iid and Gaussian, which is quite reasonable. It would not take a PhD in mathematics to prove that the arithmetic mean is also a Gaussian random variable with a standard deviation that is the L2-norm of the sigma vector (divided by the number of series). This is rather simple-minded, but since the approach is deliberately simple, it would be consistent with this framework. There maybe more elaborate ways, as discussed by Hu McCullough here.
In any case, it is unacceptable to make the quantitative claim that the data ”would indeed seem to show the MWP to be warmer than the late 20th century” , “by about 0.3 degrees. This number is worthless without an error bar, or at least an insightful discussion of uncertainty.
Rejection again, sunday or friday.
Again, impressive concision ! Where are the CE, RE, and most importantly R-squared statistics that are so dear to ClimateAuditers when used by the Team ? How are we supposed to guess whether the reconstruction has any skill ? (skill, by the way, is a ubiquitous term in the numerical weather and climate forecasting literature. It is intuitive enough, but if statisticians need a more statistical term, i can try and dig one up over the next few days). In his original response, Loehle argues that these statistics are irrelevant because he does not have a model. This is false, as pointed out by bender, and myself. Despite claims of using a “simple mean”, the author’s approach is effectively a multivariate regression. Indeed, each of the proxies was calibrated against temperature using such regression methods, usually linear, but sometimes non-linear (Mg/Ca ratios depend exponentially on seawater temperature, for instance). So by doing even a simple average, the author is effectively using a statistical model, and is therefore not exempt from the verification exercise.
(incidentally, i regret that bender seems to believe i have been plagiarizing his ideas. I was responding to earlier comments when he posted his and did not see them until after i had submitted mine – an unfortunate consequence of the high traffic at CA… This convergence does, however, suggest we have more in common than what superficial disputes would suggest, so perhaps one day we’ll have real dialogue)
Minor comments :
– I could not find a reference for “simple mean” (p1053). It may be a novel contribution to the mathematical literature – which had heretofore used “arithmetic mean”. I only point this out since CA statisticians seem always eager to correct climatologists on their improper use of statistical terminology. While i am genuinely eager to be educated in statistics, i wish the same standards applied to everyone.
– I believe it is the first time i read ‘I believe’ in a scientific paper (it’s different in politics), and it is there 3 times in the discussion. Hmmmmm…
– R. Villabala does not appear to have any publications on the Medieval Warm Period. Ricardo Villalba, other other hand, does. (the error is present in the text and reference list). This reinforces the feeling that the basic homework really hasn’t been done.
In summary, this article is a poorly-described compilation of proxy data, with a conciseness in methodology that borders on farce. In the present form , it is unacceptable in any scientific journal that i am aware of. Though the approach is conceptually useful, it is not novel : the author himself acknowledges that Moberg (2005) and Viau (2006) have left out tree-rings from some parts of their reconstruction before. So what is new here ? It can only be the methodology. We have shown that its elliptic nature is naive at best, misleading at worse. Even with all the good will in the world, it is hard to grant the author the benefit of the doubt, given how loosely he handles his references and prose : this simply is not credible work. The author’s argument that his “strategy in writing this was to make it as short as possible to avoid complications during review” is distinctly unconvincing. Why not, then, bypass the whole ‘methods’ section, and simply give us a curve without any explanation ? Brevity is the soul of wit, yes – but when crucial information is missing, this “science” has an odd scent of disinformation.
I am glad to see that a healthy debate has now engaged on CA and that Steve McIntyre has undertaken a serious audit of this study (e.g. here and here). Already, some of my question regarding spatial sampling have been admirably addressed, and though it should have been the author’s job to do this before submission, the only thing that matters here is that the question be answered somehow.
I hope this debate will bring Craig Loehle to produce a scientifically-acceptable reconstruction – one that meets our basic field’s criteria for publication. He would otherwise condemn himself to sub-standard journals like Energy & Environment, ensuring that they are invisible to climate scientists – for good reason.