JoNova » That’s a 0.3% consensus, not 97%

We’ve already found enough flaws, but Christopher Monckton analyzes John Cook’s 97% consensus paper and sharpens the scythe. He finds:

It should never have been done, it’s an unscientific method — “consensus”
The “consensus” was defined in three different ways. (Which hypothesis are they testing?) None of the three definitions is specific enough to be falsifiable.
The paper strangely omitted the key results. (Why make 7 classifications, if they were not going to disclose how many papers fell into each category?)
Of nearly 12,000 abstracts analyzed, there were only 64 papers in category 1 (which explicitly endorsed man-made global warming). Of those only 41 (0.3%) actually endorsed the quantitative hypothesis as defined by Cook in the introduction. A third of the 64 papers did not belong.
None of the categories endorsed “catastrophic” warming — a warming severe enough to warrant action — though this was assumed in the introduction, discussion and publicity material.
The consensus (such as there is, and it being irrelevant) appears to be declining.

The nice thing about this commentary is that Monckton provides a summary of the philosophy of science (showing Cook et al are 2,300 years out of date). Monckton has also checked Cook’s own data which was finally provided (several weeks after publication) and compares Cook to Oreskes, Anderegg, and Doran and Zimmerman and explains why they are wrong too.

Previously I’ve also pointed out the 12 reasons the paper fails, including that the number of papers is merely a proxy for funding, not evidence about the climate; most of the papers merely assume man-made warming is real, and some papers are 20 years old and the evidence has changed.

Monckton’s full commentary is here, selected excerpts below. – Jo

———————————————————————————————————

‘Quantifying the consensus on global warming

in the literature’: a comment

Christopher Monckton of Brenchley

Science and Public Policy Institute
5501 Merchants’ View Square, #209, Haymarket, VA 20169
[email protected]

Capsule

The latest paper apparently showing 97% endorsement of a consensus that more than half of recent global warming was anthropogenic really shows only 0.3% endorsement of that now-dwindling consensus.

Abstract

Cook et al. (2013) stated that abstracts of nearly all papers expressing an opinion on climate change endorsed consensus, which, however, traditionally has no scientific role; used three imprecise definitions of consensus interchangeably; analyzed abstracts only; excluded 67% expressing no opinion; omitted some key results; misstated others; and thus concluded that 97.1% endorsed the hypothesis as defined in their introduction, namely that the “scientific consensus that human activity is very likely causing most of the current GW (anthropogenic global warming, or AGW)”. The authors’ own data file categorized 64 abstracts, or only 0.5% of the sample, as endorsing the consensus hypothesis as thus defined. Inspection shows only 41 of the 64, or 0.3% of the entire sample, actually endorsed their hypothesis. Criteria for peer review of papers quantifying scientific consensus are discussed.

Introduction: no role for consensus in science

Though Cook et al. (2013) reviewed abstracts of 11,944 papers on climate change and concluded that 97.1% of those expressing an opinion supported consensus, the philosophy of science allows no role for head-count. Aristotle, in his Sophistical Refutations, (c. 350 B.C.E.), identified the argument from consensus as one of the dozen commonest logical fallacies in human discourse.

Al-Haytham, the astronomer and philosopher of science in 11^th-century Iraq who is recognized as the father of the scientific method, wrote that “the seeker after truth” – his phrase for the scientist – does not place his faith in any mere consensus, however venerable. Instead, he checks. “The road to the truth,” said al-Haytham, “is long and hard, but that is the road we must follow.”

In 1860 T.H. Huxley said: “The improver of natural knowledge absolutely refuses to acknowledge authority, as such. For him, skepticism is the highest of duties: blind faith the one unpardonable sin.”

Albert Einstein, when told that 100 Nazi scientists had published a book rejecting his theory of special relativity, responded that a single paper would have sufficed to refute his hypothesis. His own single paper of 1905 on the electrodynamics of moving objects had demonstrated why Newton’s laws, till then universally accepted as true, incompletely described the motion of celestial objects.

Popper (1934) formalized the scientific method as an iterative algorithm starting with a general problem (GP₀), to address which a scientist proposed a falsifiable hypothesis or tentative theory (TT₀). Thereupon others would either demonstrate during the error-elimination phase (EE₀) that the hypothesis was false, in which event it was rejected, or, more rarely, demonstrate that it was true.

By far the commonest outcome, however, especially in the physical sciences, is that error elimination will fall short of demonstrating the hypothesis but will fail to disprove it, in which event it gains some credibility. The statement of the general problem may then be modified accordingly (GP₁), and a new tentative theory (TT₁) may later be advanced to address the modified problem; and so on. Pedetemptim, and if necessary ad infinitum, science iteratively converges upon the truth (Fig. 1). Consensus adds no value to this process.

In the scientific method, then, there is no place for mere consensus. A hypothesis that is demonstrated – such as Pythagoras’ theorem – needs no consensus, for it is objectively true. A hypothesis that is disproven needs no consensus, for it is objectively false. A hypothesis that is neither demonstrated nor disproven gains credibility, and not because a dozen or even 12,000 papers endorse it but because – and to the extent that – it has not been demonstrated to be false. Science is not a belief system. A priori, then, head-counts are inappropriate tests of scientific results.

Problems in defining the climate consensus

… the definition of the hypothesis should be expressed quantitatively. An imprecisely defined hypothesis, especially if it is not quantitative, may be insufficiently rigorous to be testable. If it be untestable, then, stricto sensu, it is not of interest to science. It is a mere curiosity. Yet Cook et al. do not confine themselves to a single definition of the hypothesis to which their consensus is said to adhere. Three definitions of climate consensus coexist in the paper –

Definition (1): “the consensus position that humans are causing global warming” (abstract);

Definition (2): the “scientific consensus that human activity is very likely causing most of the current GW (anthropogenic global warming, or AGW)” (introduction);

Definition (3): that our enhancement of the greenhouse effect will be dangerous enough to be “catastrophic”; (implicit in the introduction, in discussion of the need to raise awareness of scientific consensus to justify a “climate policy”, and explicit in Table 2 of Cook et al., citing a paper opposing “the catastrophic view of the greenhouse effect”).

Definitions (1, 3) fall short of the criteria for definition of a Popper-falsifiable hypothesis, and definition (2) could have been clearer. Not only do Cook et al. adopt the definitions interchangeably, but each definition is imprecise and insufficiently quantified to allow rigorous Popper-falsification. None of the definitions specifies the period to which it applies, or how much global warming was observed over that period, or whether the warming is continuing, or, if so, at what rate, or whether that rate is considered dangerous, or what rate if any is considered dangerous.

Additionally, definitions (1) and (3) do not specify what fraction of warming was considered anthropogenic, and definition (2) assigns no quantitative value to the term “very likely”. Such imprecisions render the hypotheses unfalsifiable and hence beyond the realm of legitimate scientific inquiry.

(see the full commentary for more discussion of this)

[Cook cites Doran and Zimmerman and Anderegg, Monckton explains why they are wrong too]

Doran and Zimmerman (2009)

The two authors sent a 2-minute online survey to 10,257 earth scientists at universities and government research agencies. Only 5% of the 3,146 respondents identified themselves as climate scientists; 90% believed mean global temperatures had generally risen compared with pre-1800s levels; and 82% believed human activity was a significant contributing factor in changing mean global temperatures. Only 79 of the respondents listed climate science as their area of expertise and had also published more than half of their recent peer-reviewed papers on climate change. Of these, 98% believed human activity was a significant contributing factor in changing mean global temperatures. However, the sample size was insufficient to deliver a statistically reliable result, and the respondents were not asked whether they believed the anthropogenic influence on temperature might become sufficiently damaging to require a “climate policy”.

Anderegg et al. (2010)

From publication and citation data, the authors selected 908 of 1372 climate researchers, defined as people who had published at least 20 climate papers and had either signed petitions opposing or supporting the IPCC’s positions or had co-authored IPCC reports. Of these, 97-98% believed that “anthropogenic greenhouse gases have been responsible for ‘most’ of the ‘unequivocal’ warming of the Earth’s average global temperature over the second half of the 20th century”. The definition of the consensus in Anderegg et al. is less imprecise than definition (2) in Cook et al. Yet, like Cook et al., Anderegg et al. did not seek to determine how many researchers considered global warming to be actually or potentially damaging enough to require a climate policy. Nevertheless, the two surveys are often cited as demonstrating a near-unanimous scientific consensus in favor of a climate policy, when in fact, like Cook et al., neither survey had asked any question either about whether and to what extent the anthropogenic component in recent warming might be dangerous or about whether a “climate policy” should be adopted in attempted mitigation of future warming.

Incomplete statement of the survey results

None of the seven “levels of endorsement” by which Cook et al. categorize their selected abstracts provides evidence that any of the 11,944 abstracts encompasses the catastrophist definition (3):

“Explicitly states that humans are the primary cause of global warming”
“Explicit endorsement without quantification”
“Implicit endorsement”
“No opinion, or uncertain”
“Implicit rejection”
“Explicit rejection without quantification”
“Explicit rejection with quantification”

The first endorsement level, “Explicitly states that humans are the primary cause of global warming”, reflects definition (2) and is akin to the other definitions in Table 1. The second and third levels, “Explicit endorsement without quantification” and “Implicit endorsement”, reflect definition (1) in that, like it, they are not quantitative. Yet the first three levels of endorsement are treated as one in the results:

“To simplify the analysis, ratings were consolidated into three groups: endorsements (including implicit and explicit: categories 1-3) …”.

Results of an inspection of the Cook et al. data file

It is not possible to discern either from the paper or from the supplementary information what fraction of all abstracts endorse definition (2). A file of raw data was supplied, though it was only posted online some weeks after publication. This comma-delimited text file was dowloaded and the abstracts allocated by Cook et al. to each level of endorsement were counted. Results are given in Table 2:

E.Level	1	2	3	4	5	6	7	Total
	Explicit support +quant	Explicit support –quant	Implicit support –quant	No opinion /uncert	Implicit rejectn. –quant	Explicit rejectn. –quant	Explicit rejectn. +quant

Papers	64	922	2910	7970	54	15	9	11,944
% all	0.5%	7.7%	24.4%	66.7%	0.5%	0.1%	0.1%	100%
% opin	1.6%	23.0%	72.5%		1.3%	0.4%	0.2%	99%

Table 2. Abstracts in the seven levels of endorsement specified in Cook et al. (2013). Only 64 abstracts, according to the authors’ data file, explicitly endorsed definition (2), the quantitative hypothesis. NB: “+quant” indicates “with quantification”; “–quant” indicates “without quantification”; “% all” indicates the percentage of all 11,944 abstracts that fell in each level of endorsement; “% opin” indicates the percentages of all 4014 abstracts, excluding the 7930 that expressed no opinion but including the 40 that expressed uncertainty (1% of all papers). These 40 are not shown separately in the datafile or in the table. Therefore, the percentages of papers expressing an opinion sum to 99%, not 100%.

Evidence that the climate consensus is declining

(see the full commentary)

Has there been any ‘current’ warming?

(see the full commentary)

Discussion

The defects identified in the surveys of climate consensus by Cook et al. and by the authors of some of the papers they cite follow a pattern to whose existence peer-reviewers should be alert. First, any argument from consensus on a question such as the extent to which anthropogenic global warming may prove dangerous is defective a priori and ought really to be rejected without further review.

Secondly, no survey of opinion for or against a consensus hypothesis ought to be regarded as scientific where it is not made clear which hypothesis is under test, or where the hypothesis under test is not clearly and precisely formulated. A fortiori, a survey paper that exhibits multiple definitions of the consensus hypothesis and fails to state clearly the identity and definition of the hypothesis on the basis of which the survey was actually conducted should surely be rejected.

Thirdly, the consensus hypothesis under test ought to be expressed in quantitative terms. Mere qualitative definitions of any scientific hypothesis run the risk of appearing more political than scientific in their formulation, and papers based on such definitions may also prove more political than scientific in their effect.

Fourthly, if several “levels of endorsement” are specified, then the number of abstracts, papers, or scientists considered to have supported each level of endorsement ought to be explicitly stated in the paper under review. Cook et al. specified three levels of endorsement that supported the notion of anthropogenic warming (however defined); yet, on the stated ground of simplifying the analysis, the number of papers allocated to each of the three levels of endorsement – a key result on any view – was not stated. The analysis would indeed have been simpler if one endorsement level supporting one definition of climate consensus had been adopted.

Fifthly, all data files and programs should be archived at the time of submission to the journal and included at the time of publication as part of the supplementary material. Reviewers should ask for the datafiles and programs if they are not available.

Conclusion

The non-disclosure in Cook et al. of the number of abstracts supporting each specified level of endorsement had the effect of not making available the fact that only 41 papers – 0.3% of all 11,944 abstracts or 1.0% of the 4014 expressing an opinion, and not 97.1% – had been found to endorse the quantitative hypothesis, stated in the introduction to Cook et al. and akin to similar definitions in the literature, that “human activity is very likely causing most of the current GW (anthropogenic global warming, or AGW)”.

——————————————–

Some previous posts on Cook’s work:

9 out of 10 based on 82 ratings