Tag Archives: hypothesis testing

A Survey of Strong Inference in Ecology Papers: Platt’s Test and Medawar’s Fraud Model

10 Replies

In 1897 Chamberlin wrote an article in the Journal of Geology on the method of multiple working hypotheses as a way of experimentally testing scientific ideas (Chamberlin 1897 reprinted in Science). Ecology was scarcely invented at that time and this has stimulated my quest here to see if current ecology journals subscribe to Chamberlin’s approach to science. Platt (1964) formalized this approach as “strong inference” and argued that it was the best way for science to progress rapidly. If this is the case (and some do not agree that this approach is suitable for ecology) then we might use this model to check now and then on the state of ecology via published papers.

I did a very small survey in the Journal of Animal Ecology for 2015. Most ecologists I hope would classify this as one of our leading journals. I asked the simple question of whether in the Introduction to each paper there were explicit hypotheses stated and explicit alternative hypotheses, and categorized each paper as ‘yes’ or ‘no’. There is certainly a problem here in that many papers stated a hypothesis or idea they wanted to investigate but never discussed what the alternative was, or indeed if there was an alternative hypothesis. As a potential set of covariates, I tallied how many times the word ‘hypothesis’ or ‘hypotheses’ occurred in each paper, as well as the word ‘test’, ‘prediction’, and ‘model’. Most ‘model’ and ‘test’ words were used in the context of statistical models or statistical tests of significance. Singular and plural forms of these words were all counted.

This is not a publication and I did not want to spend the rest of my life looking at all the other ecology journals and many issues, so I concentrated on the Journal of Animal Ecology, volume 84, issues 1 and 2 in 2015. I obtained these results for the 51 articles in these two issues: (number of times the word appeared per article, averaged over all articles)

Explicit hypothesis and alternative hypotheses

“Hypothesis”

“Test”

“Prediction”

“Model”

Yes

22%

Mean

3.1

7.9

6.5

32.5

No

78%

Median

1

6

4

20

No. articles

51

Range

0-23

0-37

0-27

0-163

There are lots of problems with a simple analysis like this and perhaps its utility may lie in stimulating a more sophisticated analysis of a wider variety of journals. It is certainly not a random sample of the ecology literature. But maybe it gives us a few insights into ecology 2015.

I found the results quite surprising in that many papers failed Platt’s Test for strong inference. Many papers stated hypotheses but failed to state alternative hypotheses. In some cases the implied alternative hypothesis is the now-discredited null hypothesis (Johnson 2002). One possible reason for the failure to state hypotheses clearly was discussed by Medawar many years ago (Howitt and Wilson 2014; Medawar 1963). He pointed out that most scientific papers were written backwards, analysing the data, finding out what it concluded, and then writing the introduction to the paper knowing the results to follow. A significant number of papers in these issues I have looked at here seem to have been written following Medawar’s “fraud model”.

But make of such data as you will, and I appreciate that many people write papers in a less formal style than Medawar or Platt would prefer. And many have alternative hypotheses in mind but do not write them down clearly. And perhaps many referees do not think we should be restricted to using the hypothetical deductive approach to science. All of these points of view should be discussed rather than ignored. I note that some ecological journals now turn back papers that have no clear statement of a hypothesis in the introduction to the submitted paper.

The word ‘model’ is the most common word to appear in this analysis, typically in the case of a statistical model evaluated by AIC kinds of statistics. And the word ‘test’ was most commonly used in statistical tests (‘t-test’) in a paper. Indeed virtually all of these paper overflow with statistical estimates of various kinds. Few however come back in the conclusions to state exactly what progress has been made by their paper and even less make statements about what should be done next. From this small survey there is considerable room for improvement in ecological publications.

Chamberlin, T.C. 1897. The method of multiple working hypotheses. Journal of Geology 5: 837-848 (reprinted in Science 148: 754-759 in 1965). doi:10.1126/science.148.3671.754

Howitt, S.M., and Wilson, A.N. 2014. Revisiting “Is the scientific paper a fraud?”. EMBO reports 15(5): 481-484. doi:10.1002/embr.201338302

Johnson, D.H. (2002) The role of hypothesis testing in wildlife science. Journal of Wildlife Management 66(2): 272-276. doi: 10.2307/3803159

Medawar, P.B. 1963. Is the scientific paper a fraud? In “The Threat and the Glory”. Edited by P.B. Medawar. Harper Collins, New York. pp. 228-233. (Reprinted by Harper Collins in 1990. ISBN: 9780060391126.)

Platt, J.R. 1964. Strong inference. Science 146: 347-353. doi:10.1126/science.146.3642.347

Is Community Ecology Impossible?

1 Reply

John Lawton writing in 1999 about general laws in ecological studies stated:

“…. ecological patterns and the laws, rules and mechanisms that underpin them are contingent on the organisms involved, and their environment…. The contingency [due to different species’ attributes] becomes overwhelmingly complicated at intermediate scales, characteristic of community ecology, where there are a large number of case histories, and very little other than weak, fuzzy generalizations….. To discover general patterns, laws and rules in nature, ecology may need to pay less attention to the ‘middle ground’ of community ecology, relying less on reductionism and experimental manipulation, but increasing research efforts into macroecology.” (Lawton 1999, page 177)

There are two generalizations here to consider: first that macroecology is the way forward, and second that community ecology is a difficult area that can lead only to fuzzy generalizations. I will leave the macroecology issue to later, and concentrate on the idea that community ecology can never develop general laws.

The last 15 years of ecological research has partly justified Lawton’s skepticism because progress in community ecology has largely rested on local studies and local generalizations. One illustration of the difficulty of devising generalities is the controversy over the intermediate disturbance hypothesis (Schwilk, Keeley & Bond 1997; Wilkinson 1999; Fox 2013a; Fox 2013b; Kershaw & Mallik 2013; Sheil & Burslem 2013). In their recent review Kershaw and Mallik (2013) concluded that confirmation of the intermediate disturbance hypothesis for all studies was around 20%. For terrestrial ecosystems only, support was about 50%. What should we do with hypotheses that fail as often as succeed? That is perhaps a key question in community ecology. Kershaw and Mallik (2013) adopt the approach that states that the intermediate disturbance hypothesis will apply only to grassland communities of moderate productivity. The details here are not important, the strategy of limiting a supposedly general hypothesis to a small set of communities is critical. We are back to the issue of generality. It is certainly progress to set limits on particular hypotheses, but it does leave the land managers hanging. Kershaw and Mallik (2013) state that the rationale for current forest harvesting models in the boreal forest relies on the intermediate disturbance hypothesis being correct for this ecosystem. Does this matter or not? I am not sure.

Prins and Gordon (2014) evaluated a whole series of hypotheses that represented the conventional wisdom in community ecology and concluded that much of what is accepted as well supported community ecological theory has only limited support. If this is accepted (and Simberloff (2014) does not accept it) we are left in an era of chaos in which practical ecosystem management has few clear models for how to proceed unless studies are available at the local level.

Should we conclude that community ecology is impossible? Certainly not, but it may be much more difficult than our simple models suggest, and the results of studies may be more local in application than our current general overarching theories like the intermediate disturbance hypothesis.

The devil is in the details again, and the most successful community ecological studies have essentially been population ecology studies writ large for the major species in the community. Evolution rears its ugly head to confound generalization. There is not, for example, a generalized large mammal predator in every community, and the species of predators that have evolved on different continents do not all follow the same ecological rules. Ecology may be more local than we would like to believe. Perhaps Lawton (1999) was right about community ecology.

Fox, J.W. (2013a) The intermediate disturbance hypothesis is broadly defined, substantive issues are key: a reply to Sheil and Burslem. Trends in Ecology & Evolution, 28, 572-573.

Fox, J.W. (2013b) The intermediate disturbance hypothesis should be abandoned. Trends in Ecology & Evolution, 28, 86-92.

Kershaw, H.M. & Mallik, A.U. (2013) Predicting plant diversity response to disturbance: Applicability of the Intermediate Disturbance Hypothesis and Mass Ratio Hypothesis. Critical Reviews in Plant Sciences, 32, 383-395.

Lawton, J.H. (1999) Are there general laws in ecology? Oikos, 84, 177-192.

Prins, H.H.T. & Gordon, I.J. (eds.) (2014) Invasion Biology and Ecological Theory: Insights from a Continent in Transformation.  Cambridge University Press, Cambridge. 540 pp.

Schwilk, D.W., Keeley, J.E. & Bond, W.J. (1997) The intermediate disturbance hypothesis does not explain fire and diversity pattern in fynbos. Plant Ecology, 132, 77-84.

Sheil, D. & Burslem, D.F.R.P. (2013) Defining and defending Connell’s intermediate disturbance hypothesis: a response to Fox. Trends in Ecology & Evolution, 28, 571-572.

Simberloff, D. (2014) Book Review: Herbert H. T. Prins and Iain J. Gordon (eds.): Invasion biology and ecological theory. Insights from a continent in transformation. Biological Invasions, 16, 2757-2759.

Wilkinson, D.M. (1999) The disturbing history of intermediate disturbance. Oikos, 84, 145-147.

On Research Questions in Ecology

Leave a reply

I have done considerable research in arctic Canada on questions of population and community ecology, and perhaps because of this I get e mails about new proposals. This one just arrived from a NASA program called ABoVE that is just now starting up.

“Climate change in the Arctic and Boreal region is unfolding faster than anywhere else on Earth, resulting in reduced Arctic sea ice, thawing of permafrost soils, decomposition of long- frozen organic matter, widespread changes to lakes, rivers, coastlines, and alterations of ecosystem structure and function. NASA’s Terrestrial Ecology Program is in the process of planning a major field campaign, the Arctic-Boreal Vulnerability Experiment (ABoVE), which will take place in Alaska and western Canada during the next 5 to 8 years.“

“The focus of this solicitation is the initial research to begin the Arctic-Boreal Vulnerability Experiment (ABoVE) field campaign — a large-scale study of ecosystem responses to environmental change in western North America’s Arctic and boreal region and the implications for social-ecological systems. The Overarching Science Question for ABoVE is: “How vulnerable or resilient are ecosystems and society to environmental change in the Arctic and boreal region of western North America? “

I begin by noting that Peters (1991) wrote very much about the problems with these kinds of ‘how’ questions. First of all note that this is not a scientific question. There is no conceivable way to answer this question. It contains a set of meaningless words to an ecologist who is interested in testing alternative hypotheses.

One might object that this is not a research question but a broad brush agenda for more detailed proposals that will be phrased in such a way to become scientific questions. Yet it boggles the mind to ask how vulnerable ecosystems are to anything unless one is very specific. One has to define an ecosystem, difficult if it is an open system, and then define what vulnerable means operationally, and then define what types of environmental changes should be addressed – temperature, rainfall, pollution, CO2. And all of that over the broad expanse of arctic and boreal western North America, a sampling problem on a gigantic scale. Yet an administrator or politician could reasonably ask at the end of this program, ‘Well, what is the answer to this question?’ That might be ‘quite vulnerable’, and then we could go on endlessly with meaningless questions and answers that might pass for science on Fox News but not I would hope at the ESA. We can in fact measure how primary production changes over time, how much CO2 is sequestered or released from the soils of the arctic and boreal zone, but how do we translate this into resilience, another completely undefined empirical ecological concept?

We could attack the question retrospectively by asking for example: How resilient have arctic ecosystems been to the environmental changes of the past 30 years? We can document that shrubs have increased in abundance and biomass in some areas of the arctic and boreal zone (Myers-Smith et al. 2011), but what does that mean for the ecosystem or society in particular? We could note that there are almost no data on these questions because funding for northern science has been pitiful, and that raises the issue that if these changes we are asking about occur on a time scale of 30 or 50 years, how will we ever keep monitoring them over this time frame when research is doled out in 3 and 5 year blocks?

The problem of tying together ecosystems and society is that they operate on different time scales of change. Ecosystem changes in terrestrial environments of the North are slow, societal changes are fast and driven by far more obvious pressures than ecosystem changes. The interaction of slow and fast variables is hard enough to decipher scientifically without having many external inputs.

So perhaps in the end this Arctic-Boreal Vulnerability Experiment (another misuse of the word ‘experiment’) will just describe a long-term monitoring program and provide the funding for much clever ecological research, asking specific questions about exactly what parts of what ecosystems are changing and what the mechanisms of change involve. Every food web in the North is a complex network of direct and indirect interactions, and I do not know anyone who has a reliable enough understanding to predict how vulnerable any single element of the food web is to climate change. Like medieval scholars we talk much about changes of state or regime shifts, or tipping points with a model of how the world should work, but with little long term data to even begin to answer these kinds of political questions.

My hope is that this and other programs will generate some funding that will allow ecologists to do some good science. We may be fiddling while Rome is burning, but at any rate we could perhaps understand why it is burning. That also raises the issue of whether or not understanding is a stimulus for action on items that humans can control.

Myers-Smith, I.H., et al. (2011) Expansion of canopy-forming willows over the 20th century on Herschel Island, Yukon Territory, Canada. Ambio, 40, 610-623.

Peters, R.H. (1991) A Critique for Ecology. Cambridge University Press, Cambridge, England. 366 pp.

On Political Ecology

6 Replies

When I give a general lecture now, I typically have to inform the audience that I am talking about scientific ecology not political ecology. What is the difference? Scientific ecology is classical boring science, stating hypotheses, doing experiments or observations to gather the data, testing the idea, and accepting or rejecting it, outlined clearly in many papers (Platt 1963, Wolff and Krebs (2008), and illustrated in this diagram:

Scientific ecology is clearly out-of-date, and no longer ‘cool’ when compared to the new political ecology.

Political ecology is a curious mix of traditional ecology added to the advocacy issue of protecting biodiversity. Political ecology is aimed at convincing society in general and politicians in particular to protect the Earth’s biodiversity. This is a noble cause, and my complaint is only that when we advocate and use scientific ecology in pursuit of a political agenda we should be scientifically rigorous. Yet much of biodiversity science is a mix of belief and evidence, with unsuitable evidence used in support of what is a noble belief. If we believe that the end justifies the means, we would be happy with this. But I am not.

One example will illustrate my frustration with political ecology. Dirzo et al. (2014) in a recent Science paper give an illustration of the effects of removing large animals from an ecosystem. In their Figure 4, page 404, a set of 4 graphs purport to show experimentally what happens when you remove large wildlife species in Kenya, the Kenya Long-term Exclosure Experiment (Young et al. 1997). But this experiment is hopelessly flawed in being carried out on a set of plots of 4 ha, a postage stamp of habitat relative to large mammal movements and ecosystem processes. But the fact that this particular experiment was not properly designed for the questions it is now being used to address is not a problem if this is political ecology rather than scientific ecology. The overall goal of the Dirzo et al. (2014) paper is admirable, but it is achieved by quoting a whole series of questionable extrapolations given in other papers. The counter-argument in conservation biology has always been that we do not have time to do proper research and we must act now. The consequence is the elevation of expert opinion in conservation science to the realm of truth without going through the proper scientific process.

We are left with this prediction from Dirzo et al. (2014):

“Cumulatively, systematic defaunation clearly threatens to fundamentally alter basic ecological functions and is contributing to push us toward global-scale “tipping points” from which we may not be able to return ……. If unchecked, Anthropocene defaunation will become not only a characteristic of the planet’s sixth mass extinction, but also a driver of fundamental global transformations in ecosystem functioning.”

I fear that statements like this are more akin to something like a religion of conservation fundamentalism, while we proclaim to be scientists.

Dirzo, R., Young, H.S., Galetti, M., Ceballos, G., Isaac, N.J.B. & Collen, B. (2014) Defaunation in the Anthropocene. Science, 345, 401-406.

Platt, J.R. (1964) Strong inference. Science, 146, 347-353.

Wolff, J.O. & Krebs, C.J. (2008) Hypothesis testing and the scientific method revisited. Acta Zoologica Sinica, 54, 383-386.

Young, T.P., Okello, B.D., Kinyua, D. & Palmer, T.M. (1997) KLEE: A long‐term multi‐species herbivore exclusion experiment in Laikipia, Kenya. African Journal of Range & Forage Science, 14, 94-102.

Is Ecology like Economics?

2 Replies

One statement in Thomas Piketty’s book on economics struck me as a possible description of ecology’s development. On page 32 he states:

“To put it bluntly, the discipline of economics has yet to get over its childish passion for mathematics and for purely theoretical and often highly ideological speculation at the expense of historical research and collaboration with the other social sciences. Economists are all too often preoccupied with petty mathematical problems of interest only to themselves. This obsession with mathematics is an easy way of acquiring the appearance of scientificity without having to answer the far more complex questions posed by the world we live in.”

If this is at least a partially correct summary of ecology’s history, we could argue that finally in the last 20 years ecology has begun to analyze the far more complex questions posed by the ecological world. But it does so with a background of oversimplified models, whether verbal or mathematical, that we are continually trying to fit our data into. Square pegs into round holes.

Part of this problem arises from the hierarchy of science in which physics and in particular mathematics are ranked as the ideals of science to which we should all strive. It is another verbal model of the science world constructed after the fact with little attention to the details of how physics and the other hard sciences have actually progressed over the past three centuries.

Sciences also rank high in the public mind when they provide humans with more gadgets and better cars and airplanes, so that technology and science are always confused. Physics led to engineering which led to all our modern gadgets and progress. Biology has assisted medicine in continually improving human health, and natural history has enriched our lives by raising our appreciation of biodiversity. But ecology has provided a less clearly articulated vision for humans with a new list of commandments that seem to inhibit economic ‘progress’. Much of what we find in conservation biology and wildlife management simply states the obvious that humans have made a terrible mess of life on Earth – extinctions, overharvesting, pollution of lakes and the ocean, and invasive weeds among other things. In some sense ecologists are like the priests of old, warning us that God or some spiritual force will punish us if we violate some commandments or regulations. In our case it is the Earth that suffers from poorly thought out human alterations, and, in a nutshell, CO2 is the new god that will indeed guarantee that the end is near. No one really wants to hear or believe this, if we accept the polls taken in North America.

So the bottom line for ecologists should be to concentrate on the complex questions posed by the biological world, and try first to understand the problems and second to suggest some way to solve them. Much easier said than done, as we can see from the current economic mess in what might be a sister science.

Piketty, T. 2014. Capital in the Twenty-First Century. Belknap Press, Harvard University, Boston. 696 pp. ISBN 9780674430006

Back to p-Values

1 Reply

Alas ecology has slipped lower on the totem-pole of serious sciences by an article that has captured the attention of the media:

Low-Décarie, E., Chivers, C., and Granados, M. 2014. Rising complexity and falling explanatory power in ecology. Frontiers in Ecology and the Environment 12(7): 412-418. doi: 10.1890/130230.

There is much that is positive in this paper, so you should read it if only to decide whether or not to use it in a graduate seminar in statistics or in ecology. Much of what is concluded is certainly true, that there are more p-values in papers now than there were some years ago. The question then comes down to what these kinds of statistics mean and how this would justify a conclusion captured by the media that explanatory power in ecology is declining over time, and the bottom line of what to do about falling p-values. Since as far as I can see most statisticians today seem to believe that p-values are meaningless (e.g. Ioannidis 2005), one wonders what the value of showing this trend is. A second item that most statisticians agree about is that R2 values are a poor measure of anything other than the items in a particular data set. Any ecological paper that contains data to be analysed and reported summarizes many tests providing p-values and R2 values of which only some are reported. It would be interesting to do a comparison with what is recognized as a mature science (like physics or genetics) by asking whether the past revolutions in understanding and prediction power in those sciences corresponded with increasing numbers of p-values or R2 values.

To ask these questions is to ask what is the metric of scientific progress? At the present time we confuse progress with some indicators that may have little to do with scientific advancement. As journal editors we race to increase their impact factor which is interpreted as a measure of importance. For appointments to university positions we ask how many citations a person has and how many papers they have produced. We confuse scientific value with some numbers which ironically might have a very low R2 value as predictors of potential progress in a science. These numbers make sense as metrics to tell publication houses how influential their journals are, or to tell Department Heads how fantastic their job choices are, but we fool ourselves if we accept them as indicators of value to science.

If you wish to judge scientific progress you might wish to look at books that have gathered together the most important papers of the time, and examine a sequence of these from the 1950s to the present time. What is striking is that papers that seemed critically important in the 1960s or 1970s are now thought to be concerned with relatively uninteresting side issues, and conversely papers that were ignored earlier are now thought to be critical to understanding. A list of these changes might be a useful accessory to anyone asking about how to judge importance or progress in a science.

A final comment would be to look at the reasons why a relatively mature science like geology has completely failed to be able to predict earthquakes in advance and even to specify the locations of some earthquakes (Steina et al. 2012; Uyeda 2013). Progress in understanding does not of necessity dictate progress in prediction. And we ought to be wary of confusing progress with p-and R2 values.

Ioannidis, J.P.A. 2005. Why most published research findings are false. PLoS Medicine 2(8): e124.

Steina, S., Gellerb, R.J., and Liuc, M. 2012. Why earthquake hazard maps often fail and what to do about it. Tectonophysics 562-563: 1-24. doi: 10.1016/j.tecto.2012.06.047.

Uyeda, S. 2013. On earthquake prediction in Japan. Proceedings of the Japan Academy, Series B 89(9): 391-400. doi: 10.2183/pjab.89.391.

On Important Questions in Ecology

Leave a reply

There is a most interesting paper that you should read about the important questions in ecology:

Sutherland, W.J. et al. (2013) Identification of 100 fundamental ecological questions. Journal of Ecology, 101, 58-67.

This paper represents the views of 388 ecologists who culled through all of the 754 questions submitted and vetted in a two day workshop in London in April 2012. There are many thesis topics highlighted in this list and it gives a good overview of what many ecologists think is important. But there are some problems with this approach that you might wish to consider after you read this paper.

We can begin with a relatively trivial point. The title indicates that it will discuss ‘fundamental’ questions in ecology but the Summary changes this to ‘important’ questions. To be sure the authors recognize that what we now think is ‘important’ may be judged in the future to be less than important, so in a sense they recognize this problem. ‘Important’ is not an operational word in science, and consequently it is always a focus for endless argument. But let us not get involved with semantics and look at the actual 100 questions.

As I read the paper I was reminded of the discussion in Peters (1991, p. 13) who had the audacity to point out that academic ecologists thrived on unanswerable questions. In particular Peters (1991) focused on ‘why’ questions as being high on the list of unanswerable ones, and it is good to see that there are only 2 questions out of 100 that have a ‘why’ in them. Most of the questions posed are ‘how’ questions (about 65 instances) and ‘what’ questions (about 52 instances).

In framing questions in any science there is a fine line in the continuum of very broad questions that define an agenda and at the other extreme to very specific questions about one species or community. With very broad questions there will never be a clear point at which we can say that we have answered that question so we can move on. With very specific questions we can answer them experimentally and move on. So where do we cut the cake of questions? Most of these 100 questions are very broad and so they both illuminate and frustrate me because they cannot be answered without making them more specific.

Let me go over just one example. Question 11 What are the evolutionary and ecological mechanisms that govern species’ range margins? First, we might note that this question goes back at least 138 years to Alfred Wallace (1876, The Geographical Distribution of Animals), and has been repeated in many ecology textbooks ever since. There are few organisms for which it has been answered and very much speculation about it. At the moment the ecological mechanism in favour is ‘climate’. This is a question that can be answered ecologically only for particular species, and cannot be answered in real (human) time for the evolutionary mechanisms. Consequently it is an area rife for correlational ecology whose conclusions could possibly be tested in a hundred years if not longer. All of these problems should not stand in the way of doing studies on range margins, and there are many hundreds of papers that attest to this conclusion. My question is when will we know that we have answered this question, and my answer is never. We can in some cases use paleoecology to get at these issues, and then extrapolate that the future will be like the past, a most dubious assumption. My concern is that if we have not answered this question in 138 years, what is the hope that we will answer it now?

It is good to be optimistic about the future development of ecological science. Perhaps I have picked a poor example from the list of 100 questions, and my concern is that in this case at least this is not a question that I would suggest to a new PhD student. Still I am glad to have this list set out so clearly and perhaps the next step would be to write a synthesis paper on each of the 100 topics and discuss how much progress has been made on that particular issue, and what exactly we might do to answer the question more rapidly. How can we avoid in ecology what Cox (2007) called a “yawning abyss of vacuous generalities”?

Cox, D. R. (2007) Applied statistics: A review. Annals of Applied Statistics, 1, 1-16.

Peters, R. H. (1991) A Critique for Ecology, Cambridge University Press, Cambridge, England.

Sutherland, W. J., Freckleton, R. P., Godfray, H. C. J., Beissinger, S. R., Benton, T., Cameron, D. D., Carmel, Y., Coomes, D. A., Coulson, T., Emmerson, M. C., Hails, R. S., Hays, G. C., Hodgson, D. J., Hutchings, M. J., Johnson, D., Jones, J. P. G., Keeling, M. J., Kokko, H., Kunin, W. E. & Lambin, X. (2013) Identification of 100 fundamental ecological questions. Journal of Ecology, 101, 58-67.

In Defence of Hypothesis Testing in Ecology

Leave a reply

In two recent scientific meetings I have attended (which must remain nameless to protect the innocent), I have found myself wondering about the state of hypothesis testing in ecological science. I have always assumed that science consists of testing hypotheses, yet I would estimate roughly that 75% of the talks I have been able to attend showed no sign of any hypothesis. I need to qualify that. Some of these studies are completely descriptive – what species of ferns occur in national park X? Much effort now is devoted to sequencing genomes, the ultimate in descriptive biology. This kind of research work can be classified as alpha-biology, basic description which is necessary before any problems can be formulated. In my particular specialty of population cycles in mammals, much descriptive work had to be carried out to recognize the phenomenon of “cycles”. But then the question arises – at what point should we stop simple descriptions of mammal populations rising and falling? Do we need to study the dynamics of every rodent species that exists? Or in genetics, is our objective to sequence the genome of every species on earth? My point is that after we have enough basic description, we should move into hypothesis testing, or asking why some phenomenon occurs, the mechanisms behind the simple observations. The important point here is that we should not have a single hypothesis or explanation for any set of observations but rather several alternative hypotheses. As a simple example, if we find our favourite plant species is declining in abundance, we should not simply try to connect this decline with climatic warming without having a series of alternative explanations with the emphasis that our observations or experiments should be capable of distinguishing among the alternative hypotheses.

The alternative argument is that we do not know enough about ecological systems to set up a series of credible alternative hypotheses. It is quite possible to go on describing events endlessly in science in the hope that some wisdom will emerge. I do not think this is a profitable use of time or money in science. In ecology in particular I would argue that there is not a single question one can ask that cannot be answered by at least 2 or 3 different mechanistic hypotheses. Our job is to articulate these alternatives and to do whatever studies or experiments are needed to distinguish among them. Of course it is always possible that the correct answer is not among the 2 or 3 hypotheses we suggest at the start of an investigation, and this is often why one study leads to a further one. Consequently we cannot accept statements like “I have no idea why this observation has occurred”. Such a statement means you have not thought deeply enough about what you are studying. Ecological surprises certainly occur while we study any particular community or ecosystem, but we know enough now to suggest several possible mechanisms by which any ecological surprise might be generated.

So I think it incumbent on every ecologist to ask (1) what is the problem or question my research is addressing? And (2) what probable mechanisms can be invoked as the cause of this problem or the answer to this question. Vagueness may be a virtue in politics but it is not a virtue in science. And I look forward to future conferences in which every paper specifies a precise hypothesis and alternative hypotheses. Chamberlin (1897) stated the case for multiple hypotheses, Karl Popper (1963) asked very specifically what your hypothesis forbids from happening, and John Platt (1964) pulled it together in a critical paper. There was important work done before the Iphone was invented. Good reading.

Chamberlin, T. C. 1897. The method of multiple working hypotheses. Journal of Geology 5:837-848 (reprinted in Science 148: 754-759 in 1965).

Platt, J. R. 1964. Strong inference. Science 146:347-353.

Popper, K. R. 1963. Conjectures and Refutations: The Growth of Scientific Knowledge. Routledge and Kegan Paul, London.