Category Archives: Evaluating Research Quality

Should Ecology Abandon Popper?

2 Replies

The first question I must ask is whether you the reader have ever heard of Karl Popper. If the answer is no, then you could profit from reading Popper (1963) before you read this. An abbreviated version of the Popperian approach to science is presented in a short paper by Platt (1963) The simplest version of Popper and Platt is that we should have a hypothesis with specific predictions and one or more alternative hypotheses with other predictions, and science advances by finding out which hypotheses could be rejected with empirical evidence. The focus of this blog is on a recent paper by Raerinne (2024) claiming that Popperian ecology is a delusion. This is a claim well worth discussing particularly since most of the sciences progress using a Popperian approach to testing hypotheses.

To begin perhaps we should recognize two kinds of papers that appear in ecological journals. A very large set of ecological papers appear to be largely or entirely descriptive natural history typically of past or present events with no hypotheses in mind. Many of these papers end with a conclusion that could be designated as a hypothesis but with little discussion of alternatives. These papers can be very valuable in giving us the state of populations, communities, or ecosystems with recommendations for changes that should be made to alleviate developing problems. A good example are papers describing forest and grassland fires of recent years which can end with some management recommendations, and perhaps with alternative recommendations. These recommendations usually arise from experience and judgements, and they may or not be valid. The Popperian approach would be to set up hypotheses and test them empirically, but if we are people of action, we press onward with a preferred management action. The non-Popperian approach would be very efficient if we were correct in our diagnosis, and in many cases this approach works well. The basis of the issue here is what is evidence in ecology and how should it be sharpened into recommendations for conservation and management.  

The Popperian approach to ecological science is to recognize problems that require a solution to increase our knowledge base, and to suggest a series of alternative set of mechanisms that could solve or alleviate the problem. Ecological papers supporting this approach can often be recognized by searching for the word “hypothesis” in the text. A simple example of this Popperian approach could be finding the causes of the continuing decline of a commercial fishery. The decline might be due to predation on the target fish or invertebrate, a disease, added pollution to the water body, climate change increasing the water temperature and thus metabolic functions, introduced species of competitors for food or space. One or more of these causal factors could be involved and the job of the ecologist is to find out which one or several are diagnostic. Given the complexity of ecological problems, it is typically not possible to test these alternative hypotheses in one grand experiment, and the typical approach will involve adaptive management or evidence-based conservation (Gillson et al. 2019, Serrouya et al., Westgate et al. 2013). Complexity however should not be used as an excuse to do poor science.

What is the alternative if we abandon Popper? We could adopt the inductive approach and gather data that we put together with our judgement to declare that we have a correct answer to our questions, “seat of the pants” ecology. But this approach is heavily dependent on the idea that “the future will be like the past”. This approach to ecological problems will be most useful for the very short term. The simplest example comes from weather forecasting in which the prognosis for today’s weather is what it was like yesterday with minor adjustments. We could observe trends with this approach but then we must have a statistical model that predicts, for example, that the trend is linear or exponential. But the history of science is that we can do much better by understanding the mechanisms underlying the changes we see. A good overview of the dilemmas of this inductive approach for conservation biology is provided by Caughley (1994). The operative question here is whether the inductive approach achieves problem resolutions more efficiently than the Popperian approach through conjecture and refutation.

Raerinne (2023, 2024) does biology in general and ecology in particular a disservice in criticizing Popper’s approach to ecology by arguing that ecology should not be criticized nor evaluated from the Popperian perspective. I think this judgement is wrong, and Raerinne’s conclusion arises from a philosophical viewpoint which could well have little applicability to how ecologists solve empirical problems in the real world. But you can judge.  

Carducci, A., Federigi, I. & Verani, M. (2020) Airborne transmission and its prevention: Waiting for evidence or applying the Precautionary Principle? Atmosphere, 11 (7), 710.doi: 10.3390/atmos11070710.

Caughley, G. (1994) Directions in conservation biology. Journal of Animal Ecology, 63, 215-244. doi: 10.2307/5542.

Gillson, L., Biggs, H. & Rogers, K. (2019) Finding common ground between adaptive management and evidence-based approaches to biodiversity conservation. Trends in Ecology & Evolution, 34, 31-44.doi: 10.1016/j.tree.2018.10.003.

Platt, J.R. (1964) Strong inference. Science, 146, 347-353.doi: 10.1126/science.146.3642.347.

Popper, K.R. (1963) Conjectures and Refutations: The Growth of Scientific Knowledge. Routledge and Kegan Paul, London.

Raerinne, J. (2023) Myths of past biases and progress in biology. Theory in Biosciences, 142, 383-399.doi: 10.1007/s12064-023-00403-2.

Raerinne, J. (2024) Popperian ecology is a delusion. Ecology and Evolution, 14, e11106.doi: 10.1002/ece3.11106.

Serrouya, R., Seip, D.R., Hervieux, D., McLellan, B.N., McNay, R.S., Steenweg, R., Heard, D.C., Hebblewhite, M., Gillingham, M. & Boutin, S. (2019) Saving endangered species using adaptive management. Proceedings of the National Academy of Sciences, 116, 6181-6186.doi: 10.1073/pnas.1816923116 .

Westgate, M.J., Likens, G.E. & Lindenmayer, D.B. (2013) Adaptive management of biological systems: A review. Biological Conservation, 158, 128-139.doi: 10.1016/j.biocon.2012.08.016 .

Do We Need to Replicate Ecological Experiments?

Leave a reply

If you read papers on the philosophy of science you will very quickly come across the concept of replication, the requirement to test the same hypothesis twice or more before you become too attached to your conclusions. As a new student or a research scientist you face this problem when you wish to replicate some previous study. If you do replicate, you risk being classed as an inferior scientist with no ideas of your own. If you refuse to replicate and try something new, you will be criticized as reckless and not building a solid foundation in your science.  

There is an excellent literature discussing the problem of replication in ecology in particular and science in general. Nichols et al. (2019) argue persuasively that a single experiment is not enough. Amrheim et al. (2019) approach the problem from a statistical point of view and caution that single statistical tests are a shaky platform for drawing solid conclusions. They point out that statistical tests not only test hypotheses, but also countless assumptions and particularly for ecological studies the exact plant and animal community in which the study takes place. In contrast to ecological science, medicine probably has more replication problems at the other extreme – too many replications – leading to a waste of research money and talent. (Siontis and Ioannidis 2018).

A graduate seminar could profitably focus on a list of the most critical experiments or generalizations of our time in any subdiscipline of ecology. Given such a list we could ask if the conclusions still stand as time has passed, or perhaps if climate change has upset the older predictions, or whether the observations or experiments have been replicated to test the strength of conclusions. We can develop a stronger science of ecology only if we recognize both the strengths and the limitations of our current ideas.

Baker (2016) approached this issue by asking the simple question “Is there a reproducibility crisis?” Her results are well worth visiting. She had to cast a wide net in the sciences so unfortunately there are no details specific to ecological science in this paper. A similar question in ecology would have to distinguish observational studies and experimental manipulations to narrow down a current view of this issue. An interesting example is explored in Parker (2013) who analyzed a particular hypothesis in evolutionary biology about plumage colour in a single bird species, and the array of problems of an extensive literature on sexual selection in this field is astonishing.

A critic might argue that ecology is largely a descriptive science that should not expect to develop observational or experimental conclusions that will extend very much beyond the present. If that is the case, one might argue that replication over time is important for deciding when an established principle is no longer valid. Ecological predictions based on current knowledge may have much less reliability than we would hope, but the only way to find out is to replicate. Scientific progress depends on identifying goals and determining how far we have progressed to achieving these goals (Currie 2019). To advance we need to discuss replication in ecology.

Amrhein, V., Trafinnow, D. & Greenland, S. (2019) Inferential statistics as descriptive statistics: There is no replication crisis if we don’t expect replication. American Statistician, 73, 262-270. doi: 10.1080/00031305.2018.1543137.

Baker, M. (2016) Is there a reproducibility crisis in science? Nature, 533, 452-454.

Currie, D.J. (2019) Where Newton might have taken ecology. Global Ecology and Biogeography, 28, 18-27. doi: 10.1111/geb.12842.

Nichols, J.D., Kendall, W.L. & Boomer, G.S. (2019) Accumulating evidence in ecology: Once is not enough. Ecology and Evolution, 9, 13991-14004. doi: 10.1002/ece3.5836.

Parker, T.H. (2013) What do we really know about the signalling role of plumage colour in blue tits? A case study of impediments to progress in evolutionary biology. Biological Reviews, 88, 511-536. doi: 10.1111/brv.12013.

Siontis, K.C. & Ioannidis, J.P.A. (2018) Replication, duplication, and waste in a quarter million systematic reviews and meta-analyses. Circulation: Cardiovascular quality and outcomes, 11, e005212. doi: 10.1161/CIRCOUTCOMES.118.005212.

On Critical Evaluation in Ecology

Leave a reply

Science proceeds by “conjecture-and-refutation” if we agree with Karl Popper (1963). There is a rich literature on science in general and ecological science in particular that is well worth a series of graduate discussions even if it is pre-2000 ancient history (Peters 1991, Weiner 1995, Woodward and Goodstein 1996). But I wish to focus on a current problem that I think is hindering ecological progress. I propose that ecological journals at this time are focusing their publications on papers that present apparent progress and are shedding papers that are critical of apparent progress. Or in Popper’s words, they focus on publishing ‘conjecture’ and avoid ‘refutation’. The most important aspect of this issue involves wildlife management and conservation issues. The human side of this issue may involve personal criticism and on occasion the loss of a job or promotion. The issue arises in part because of a confusion between the critique of ideas or data and the interpretation that all critiques are personal. So, the first principle of this discussion is that I discuss here only critiques of ideas or data.

There are many simple reasons for critiques of experimental design and data gathering. Are the treatments replicated, are the estimates of data variables reliable and sufficient, are proxy variables good or poor? Have the studies been carried out long enough? All these critiques can be summarized under the umbrella of measurement reliability. There are many examples we can use to illustrate these ideas. Are bird populations declining across the globe or locally? Are fisheries overharvesting particular species? Can we use climate change as a universal explanation of all changes in wildlife populations? Are survey methods for population changes across very large areas reliable? The problem is tied into the need for good or bad news that must be filtered to the news media or social media with high impact but little reliability. 

The problem at the level of science is the temptation to extrapolate beyond the limits of the available data. Now we come to the critical issue – how do our scientific journals respond to critical reviews of papers already published? My concern is that in the present time journals do not wish to receive or accept manuscripts that are critical of previously published papers. These decisions are no doubt confidential for journal publishers. There is perhaps some justification for this rejection policy, given that in the few cases where critiques are published on existing papers, the citation score of the original paper may greatly exceed that of the critique. So, conjecture pays, refutation does not.

Journals are flooded with papers and for the better journals I would expect at least a 60-80% rejection rate. For Science the rejection rate is 94%, for Nature 92%, and for the Journal of Animal Ecology 85% of submitted manuscripts are rejected. Consequently, the suggestion that they reserve space for ‘refutation’ is too negative to their publication model. There is little I can suggest if one in caught in this dilemma except to try another less premium journal, and remember that web searches find papers easily no matter where published. If you need inspiration, you can follow Peters (1991) and write a book critique and suffer the brickbats from the establishment (e.g. Nature 354: 444, 12 December 1991).

But if you are upset about a particular paper or series of papers, remember critiques are valuable but follow these rules for a critique:

  1. Keep it short, 5 typed pages should be near maximal length.
  2. Raise a set of major points. Do not try to cover everything.
  3. Summarize briefly the key points you are in agreement with, so they are not confounded in the discussion.
  4. Discuss what studies might distinguish hypothesis A vs B, or A+B vs C.
  5. Discuss what better methods of measurement might be used if funding is available.
  6. Never attack individuals or research groups. The discussion is about ideas, results, and inferences.

Decisions to accept some management actions may have to be taken immediately and journal editors must take that into consideration. Prognostication over accepting critiques may be damaging. But all actions must be continually evaluated and changed once the understanding of the problem changes.

There are too many examples to recommend reading about past and present controversies in ecology, so here are only two examples. Dowding et al. (2009) report a comment on suggested methods of controlling introduced pests on Macquarie Island in the Southern Ocean. I was involved in that discussion. A much bigger controversy in Canada involves Southern Mountain caribou populations which are in rapid decline. The proximate explanation for the decline is postulated to be predation by wolves and thus the suggested management action is shooting the wolves. Johnson et al. (2022), Lamb et al. (2022) and Superbie et al. (2022) provide an entre into this literature and the decisions of what to do now and in the future to prevent extinction of these ungulates. The caribou problem is complicated by the interaction of human alteration of landscapes and the natural processes of predation and food availability. Alas nothing is simple.

All these ecological dilemmas are controversial and the important role of criticism involving evaluations of alternative hypotheses are the only way forward for ecologists involved in controversies. In my opinion most ecological journals are not doing their part is publishing critiques of the conventional wisdom.

Dowding, J.E., Murphy, E.C., Springer, K., Peacock, A.J. & Krebs, C.J. (2009) Cats, rabbits, Myxoma virus, and vegetation on Macquarie Island: a comment on Bergstrom et al. (2009). Journal of Applied Ecology, 46, 1129-1132. doi: 10.1111/j.1365-2664.2009.01690.x.

Johnson, C.J., Ray, J.C. & St-Laurent, M.-H. (2022) Efficacy and ethics of intensive predator management to save endangered caribou. Conservation Science and Practice, 4: e12729. doi: 10.1111/csp2.12729.

Lamb, C.T., Willson, R., Richter, C., Owens-Beek, N., Napoleon, J., Muir, B., McNay, R.S., Lavis, E., Hebblewhite, M., Giguere, L., Dokkie, T., Boutin, S. & Ford, A.T. (2022) Indigenous-led conservation: Pathways to recovery for the nearly extirpated Klinse-Za mountain caribou. Ecological Applications 32 (5): e2581. doi: 10.1002/eap.2581.

Peters, R.H. (1991) A Critique for Ecology. Cambridge University Press, Cambridge, England. 366 pp. ISBN:0521400171.

Popper, K.R. (1963) Conjectures and Refutations: The Growth of Scientific Knowledge. Routledge and Kegan Paul, London. 412 pp. ISBN-13: 978-0415285940.

Superbie, C., Stewart, K.M., Regan, C.E., Johnstone, J.F. & McLoughlin, P.D. (2022) Northern boreal caribou conservation should focus on anthropogenic disturbance, not disturbance-mediated apparent competition. Biological Conservation, 265, 109426. doi: 10.1016/j.biocon.2021.109426.

Weiner, J. (1995) On the practice of ecology. Journal of Ecology, 83, 153-158.

Woodward, J. & Goodstein, D. (1996) Conduct, misconduct and the structure of science. American Scientist, 84, 479-490.

The Problem of Evidence in Ecology

Leave a reply

The good news is that the general public are becoming more concerned about the problems of wildlife management and conservation in general. The bad news arising from this interest is the lack of understanding exhibited by many of the comments in the media about ecological problems. This leads to a suggestion that we need an ecological “fact checking” team that looks at what is said about broad scale environmental issues and points out how much evidence there is for what is stated in the media. My interest in this issue is driven by so many news stories that are stated as fact with very little scientific understanding. Too many well-meaning reports fly around the media that border on complete error or complete nonsense. One consequence of this problem is a failure of evidence-based decision making for ecological problems (Christie et al. 2022).

This is not of course a problem confined to ecological science as you can see by reading nonsense claims about medical issues like Covid. It will not go away and with the climate crisis the number of ‘experts’ has multiplied. The problem comes down to the issue of evidence and how we evaluate evidence. A partial solution to this is better education about what is evidence in ecology as well as all of science. We need to teach workshops or courses on concrete examples of what is suggested to be evidence in ecological papers. The first step might be to analyse one or a few papers with the following procedure:

  1. What is the major conclusion of the paper?
  2. What data are presented to reach this conclusion?
  3. What background assumptions are being made to move from data to conclusions?

These questions lead us back to basic questions illustrated well by statistical inference. What is the ‘population’ to which the major conclusions apply? There is very little discussion of this in most ecological papers and the consequence can be overgeneralizations. Suppose for example we are examining the hypothesis that the geographic range of a species set is moving toward the poles because of a warming climate. We must for practical purposes restrict our study to a small set of species, so this is a major assumption that the species selected are a random sample of the biota under discussion. Another limitation is that it may be difficult to isolate climate change without considering for example human disturbances to the landscape from forestry and agriculture. A consequence of these complications is that our major conclusion for all this research rests on minimal data. So, a conclusion might be that we need to design further extensive studies. But perhaps of the 6 species under study, 4 are moving as the climate hypothesis predicts, but one is not moving at all, and one is moving in the opposite direction to what is predicted. Do we now turn our attention to these anomalous species that do not follow our major hypothesis? Or should we be happy that most of our candidate species follow the rule specified in our major conclusion?

       By doing manipulative experiments ecologists attempt to insert more rigor into their conclusions, but many of the generic questions mentioned above apply equally to these experimental designs. If we do a set of experiments in Iowa and in Germany, should we get the same results? We are back to the question of generality in all our studies. We hope for global rules, but experiments are all limited in time and space.

Can we escape all these bottlenecks with models that capture the generality and behave according to our assumptions? But models suffer from the same problems that make empirical studies difficult – what are the hidden assumptions? Taper et al. (2021) discuss the problem of errors arising from model misspecification in evaluating empirical data. Perhaps every ecological publication should end with an additional short section listing the assumptions made in reaching the major conclusions of the research.

These points come to the fore when we attempt to predict future environmental changes. A simple example is the hypothesis that, by humans increasing CO2 in the atmosphere, plants will increase photosynthesis and thus negate part or all the effects of climate change on our current ecosystems. This has caused much discussion ranging from planting more trees to alleviate climate change to relying on engineering solutions to climate change.

The bottom line that we should all recognize is that our predictions in ecology and our understanding of ecosystem changes are more limited than we admit. We know that we cannot rely on the old adage of the equilibrium hypothesis that “Mother Nature will take care of the earth” so all will be well. Wisdom always relies on critical evaluations which are too often lost in the media of our current world.

An important alternative approach is illustrated by the Conservation Evidence Journal and the approaches recommended by Sutherland et al. (2022) to specify local actions that can improve the conservation status of particular species or groups of species, for example by reintroducing birds to islands or areas from which they have been extirpated. The dichotomy here is a divide between the particular and the general, from short-term local questions to long-term general questions (Saunders et al. 2020). The hope is that progress on local questions will gradually inform the dominant global theories of ecology to bring them together and support the “devil in the details’ approach that can define ecological progress in our time (Sutherland et al. 2021).

Christie, A.P., et al. (2022) Principles for the production of evidence-based guidance for conservation actions. Conservation Science and Practice, 4, e579.doi: 10.1111/csp2.12663 .

Saunders, M.E., Janes, J.K. & O’Hanlon, J.C. (2020) Moving on from the Insect Apocalypse Narrative: Engaging with Evidence-Based Insect Conservation. BioScience, 70, 80-89.doi: 10.1093/biosci/biz143.

Sutherland, W.J., Downey, H., Frick, W.F., Tinsley-Marshall, P. & McPherson, T. (2021) Planning practical evidence-based decision making in conservation within time constraints: the Strategic Evidence Assessment Framework. Journal for Nature Conservation, 60, 125975.doi: 10.1016/j.jnc.2021.125975.

Sutherland, W.J. et al. (2022) Creating testable questions in practical conservation: a process and 100 questions. Conservation Evidence Journal, 19, 1-7.doi: 10.52201/CEJ19XIFF2753.

Taper, M., Lele, S., Ponciano, J., Dennis, B. & Jerde, C. (2021) Assessing the global and local uncertainty of scientific evidence in the presence of model misspecification Frontiers in Ecology and Evolution, 9, 679155.doi: 10.3389/fevo.2021.679155.

The Problem of Time in Ecology

Leave a reply

There is a problem in doing ecological studies that is too little discussed – what is the time frame of a good study? The normal response would be that the time frame varies with each study so that no guidelines can be provided. There is increasing recognition that more long-term studies are needed in ecology (e.g. Hughes et al. 2017) but the guidelines remain unclear.

The first issue is usually to specify a time frame, e.g. 5 years, 10 years. But this puts the cart before the horse, as the first step ought to be to define the hypothesis being investigated. In practice hypotheses in many ecological papers are poorly presented and there should not be one hypothesis but a series of alternative hypotheses. Given that, the question of time can be given with more insight. How many replicated time periods do you need to measure the ecological variables in the study? If your time scale unit is one year, 2 or 3 years is not enough to come to any except very tentative conclusions. We have instantly fallen into a central dilemma of ecology – studies are typically planned and financed on a 3–5-year time scale, the scale of university degrees.

Now we come up against the fact of climate change and the dilemma of trying to understand a changing system when almost all field work assumes an unchanging environment. Taken to some extreme we might argue that what happens in this decade tells us little about what will happen in the next decade. The way around this problem is to design experiments to test the variables that are changing ahead of time, e.g., what a 5⁰C temperature increase will do to the survival of your corals. To follow this approach, which is the classic experimental approach of science, we must assume we know the major variables affecting our population or community changes. At present we do not know the answer to this question, and we rely on correlations of a few variables as predictors of how large a change to expect.

There is no way out of this empirical box, which defines clearly how physics and chemistry differ from ecology and medicine. There are already many large-scale illustrations of this problem. Forest companies cut down old-growth timber on the assumption that they can get the forest back by replanting seedlings in the harvested area. But what species of tree seedlings should we replant if we are concerned that reforestation often operates on a 100–500-year time scale? And in most cases, there is no consideration of the total disruption of the ecosystem, and we ignore all the non-harvestable biodiversity. Much research is now available on reforestation and the ecological problems it produces. Hole-nesting birds can be threatened if old trees with holes are removed for forestry or agricultural clearing (Saunders et al. 2023). Replanting trees after fire in British Columbia did not increase carbon storage over 55 years of recovery when compared with unplanted sites (Clason et al. 2022). Consequently, in some forest ecosystems tree planting may not be useful if carbon storage is the desired goal.

At the least we should have more long-term monitoring of the survival of replanted forest tree seedlings so that the economics of planting could be evaluated. Short-term Australian studies in replanted agricultural fields showed over 4 years differences in survival of different plant species (Jellinek et al. 2020). For an on-the-ground point of view story about tree planting in British Columbia see:
https://thetyee.ca/Opinion/2023/11/02/Dont-Thank-Me-Being-Tree-Planter/. But we need longer-term studies on control and replanted sites to be more certain of effective restoration management. Gibson et al. (2022) highlighted the fact that citizen science over a 20-year study could make a major contribution to measuring the effectiveness of replanting. Money is always in short supply in field ecology and citizen science is one way of achieving goals without too much cost. 

Forest restoration is only one example of applied ecology in which long-term studies are too infrequent. The scale of restoration of temperate and boreal ecosystems is around 100 years, and this points to one of the main failures of long-term studies, that they are difficult to carry on after the retirement of the principal investigators who designed the studies.

The Park Grass Experiment begun in 1856 on 2.8 ha of grassland in England is the oldest ecological experiment in existence (Silvertown et al. 2006). As such it is worth a careful evaluation for the questions it asked and did not ask, for the scale of the experiment, and for the experimental design. It raises the question of generality for all long-term studies and cautions us about the utility and viability of many of the large-scale, long-term studies now in progress or planned for the future.

The message of this discussion is that we should plan for long-term studies for most of our critical ecological problems with clear hypotheses of how to conserve biodiversity and manage our agricultural landscapes and forests. We should move away from 2–3-year thesis projects on isolated issues and concentrate on team efforts that address critical long-term issues with specific hypotheses. Which says in a nutshell that we must develop a vision that goes beyond our past practices in scatter-shot, short-term ecology and at the same time avoid poorly designed long-term studies of the future.

Clason, A.J., Farnell, I. & Lilles, E.B. (2022) Carbon 5–60 Years After Fire: Planting Trees Does Not Compensate for Losses in Dead Wood Stores. Frontiers in Forests and Global Change, 5, 868024. doi: 10.3389/ffgc.2022.868024.

Gibson, M., Maron, M., Taws, N., Simmonds, J.S. & Walsh, J.C. (2022) Use of citizen science datasets to test effects of grazing exclusion and replanting on Australian woodland birds. Restoration Ecology, 30, e13610. doi: 10.1111/rec.13610.

Hughes, B.B.,et al. (2017) Long-term studies contribute disproportionately to ecology and policy. BioScience, 67, 271-281. doi.: 10.1093/biosci/biw185.

Jellinek, S., Harrison, P.A., Tuck, J. & Te, T. (2020) Replanting agricultural landscapes: how well do plants survive after habitat restoration? Restoration Ecology, 28, 1454-1463. doi: 10.1111/rec.13242.

Saunders, D.A., Dawson, R. & Mawson, P.R. (2023) Artificial nesting hollows for the conservation of Carnaby’s cockatoo Calyptorhynchus latirostris: definitely not a case of erect and forget. Pacific Conservation Biology, 29, 119-129. doi: 10.1071/PC21061.

Silvertown, J., Silvertown, J., Poulton, P. & Biss, P.M. (2006) The Park Grass Experiment 1856–2006: its contribution to ecology. Journal of Ecology, 94, 801-814. doi: 10.1111/j.1365-2745.2006.01145.x.

Back to Nature vs. Nurture

2 Replies

The ancient argument of ‘nature’ versus ‘nurture’ continues to arise in biology. The question has arisen very forcefully in a new book by James Tabery (Tabery 2023). The broad question he examines in this book is the conflict between ‘nature’ and ‘nurture’ in western medicine. In a broad sense ‘nature’ is discussed as the modern push in medicine to find the genetic basis of some of the common human degenerative diseases – Parkinson’s, dementia, asthma, diabetes, cancer, hypertension – to mention only a few medical problems of our day. The ‘nature’ approach to medicine in this book is represented by molecular genetics and the Human Genome Project. The ‘nurture’ approach to treating these medical conditions is via studying health outcomes in people subject to environmental contamination, atmospheric pollution, water quality, chemicals in food preparations, asbestos in buildings, and other environmental issues including how children are raised and educated. The competition over these two approaches was won very early by the Human Genome Project, and many of the resources for medicine over the last 30 years were put into molecular biology which made spectacular progress in diving into the genome of affected people and then making great promises of personalized medicine. The environmental approach to these medical conditions received much less money and was not viewed as sufficiently scientific. The irony of all this in retrospect is that the ‘nature’ or DNA school had no hypotheses about the problems being investigated but relied on the assumption that if we got enough molecular genetic data on thousands of people that something would jump out at us, and we would locate for example the gene(s) causing Parkinson’s, and then we could alter these genes with gene therapy or specific pharmaceuticals. By contrast the ‘nurture’ school had many specific hypotheses to test about air pollution and children’s health, about lead in municipal water supply and brain damage, and a host of very specific insights about how some of these health problems could be alleviated by legislation and changes in diet for example.

So, the question then becomes where are we today? The answer Tabery (2023) gives is that the ‘nature’ or molecular genetic “personalized medicine” approach has largely failed in achieving its goals despite the large amount of money invested because there is no single or small set of genes that cause specific diseases, but many genes that have complex interactions. In contrast, the ‘nurture’ school has made progress in identifying conditions that help decrease the occurrence of some of our common diseases, realizing that the problems are often difficult because they require changes in human behaviour like stopping smoking or improving diets.

All this discussion would possibly produce the simple conclusion that both “nature” and “nurture” are both involved in these complex human conditions. So, what could this medical discussion tell us about the condition of modern ecological science? I think two things perhaps. First, it is a general error to use science without hypotheses. Yet this is too often what ecologists do – gather a large amount of data that can be measured without too much prolonged effort and then try to make sense of it by applying hypotheses after the fact. And second, technology in ecology can be a benefit or a curse. Take, for example, the advances in vertebrate ecology that have come from the ability to describe the movements of individual animals in space. To have a map of hundreds of locations of an individual animal provides good natural history but does not address any specific hypothesis. Contrast this approach with that of Studd et al. (2021) and Shiratsuru et al. (2023) who use movement data to test important questions about kill rates of predators on different species of prey.

Many large-scale ecological approaches suffer from the same problem as the ‘nature’ paradigm – use ‘big science’ to measure many variables and then try to answer some important question for example about how climate change is affecting communities of plants and animals. Nagy et al. (2021) and Li et al. (2022) provide excellent examples of this approach. Schimel and Keller (2015) discuss what is needed to bring hypothesis testing to ‘big science’. Lindenmayer et al. (2018) discuss how conventional, question-driven long-term monitoring and hypothesis testing need to be combined with ‘big science’ to better ecological understanding. Pau et al. (2022) give a warning of how ‘big science’ data from airborne imaging can fail to agree with ground-based field studies in one core NEON grassland site in central USA.

The conclusion to date is that there is little integration in ecology of the equivalent of “nature” and “nurture” in medicine if in ecology we match ‘big science’ with ‘nature’ and field studies on the ground with ‘nurture’. Without that integration we risk in future another negative review in ecology like that provided now by Tabery (2023) for medical approaches to human diseases.

Lindenmayer, D.B., Likens, G.E. & Franklin, J.F. (2018) Earth Observation Networks (EONs): Finding the Right Balance. Trends in Ecology & Evolution, 33, 1-3.doi: 10.1016/j.tree.2017.10.008.

Li, D., et al. (2022) Standardized NEON organismal data for biodiversity research. Ecosphere, 13, e4141.doi:10.1002/ecs2.4141.

Nagy, R.C., et al. (2021) Harnessing the NEON data revolution to advance open environmental science with a diverse and data-capable community. Ecosphere, 12, e03833.doi: 10.1002/ecs2.3833.

Pau, S., et al. (2022) Poor relationships between NEON Airborne Observation Platform data and field-based vegetation traits at a mesic grassland. Ecology, 103, e03590.doi: 10.1002/ecy.3590.

Schimel, D. & Keller, M. (2015) Big questions, big science: Meeting the challenges of global ecology. Oecologia, 177, 925-934.doi: 10.1007/s00442-015-3236-3.

Shiratsuru, S., Studd, E.K., Majchrzak, Y.N., Peers, M.J.L., Menzies, A.K., Derbyshire, R., Jung, T.S., Krebs, C.J., Murray, D.L., Boonstra, R. & Boutin, S. (2023) When death comes: Prey activity is not always predictive of diel mortality patterns and the risk of predation. Proceedings of the Royal Society B, 290, 20230661.doi.

Studd, E.K., Derbyshire, R.E., Menzies, A.K., Simms, J.F., Humphries, M.M., Murray, D.L. & Boutin, S. (2021) The Purr-fect Catch: Using accelerometers and audio recorders to document kill rates and hunting behaviour of a small prey specialist. Methods in Ecology and Evolution, 12, 1277-1287.doi. 10.1111/2041-210X.13605

Tabery, J. (2023) Tyranny of the Gene: Personalized Medicine and the Threat to Public Health. Knopf Doubleday Publishing Group, New York. 336 pp. ISBN: 9780525658207.

The Meaningless of Random Sampling

Leave a reply

Statisticians tell us that random sampling is necessary for making general inferences from the particular to the general. If field ecologists accept this dictum, we can only conclude that it is very difficult to nearly impossible to reach generality. We can reach conclusions about specific local areas, and that is valuable, but much of our current ecological wisdom on populations and communities relies on the faulty model of non-random sampling. We rarely try to define the statistical ‘population’ which we are studying and attempting to make inferences about with our data. Some examples might be useful to illustrate this problem.

Marine ecologists ae mostly agreed that sea surface temperature rise is destroying coral reef ecosystems. This is certainly true, but it camouflages the fact that very few square kilometres of coral reefs like the Great Barrier Reef have been comprehensively studied with a proper sampling design (e.g. Green 1979, Lewis 2004). When we analyse the details of coral reef declines, we find that many species are affected by rising sea temperatures, but some are not, and it is possible that some species will adapt by natural selection to the higher temperatures. So we quite rightly raise the alarm about the future of coral reefs. But in doing so we neglect in many cases to specify the statistical ‘population’ to which our conclusions apply.

Most people would agree that such an approach to generalizing ecological findings is tantamount to saying the problem is “how many angels can dance on the head of a pin”, and in practice we can ignore the problem and generalize from the studied reefs to all reefs. And scientists would point out that physics and chemistry seek generality and ignore this problem because one can do chemistry in Zurich or in Toronto and use the same laws that do not change with time or place. But the ecosystems of today are not going to be the ecosystems of tomorrow, so generality in time cannot be guaranteed, as paleoecologists have long ago pointed out.

It is the spatial problem of field studies that collides most strongly with the statistical rule to random sample. Consider a hypothetical example of a large national park that has recently been burned by this year’s fires in the Northern Hemisphere. If we wish to measure the recovery process of the vegetation, we need to set out plots to resample. We have two choices: (1) lay out as many plots as possible, and sample these for several years to plot recovery. Or (2) lay out plots at random each year, never repeating the same exact areas to satisfy the specifications of statisticians to “random sample” the recovery in the park. We typically would do (1) for two reasons. Setting up new plots each year as per (2) would greatly increase the initial field work of defining the random plots and would probably mean that travel time between the plots would be greatly increased. Using approach (1) we would probably set out plots with relatively easy access from roads or trails to minimize costs of sampling. We ignore the advice of statisticians because of our real-world constraints of time and money. And we hope to answer the initial questions about recovery with this simpler design.

I could find few papers in the ecological literature that discuss this general problem of inference from the particular to the general (Ives 2018, Hauss 2018) and only one that deals with a real-world situation (Ducatez 2019). I would be glad to be sent more references on this problem by readers.

The bottom line is that if your supervisor or research coordinator criticizes your field work because your study areas are not randomly placed or your replicate sites were not chosen at random, tell him or her politely that virtually no ecological research in the field is done by truly random sampling. Does this make our research less useful for achieving ecological understanding – probably not. And we might note that medical science works in exactly the same way field ecologists work, do what you can with the money and time you have. The law that scientific knowledge requires random sampling is often a pseudo-problem in my opinion.  

Ducatez, S. (2019) Which sharks attract research? Analyses of the distribution of research effort in sharks reveal significant non-random knowledge biases. Reviews in Fish Biology and Fisheries, 29, 355-367. doi. 10.1007/s11160-019-09556-0

Green, R.H. (1979) Sampling Design and Statistical Methods for Environmental Biologists. Wiley, New York. 257 pp.

Hauss, K. (2018) Statistical Inference from Non-Random Samples. Problems in Application and Possible Solutions in Evaluation Research. Zeitschrift fur Evaluation, 17, 219-240. doi.

Ives, A.R. (2018) Informative Irreproducibility and the Use of Experiments in Ecology. BioScience, 68, 746-747. doi. 10.1093/biosci/biy090

Lewis, J. (2004) Has random sampling been neglected in coral reef faunal surveys? Coral Reefs, 23, 192-194. doi: 10.1007/s00338-004-0377-y.

The Time Frame of Ecological Science

Leave a reply

Ecological research differs from many branches of science in having a more convoluted time frame. Most of the sciences proceed along paths that are more often than not linear – results A → results B → results C and so on. Of course, these are never straight linear sequences and were described eloquently by Platt (1964) as strong inference:

“Strong inference consists of applying the following steps to every problem in science, formally and explicitly and regularly: 1) Devising alternative hypotheses; 2) Devising a crucial experiment (or several of them), with alternative possible outcomes, each of which will, as nearly as possible, exclude one or more of the hypotheses; 3) Carrying out the experiment so as to get a clean result; “Recycling the procedure, making sequential hypotheses to refine the possibilities that remain; and so on. It is like climbing a tree.” (page 347 in Platt).

If there is one paper that I would recommend all ecologists read it is this paper which is old but really is timeless and critical in our scientific research. It should be a required discussion topic for every graduate student in ecology.

Some ecological science progresses as Platt (1964) suggests and makes good progress, but much of ecology is lost in a failure to specify alternative hypotheses, in changing questions, in abandoning topics because they are too difficult, and in a shortage of time. It is the time component of ecological research that I wish to discuss in this blog.

The idea of long-term studies has always been present in ecology but was perhaps brought to our focus by the compilation by Gene Likens in 1989 in a book of 14 chapters that are as vital now as they were 34 years ago. Many discussions of long-term studies are now available to examine this issue. Buma et al. (2019) for example discuss plant primary succession at Glacier Bay, Alaska which has 100 years of data, and which illustrates in a very slow ecosystem a test of conventional rules of community development. Cusser et al. (2021) follow this by asking a critical question of how long field experiments need to be. They restrict long-term to be > 10 years of study and used data from the USA LTER sites. This question depends very much on the community or ecosystem of study. Studies in areas with a stable climate produced results more quickly than those in highly seasonal environments, and plant studies needed to be longer term than animal studies to reach stable conclusions. Ten years may not be enough.

Reinke et al. (2019) reviewed 3 long term field studies and suggest that long-term studies can be useful to allow us to predict how ecosystems will change with time. All these studies lead to three unanswered questions that are critical for progress in ecology. The first question is how we decide as a community exactly which ecological system we should be studying long-term. No one knows how to answer this question, and a useful graduate seminar could debate the utility of what are now considered model long-term studies, such as the three highlighted in Reinke et al. (2019) or the Park Grass Experiment (Addy et al. 2022). At the moment these decisions are opportunistic, and we should debate how best to proceed. Clearly, we cannot do everything for every population and community of interest, so how do we choose? We need model systems that can be applied to a wide variety of environments across the globe and that ask questions of global significance. Many groups of ecologists are trying to do this, but a host of decisions about who to fund and support in what institution are vital to avoid long-term studies driven more by convenience than by ecological importance.

A second question involves the implied disagreement whether many important questions in ecology today could be answered by short-term studies, so we reach a position where there is competition between short- and long-term funding. These decisions about where to do what for how long are largely uncontrolled. One would prefer to see an articulated set of hypotheses and predictions to proceed with decision making, whether for short-term studies suitable for graduate students or particularly for long-term studies that exceed the life of individual researchers.

A third question is the most difficult one of the objectives of long-term research. Given climate change as it is moving today, the hope that long-term studies will give us reliable predictions of changes in communities and ecosystems is at risk, the same problem of extrapolating a regression line beyond the range of the data. Depending on the answer to this climate dilemma, we could drop back to the suggestion that because we have only a poor ability to predict ecological change, we should concentrate more on widespread monitoring programs and less on highly localized studies of a few sites that are of unknown generality. Testing models with long-term data is enriching the ecological literature (e.g. Addy et al 2022). But the challenge is whether our current understanding is sufficient to make predictions for future populations or communities. Should ecology adopt the paradigm of global weather stations?

Addy, J.W.G., Ellis, R.H., MacLaren, C., Macdonald, A.J., Semenov, M.A. & Mead, A. (2022) A heteroskedastic model of Park Grass spring hay yields in response to weather suggests continuing yield decline with climate change in future decades. Journal of the Royal Society Interface, 19, 20220361. doi: 10.1098/rsif.2022.0361.

Buma, B., Bisbing, S.M., Wiles, G. & Bidlack, A.L. (2019) 100 yr of primary succession highlights stochasticity and competition driving community establishment and stability. Ecology, 100, e02885. doi: 10.1002/ecy.2885.

Cusser, S., Helms IV, J., Bahlai, C.A. & Haddad, N.M. (2021) How long do population level field experiments need to be? Utilising data from the 40-year-old LTER network. Ecology Letters, 24, 1103-1111. doi: 10.1111/ele.13710.

Hughes, B.B., Beas-Luna, R., Barner, A., et al. (2017) Long-term studies contribute disproportionately to ecology and policy. BioScience, 67, 271-281. doi: 10.1093/biosci/biw185.

Likens, G.E. (Editor, 1989) Long-term Studies in Ecology: Approaches and Alternatives. Springer Verlag, New York. 214 pp. ISBN: 0387967435.

Platt, J.R. (1964) Strong inference. Science, 146, 347-353. doi: 10.1126/science.146.3642.347.

Reinke, B.A., Miller, D.A.W. & Janzen, F.J. (2019) What have long-term field studies taught as about population dynamics? Annual Review of Ecology, Evolution, and Systematics, 50, 261-278. doi: 10.1146/annurev-ecolsys-110218-024717.

The Two Questions: So what? What next?

Leave a reply

Assuming that these two questions are not copyright, I wanted to explore them as a convenient part of writing a scientific or popular paper in ecology, conservation, and wildlife and fisheries management. To protect the innocent, I will not identify which of many ecological colleagues has stimulated this blog.

The first question should be addressed in every scientific paper but clearly is not if you read a random sample of the articles in many ecological journals. So what? is the critical question of exactly what current problem this paper or book will contribute to. It is the microscopic and macroscopic focus of why we do science, and it does not matter at all if it addresses a minor problem or a major catastrophe like species loss in conservation. In writing one should assume that time is the critical limiting factor in our lives, and while it is fine to be entertained by watching a movie, scientists do not read scientific papers to be entertained. Some journals demand that the abstract of every paper ends with a statement of the importance of the research findings, captured by So what? Too often these statements are weak and editors as well as granting agencies should demand more incisive statements. Asking yourself So what? can be a useful guide as you progress in your research and evaluate others.

While most scientists should agree on the findings presented in a paper or lecture, not all of them will agree about the importance of the answer to So what? What is a major and important scientific finding for some may be of minor significance to others, but the key is to remember here that science is a broad church that should be progressing on a broad front, so that differences of opinion are to be expected, and we rely on evidence to evaluate these differences of opinion. Tests of ideas that turn out to be incorrect or only partly correct must not be considered as failures. If you doubt that, interview any senior scientist in your area and ask about progress and regress during their scientific career. If you find a scientist who insists that they were correct in all their ideas, you should probably request them to go into politics to improve decision making in the real world.

The second question is probably the most critical for all scientific research. Once research is completed, there are two paths. If the original question or problem is solved or answered, the question becomes what does this work suggest needs to be done to advance the general area of research. Most typically however a research project will end up with more questions than it solves. The growing end of science is the critical one, and by asking What next? we delve deeper into the area of research to fill in details that were not evident when it was started. Read Sutherland et al. (2013, 2022) for an excellent example of this approach in conservation science. A simple example of this approach comes from many conservation problems. A particular species of bird may be thought to be declining in numbers, so the first issue is whether this is correct, and so an investigation into the changes in abundance of the species becomes the first step. This could lead to an analysis of the demography of the species population, birth, death and movement rates could be determined to isolate more precisely why abundance is changing. Given these data, the next step might be (for example) why the death rate is increasing if indeed this is the case. The next step is what management methods can be applied to reduce the death rate, and does this situation apply to other closely related species. It is important that asking What next? does not imply a linear sequence in time, and a study could be designed to address more than one question at the same time. We finish the What next? approach with a web of information and conclusions that address a broader question than the original simple question. And What next? should not be answered with a broad set of statements like “climate change is the cause” but by suggestions of very specific experiments and studies to carry investigations forward.

The result in ecology is an increasing precision of thought into ecological interactions and the processes that link species, communities, and ecosystems to very large questions such as the environmental response to climate change. Not all questions need to be large-scale because there are important local questions about the adequacy of designated parks and protected areas to protect species, communities, and ecosystems. The key message is that ecological understanding is not static but grows incrementally by well-designed research programs that by themselves seem to address only small-scale issues.

Seemingly failed research programs are not to be scorned but rather to indicate what avenues of research have not led to good insights. In a sense ecological science is like an evolutionary tree in which some branches fade away with time and others blossom into a variety of forms that surprise us all. So, my advice is to carry on asking these two simple questions in science to help sharpen your research program.

Sutherland, W.J., Freckleton, R.P., Godfray, H.C.J., Beissinger, S.R., Benton, T., Cameron, D.D., Carmel, Y., Coomes, D.A., Coulson, T., Emmerson, M.C., Hails, R.S., Hays, G.C., Hodgson, D.J., Hutchings, M.J., Johnson, D., Jones, J.P.G., Keeling, M.J., Kokko, H., Kunin, W.E. & Lambin, X. (2013) Identification of 100 fundamental ecological questions. Journal of Ecology, 101, 58-67.doi: 10.1111/1365-2745.12025.

Sutherland, W.J. & Jake M. Robinson, D.C.A., Tim Alamenciak, Matthew Armes, Nina Baranduin, Andrew J. Bladon, Martin F. Breed, Nicki Dyas, Chris S. Elphick, Richard A. Griffiths, Jonny Hughes, Beccy Middleton, Nick A. Littlewood, Roger Mitchell, William H. Morgan, Roy Mosley, Silviu O. Petrovan, Kit Prendergast, Euan G. Ritchie,Hugh Raven, Rebecca K. Smith, Sarah H. Watts, Ann Thornton (2022) Creating testable questions in practical conservation: a process and 100 questions. Conservation Evidence Journal, 19, 1-7.doi: 10.52201/CEJ19XIFF2753.

Belief vs. Evidence

2 Replies

There is an interesting game you could enter into if you classified the statements you hear or read in the media or in ecological papers. The initial dichotomy is whether or not a statement is a BELIEF or EVIDENCE BASED. There is a continuum between these polar opposites so there can easily be disagreements based on a person’s background. If I say “I believe that the earth is round” you will recognize that this is not a simple belief but a physical fact that is evidence-based. Consequently we use the word ‘belief’ in many different ways. If I say that “Aliens from outer space are firing ray guns to cause flooding in California and Australia”, it is unlikely that you will be convinced because there is no evidence of how this process could work.

If we listen to the media or read the news, you will hear many statements that I or we ‘believe’ that speed limits on streets should be reduced, or that certain types of firearms should be prohibited. The natural response of a scientist to such statements is to ask for what evidence is available that such actions will solve problems, and if there is no evidence, we deal only with opinions or beliefs. If  you lived several hundred years ago, you would be told that “malaria” was a disease caused by “bad air” coming from swamps and rivers, since there was no evidence at the time about microorganisms causing disease. So in a broad sense historical progress was made by people looking for ‘evidence’ to temper and test ‘beliefs’.

How does all this relate to ecological science? I would add the requirement to papers that state some conclusions in ecology journals to also state the beliefs the paper rely on to reach its conclusions, in addition to stating clear hypotheses and alternative hypotheses. Consider the simple case of random sampling, a basic requirement in all statistical methods. But almost no paper states what statistical population is being sampled, and if it does often the study plots are not placed randomly. The standard excuse to this is that our results apply to a large biome, and it is not physically possible to sample randomly, or that we get the same results whether we sample randomly or not. Whatever the excuse, we need to recognize this as a belief or an assumption, a less damning scientific term. And if this assumption is not accepted it is possible to sample other areas or with other methods to test if the evidence validates the assumption. Evidence can always be improved with enough funding, and this replication is exactly what many scientists are doing daily.

Until recently most scientists believed that CO2 was good for plants, and so the more CO2 the better. But the evidence provided was based on simple theory and short term lab experiments. Reich et al. (2018) and Zhu et al. (2018) pointed out that this was not correct when long-term studies were done on C3 plants like rice. So this is a good illustration of the progress of science from belief to evidence. And over the past 50 years it has become very clear that increased CO2 increases atmospheric temperature with drastic climatic and biodiversity consequences (Ripple et al. 2021). The result of these scientific advances is that now there is an extensive amount of scientific research giving the empirical evidence of climate change and CO2 effects on plants and animals. Most people agree with these broad conclusions, but there are people in large corporations and governments around the world who deny these scientific conclusions because they believe that climate change is not happening and is of little consequence to biodiversity or to daily life.

It is quite possible to ignore all the scientific literature about the consequences of climate change, CO2 increase, and biodiversity loss but the end result of passing over these problems now will fall heavily onto your children and grandchildren. The biosphere is screaming the message that ignorance will not necessarily lead to bliss.

Reich, P.B., Hobbie, S.E., Lee, T.D. & Pastore, M.A. (2018) Unexpected reversal of C3 versus C4 grass response to elevated CO2 during a 20-year field experiment. Science, 360, 317-320.doi: 10.1126/science.aas9313.

Ripple, W.J., Wolf, C., Newsome, T.M., Gregg, J.W., Lenton, T.M., Palomo, I., Eikelboom, J.A.J., Law, B.E., Huq, S., Duffy, P.B. & Rockström, J. (2021) World Scientists’ Warning of a Climate Emergency 2021. BioScience, 71, 894-898.doi: 10.1093/biosci/biab079.

Shivanna, K.R. (2022) Climate change and its impact on biodiversity and human welfare. Proceedings of the Indian National Science Academy, 88, 160-171.doi: 10.1007/s43538-022-00073-6.

Watson, R., Kundzewicz, Z.W. & Borrell-Damián, L. (2022) Covid-19, and the climate change and biodiversity emergencies. Science of The Total Environment, 844, 157188.doi: 10.1016/j.scitotenv.2022.157188.

Williams, S.E., Williams, S.E. & de la Fuente, A. (2021) Long-term changes in populations of rainforest birds in the Australia Wet Tropics bioregion: A climate-driven biodiversity emergency. PLoS ONE, 16.doi: 10.1371/journal.pone.0254307.

Zhu, C., Kobayashi, K., Loladze, I., Zhu, J. & Jiang, Q. (2018) Carbon dioxide (CO2) levels this century will alter the protein, micronutrients, and vitamin content of rice grains with potential health consequences for the poorest rice-dependent countries. Science Advances, 4, eaaq1012 doi: 10.1126/sciadv.aaq1012.