Yet more reasons to fund diverse basic science

Research is an incremental, iterative process. New advances build on those that came before, and open up new lines of research to follow afterwards. But not all research leads anywhere. The office drawers of academics are full of manuscripts that never got published, or data from studies that never showed any results. Whole fields such as phrenology enjoy periods in the sun before fading away (if you know of any modern research that directly descends from phrenology, let me know in the comments).

In this respect, research is a lot like the Tree of Life, with each project or study being a species. Species may give rise to new species (new research questions), or they may go extinct, but the Tree of Research (hopefully) endures. 

Mathematicians have tools for understanding tree-generating processes such as these: birth-death models. These specify what types of tree are likely to be generated based on the rates of speciation and extinction for individual species.

Graham Budd and I recently published a study investigating the properties of these processes. Trees generating by birth-death processes are very vulnerable; a newly created tree with only a few species can easily stop growing if all of those species go extinct. On the flip side, trees that have already generated many species can be very robust and are hard to push towards extinction. A consequence of this is that trees that do survive a long time tend to have bursts of rapid diversification at the start. Looking more deeply into the trees that survive, we find that the surviving lineages (those species that have modern descendants) are always diversifying like crazy, speciating at twice the rate we would otherwise expect.

Trees that survive for a long time tend to diversify quickly when they are small (Budd & Mann 2018)

What does this have to do with research funding? Increasingly research funding is allocated on the basis of competitive grant applications. I have written before about the waste involved in this, but another consequence is that research diversity suffers. To get a grant in the UK for example, you must convince the funder and reviewers that you have a very good chance to make notable findings and have impact in academia, industry and elsewhere. This requirement, along with the notable and growing bias towards funding senior academics who have substantial previous funding, favours research that is predictable, which follows the researcher's previously demonstrated expertise and where preliminary results are already available. This in turn reduces the diversity of possible research avenues that might be explored. 

What is the result of reducing diversity? Our research suggests that if we depress the diversification of research we risk extinguishing the Tree of Research altogether. If we focus research efforts too narrowly we put too many eggs in too few baskets. The future success of those research areas is less predictable than we might like to think - few phrenologists thought that their expertise would one day be seen as quackery. If those bets don't pay off then scientific progress may slow down or stop altogether.

Lineages that give rise to long-term descendants are always diversifying quickly (red lines). Green lines diversify slowly and go extinct (Budd & Mann 2018)

But surely, you might reply, isn't it a good idea to check on the track record of scientists and look at their ideas before giving them lots of public money? No doubt there is some value in scrutiny, but given the competition for academic jobs I think we can safely say that most academics have already been scrutinised before they start asking for money. As stated above, I believe our ability to predict what will be a success is highly limited. Moreover, several studies have shown that we can't even agree on what is good or not anyway, reducing weeks or months of labour to a lottery. Just as importantly, as another of my recent papers, this time with Dirk Helbing, has shown, the way that we allocate rewards and resources based on past success can distort the things that people choose to research, and as a result reduce the collective wisdom of academia as a whole. Dirk and I showed that too much diversity in what people choose to research is greatly preferable to too little: as a collective we need the individuals who research seemingly mad questions with little chance of success. Unfortunately, the most natural ways to reward and fund academics based on their track record would seem to create far too little diversity of research.

Rewards influence diversity and collective wisdom. Too much diversity (orange line) is better than too little (black and blue lines). (Mann & Helbing 2017).

So what can be done? Dirk and I showed that collective intelligence can be optimised by retrospectively rewarding individuals who are proved right when the majority is wrong. This mirrors approaches in statistics for ensemble learning called Boosting, wherein we train models to predict data that other models were unable to predict accurately. So I would be in favour of targeting grants to those who have gone against prevailing opinion and been proved right. However, we also showed that if agents choose what to research at random this will create greater collective intelligence than many reward schemes. This would support funding many scientists with unconditional funding that supports research wherever their curiosity takes them. This would have the additional advantage of removing much of the deadweight cost of grant applications.

References:

Budd & Mann (2018): History is written by the victors: The effect of the push of the past on the fossil record. Evolution

Mann & Helbing (2017): Optimal incentives for collective intelligence. PNAS

Pier et al. (2018) Low agreement among reviewers evaluating the same NIH grant applications. PNAS

Price (2014), The NIPS Experiment: http://blog.mrtz.org/2014/12/15/the-nips-experiment.html