deco-dots

The Flawed System for Science

August 19, 2021

Science is awesome. Armed with our senses, tools, a suitable experiment, and our reasoning we can discover facts about this universe and others. But for all the beauty, elegance, and simplicity of the scientific method, the process of contributing to the scientific community has become inane. Not only is it detrimental for researches, but it also actively hinders the effects of scientific discovery on society.

I will refer to the "scientific method" as the well-known procedure of observation, question, hypothesis, experiment, analysis, and conclusion. Instead, I will use "scientific process" for the official channels for advancing scientific knowledge, namely, that of scientific journals, such as Nature, Cell, Science, and The Lancet. These journals have garnered a large amount of prestige. If anyone wants to advance the canon of scientific knowledge, he needs to play by the rules set by them. Society considers them to be the pinnacle of epistemological rigor. But even if they do have high standards, they are far from perfect. Many studies with obvious flaws have been published in Journals, just to be retracted years later after they had already been cited multiple times in other papers. Others never passed replication. In getting deeper into this process, one discovers that this carte blanche on truth is not justified.

But the main problem with these journals is the perverse system they lay out. As Stephen Buranyi explains in Is the staggeringly profitable business of scientific publishing bad for science?

Scientists create work under their own direction – funded largely by governments – and give it to publishers for free; the publisher pays scientific editors who judge whether the work is worth publishing and check its grammar, but the bulk of the editorial burden – checking the scientific validity and evaluating the experiments, a process known as peer review – is done by working scientists on a volunteer basis. The publishers then sell the product back to government-funded institutional and university libraries, to be read by scientists – who, in a collective sense, created the product in the first place.

So while publishers like Reed-Elsevier (RELX) boast a profit margin of 36%, academic researchers have low salaries and get no money for the grueling process of peer review. Some might say that research is satisfactory in its own right. But while it's true that many scientists love the job of advancing the frontier of knowledge for humanity, this doesn't mean that they should get the short end of the stick. Even more, people working at universities don't get to do a lot of research. David Matthews writes in Times Higher Education:

"Those lucky enough to have become full professors – supposedly the light at the end of the tunnel for struggling junior scholars – spend just 17 per cent of their time on their own research. Teaching, research supervision and “management and organisational tasks” were all bigger commitments."

Given these concerning issues, we are left to wonder why scientists, perhaps the greatest minds of humanity, are subjugated by this system where they have the least amount of power. The way to get ahead in the scientific community is to get prestige as quantified by papers in renowned journals. So the goal of the academic researcher is not to investigate areas with the most promising effects for humanity, but to focus on what will get published and cited, instead. These objectives are usually at odds. Agnes Callard, Associate Professor at the University of Chicago, laments in her essay Publish or Perish that

academic writing is obsessed with other academic writing—with finding a “gap in the literature” as opposed to answering a straightforwardly interesting or important question

Another worrying effect of journals' influence is publication bias. Researchers are more likely to submit studies if the results are more surprising or statistically stronger. This means that lots of studies will never be submitted, since the publisher's incentives to go ahead with the publishing process are low. But more concerning, what is submitted and then published is more likely to be extraordinary because of methodological errors and statistical abnormalities. Just like a broken clock is right twice a day, if data is randomly collected enough times, strange effects are bound to appear. It's true that these effects usually fail replication, but not before the studies are published and cited.

In summary, the official process for advancing science is flawed:

  • Journals are expensive. Universities have to pay for them or risk having their staff missing out on the latest discoveries.
  • Universities pay low salaries, considering the high demands of the job and the credentials required to get the position in the first place.
  • As explained above, academic scientists don't have a lot of time for research.
  • Many studies are government-funded through grants, yet inaccessible to people who do not subscribe to the paywalled journals.
  • Scientists have to pay high publication fees, yet they act as reviewers for the publishers for free.
  • Publishers have an incentive to publish positive results instead of negative ones (publication bias). They are also more likely to accept studies on more fashionable subjects, and novel and challenging results, over more important and solid work. This generates negative effects on the validity of published studies.
  • People who are not specialists, most of us, cannot read papers since the language is aimed at other scientists. This is, many times, the result of scientists trying to sound smart to their peers, though its effect tends to be the opposite.
  • Researchers rely too much on universities. They depend on them to get their salaries, equipment, grants, and access to other journals. That being the case, they are less likely to speak their minds on controversial topics, or to try lines of research that run counter to the line set by the University.
  • No open data. Most papers show only the results summary, but not the raw data that was used for analysis. With open access to the original numbers, it is easier to find errors, so open data would only increase the trustworthiness of studies from authors who decide to apply this approach.

For the fundamental endeavor of creating scientific knowledge, we rely on a process that exploits scientists so that a few companies can make big profits. Most scientists have chosen their work out of love for seeking out the truth. They fight to get funding for pricey projects. They move to distant countries. They visit places with poor living conditions or live on them. That we have a system that does them such a poor service should be a scandal.

Alternatives for Scientists

Until a better process arrives, there are alternatives for scientists who don't want to play the standard game:

Favoring New Promising Journals

The big players do provide higher prestige, but they don't have a monopoly on it. Nobel Prize winner Randy Schekman pledges:

I have now committed my lab to avoiding luxury journals, and I encourage others to do likewise.

Schekman is the editor of eLife, an internet journal based on open science and technology innovation. It's true that it's easier to go against the establishment after winning the Nobel Prize, but that doesn't taint the effort.

Independent Research

This is the path of science experimenters turned bloggers or cutting-edge technology consultants. They do not work as staff for any University, government, or private company. For example, Alexey Guzey writes a blog and is the president of New Science, an institution created with the goal of "do[ing] to science what Silicon Valley did to entrepreneurship".

This path has a big caveat: internet fame and marketing chops can affect the chances of getting funding and getting published. But one should note that traditional research also has its fair share of popularity contests.

Entrepreneurship

An option for brave scientists is to start a business loosely or tightly related to their main area of research. Of course, they would have to control how much time this consumes so as not to be a time sink and leave enough for actual research. The most common objection to this option is that running a business leaves no time for research but, as discussed, people in Academia don't get to spend much time on it either.

Colin Percival is a PhD, computer scientist, and winner of multiple national math competitions. He runs Tarsnap, a company that provides high-reliability cloud backup services. He wrote a heartfelt article called On The Use of a Life where he explains why he decided to leave Academia:

In many ways, starting my own company has given me the sort of freedom which academics aspire to. [...] academic institutions systemically promote exactly the sort of short-term optimization of which, ironically, the private sector is often accused. Is entrepreneurship a trap? No; right now, it's one of the only ways to avoid being trapped.