The Mathematics of Paradigm Shifts

An interesting argument that scientific innovation is probably linear, and certainly not exponential - Cogntive Social Web:

The resources that we devote to science have been exploding throughout history. The world population of scientists has been doubling roughly every 50 years from 1500 to 1900 [4], and has kept an exponential rate throughout the 20th century also, by what seems to be a factor of 10 every 50 years [5] (it is however unclear whether that growth is still being sustained now [5], and even if still exponential the rate has evidently gotten slower). Likewise the number of papers and patents has followed exponential growth during the past century, unsurprisingly since average paper count per scientist is unlikely to change significantly.

Besides available researcher brains, the computing resources available to science have followed exponential growth as well (Moore’s law) in the past 50 years.

Furthermore, each new scientific discovery accelerates the pace of science to some extent, because it gives future researcher a better understanding of the field considered, as well as new tools to work with. That’s “accelerating returns”. You get to stand on the shoulders of giants.

The author also found (as we did when we looked at it) that it waxes - and wanes

And yet despite all this, if you look at the actual understanding of the world that we have achieved, if you look at real impact rather mere volume, the general impression you get is that scientific progress has really been linear. We didn’t “get” physics 50 times better in 1990 compared to 1940. The 1940-1990 progress of physics might even have been slightly less significant than the 1890-1940 one, actually. is

Even in a new, ultra-fast growing field such as computer science, most of the state of the art algorithms currently around date back from the 60s and 70s. Our computers may be 10^6 faster now than in 1960, but our knowledge is only a few times broader, our algorithm libraries only a few times better. Likewise neuroscience and AI have followed slow, noticeably linear growth since the 50s.

The question is why is this happening. .... view is its all about the peanut butter, the increasing friction in any system as it grows:

A widely-noticed empirical fact about science is that within a given field, making an impact is getting exponentially more difficult over time. Just look: the big discoveries in quantum physics had all been reaped by 1930. Most of what could be said in Newtonian physics had been said by Newton himself. Information Theory researchers will never surpass in impact the single 1948 paper that founded their field, no matter how long and fruitful their career. The list could go on.

In essence, the argument is that the first people into a new field grab all the low hanging fruit, ie the good old Pareto Principle is at work. So, as exponentially more people come in it gets exponentially harder to find, leading to a near-linear advance going forward.

So what about the rate of increase of New Fields? The author talks about these as "Paradigm Shifts" that open up the new field:

....in real life the paradigms on which scientific fields get founded tend to evolve over time, which means that the set of available discoveries that researchers can “chose from” will tend to expand over time. Paradigm shifts increase the scope of research that can be done. For instance, inventing transistors opened up the entirely new field of what to do with them. Or coming up with the basics of quantum physics. And so on.

I’d like to argue that these paradigm shifts can be modeled using the exact same model that I used for discovery. Yay for recursive models! We start with a finite pool of possible paradigm shifts, we devote exponentially increasing resources to finding them (“resources” meaning “researchers”, since each researcher tries a little to find out-of-the box ideas), and what happens is that 1) shift volume explodes over time, like papers published did earlier, because they’re a linear function of resources, and 2), the real impact at large of each shift decreases exponentially. Resulting in a linear growth of shift impact… which means in turn that the pool of available potential scientific discoveries increases merely linearly.

What interested me is that he built mathematical models of both the rate of new innovations in a field (as you'd expect, it yielded a power law of decreasing rate of returns), and of paradignm shifts. The Paradigm Shift was what really interested me (see graph at top) as it described exacly what my research showed, ie we advance by pushes forward, then pauses. I usually describe it as layers of S curves (S curves are the running totals of power law curves), so the linear model is interesting.

Anyway, for all the ambitious New Scientists and Innovators among you....

(i) Science advances linearly given exploding resources, and thus its pace will slow down in the future as the resources that we devote to it dwindle. Keep that in mind when choosing a field.

(ii) A piece of advice: as a researcher, you’ve got to work at the 2nd level of discovery, the paradigm level, because it’s so much more impact-efficient. Don’t devote your energy to discovering tiny new things within the old paradigm you inherited from your thesis advisor, rather discover broad news ideas entirely. Be imaginative.

(iii) Do not rely solely on your intelligence and hard work to make an impact on this world, or even luck, it’s not going to work. After all the total quantity of intelligence and hard work available around is millionfold what you can provide —you’re just a drop of water in the ocean. Rather use your imagination, the one thing that makes you a beautiful unique snowflake. Intelligence and hard work should be merely at the service of our imagination. Think outside of the box.

Caveat - as the author writes, "this is not a paper and I don’t have time for this. Take my model for what it’s worth: as something you’ve read on a random blog. If you’ve got a comment, or want to prove me wrong, be sure to post here or send me an email at francois.chollet@wysp.ws!"

But often a model like this is worth a million words.