Scientific discovery is popularly believed to result from the sheer genius of intellectual stars such as Darwin and Einstein. Their work is often thought to reflect their unique contributions with little or no regard to their own prior experience or to the efforts of their lesser-known predecessors. Conventional wisdom also places great weight on insight, preconception and design in promoting breakthrough scientific achievements, as if ideas spontaneously pop into one’s head – fully formed and functional.

There may be some limited truth to this view. However, as an experimental psychologist and a philosopher of science, we believe that it largely misrepresents the real nature of scientific discovery, as well as creativity and innovation in many other realms of human endeavor.

Setting aside the Darwins and Einsteins – whose monumental contributions are duly celebrated – we suggest that innovation is more a process of trial and error, where two steps forward may sometimes come with one step back, as well as one or more steps to the right or left. Instead of revolution, think evolution. This evolutionary view of human innovation undermines the notion of creative genius and recognizes the cumulative nature of scientific progress.

Wrong ideas on the path to right ones

In a recent book, one of us (ERS) discusses seven little-known scientists whose partly “wrong ideas” yielded major advances in the hands of others.

Consider one of those unheralded scientists: John Nicholson, a Cambridge University mathematical physicist working in the 1910s, when atomic theory was in an early stage of development. Nicholson postulated the existence of “proto-elements” in outer space. Using a fanciful atomic theory, Nicholson estimated the relative weights of his atoms of coronium, nebulium, proto-fluorine and so on. By combining different numbers of these alleged proto-atoms, Nicholson could recover the weights of all the elements in the then-known periodic table. On a dramatically larger scale, Nicholson could also account for astrophysical details in the Milky Way’s Orion Nebula.

These successes are all the more noteworthy given the fact that none of Nicholson’s proto-elements actually exist.

Yet, amid his often wild speculations, Nicholson also proposed that a defining aspect of atoms – what physicists call the angular momentum of their electrons – can have only certain discrete values, or quanta. Niels Bohr, the father of modern atomic theory, jumped off from this interesting idea to conceive his now-famous model of the atom.

What are we to make of this history? The central idea of angular momentum quantization sprang directly from Nicholson’s mostly off-target theorizing. One might simply conclude that science is a collective and cumulative enterprise. Ideas spread and some scientists are more adept than others at exploiting them. That may be true, but there may be a deeper insight to be gleaned.

Orderly march or random stroll?

We propose that science is constantly evolving, much as species of animals do. In biological systems, organisms may display new characteristics that result from random genetic mutations. In the same way, random mutations of ideas may help pave the way for advances in science. If mutations in either biology or science prove beneficial, then the animal or the scientific theory will continue to thrive and perhaps reproduce.

In this decidedly Darwinian scenario, there is no design, intelligent or otherwise. There is only random variation and selection, with biological or behavioral evolution unfolding in a trial-and-error fashion.

Support for this evolutionary view of behavioral innovation comes from many diverse realms of human endeavor, as one of us (EAW) has recently documented. Consider one striking example.

A particularly influential innovation can be spotted at the thoroughbred racetrack. It requires viewing horse and jockey from either the front or rear: the jockey’s left stirrup is often placed as much as a foot lower than the right. This so-called “acey-deucey” stirrup placement is believed to confer important advantages on oval tracks, where in the U.S., only left turns are encountered in counterclockwise races. Although science has yet to prove its efficacy, acey-deucey placement may permit the horse and rider to “lean” into the turn and provide the pair with greater strength by harnessing the centripetal force of a tight turn.

A relatively unknown jockey named Jackie Westrope developed acey-deucey, although it was popularized by a far more famous rider, Eddie Arcaro. Had Westrope conducted methodical investigations or examined extensive film records in a shrewd plan to outrun his rivals? Had he foreseen the speed advantage that would be conferred by riding acey-deucey? No. He suffered a leg injury which left him unable to fully bend his left knee. It was a gimpy left leg that led to Westrope’s off-kilter style – which just happened to coincide with enhanced left-hand turning performance. That’s serendipity.

What was not serendipitous was the rapid and widespread adoption of riding acey-deucey by many of Westrope’s competitors, a racing style which continues in today’s thoroughbred racing.

Variation and selection, with no end in sight

Plenty of other examples show that, in many realms of human endeavor, fresh advances can arise from error, misadventure and serendipity. Examples such as the Fosbury Flop, Post-It Notes and the Heimlich Maneuver all give lie to the claim that ingenious, designing minds are responsible for human creativity and invention. Far more mundane and mechanical forces may be at work; forces that are fundamentally connected to the laws of physics, chemistry and biology.

The notions of insight, creativity and genius are often invoked, but they remain vague and of doubtful scientific utility, especially when one considers the diverse and enduring contributions of individuals such as Plato, Leonardo da Vinci, Shakespeare, Beethoven, Picasso and Tolstoy; Galileo, Newton, Kepler, Curie, Pasteur and Edison. These notions merely label rather than explain the evolution of human innovations. We need another approach, and there is a promising candidate.

The Law of Effect was discovered by psychologist Edward Thorndike 40 years after Charles Darwin published “The Origin of Species.” This simple law holds that organisms tend to repeat successful behaviors and to refrain from performing unsuccessful ones. Just like the Law of Natural Selection, upon which evolution depends, the Law of Effect involves an entirely mechanical process of variation and selection; further, it too blindly proceeds with no end in sight.

Of course, the origin of novel ideas and behaviors demands much further study. In particular, the provenance of the raw material on which the law of effect operates is not as clearly known as that of the genetic mutations on which the law of natural selection operates. The generation of novel ideas and behaviors may not be entirely random, but constrained by prior successes and failures – of the current individual (such as Bohr) or of predecessors (such as Nicholson).

The time seems right for jettisoning the jejune notions of intelligent design and genius, and for scientifically exploring the true origins of creative behavior.

The authors do not work for, consult, own shares in or receive funding from any company or organization that would benefit from this article, and have disclosed no relevant affiliations beyond the academic appointment above.