The Seeing Tongue. New Aspects of Exo-Darwinism

»Mais ce qui est vrai pour les fonctions pures du corps,—c’est-à-dire le marteau, la roue, etc.—est aussi vrai pour les fonctions intellectuelles, et en effet, vous voyez bien que la mémoire s’est objectivée dans l’écriture, dans l’imprimerie, dans l’informatique […]. Le corps perd en effet—mais il perd ces objets-là, qui deviennent le support d’une évolution qu’on appelle évolution technique, évolution scientifique etc. J’appelle ça l’exo-darwinisme.«–Michel Serres

The design of the human began in a rather harmless way, namely in the notion and practice of »synesthesia,« a term dating back to the nineteenth century. Here, a reprogramming of the sense organs had already taken place, which only became more exacerbated in the course of a century leading up to »exo-darwinism,« a term that was coined by the French philosopher Michel Serres. This development can be illustrated with a thesis, formulated in an essay in 1956 by the media theorist Marshall McLuhan: »Each new technology is the reprogramming of sensory life,« [1] meaning that the relationship of our sense organs to each other, as well as the relationship of the sense organs to the environment, is constantly reprogrammed by new technologies and, consequently, our sensory life rewritten. This in turn is most often interpreted as the »terrible assault« of the human race on his own programming. Contrary to this, my argument will be that this rewriting is nothing other than the logical consequence of this development of the reprogramming mentioned above.

As the composition of the word »synesthesia« from syn (together) and aesthesis (perception) already indicates, the term turns on the reciprocal perception of two different sensory stimuli. In other words, the linking of otherwise separate domains of perception; on the one hand, the sense organ of the ear, which receives sound, on the other hand, the sense organ of the eye, which receives color and light. Hence, in the present century, synesthesia signifies that which has nowadays also been experimentally proven, namely, that in the perception of certain sensory stimuli by one organ, other sense organs are also stimulated. In terms of neuroscience we know, today, that this does not concern single organs, but areas of the brain. More precisely, when sensory stimuli penetrate the brain in a certain area, in a certain surface of the brain by way of the ear, then stimuli are felt which also arouse neighboring fields of the brain. In other words, synesthesia is not a matter of the organs, but a matter of the brain. And in fact, not only is a particular area aroused in the brain by certain sense organs, but so also the neighboring areas stimulated by a single sense organ. Consequently, when optical stimuli penetrate the brain so as to stimulate a particular area there, then other neighboring areas, responsible for tactile feelings or acoustic stimuli, for example, are also aroused. This discovery of the transfer from stimuli to nerves had already been made at the beginning of the century, namely in 1866, by French neurophysiologist Alfred Vulpian. Similarly, the concept of color hearing, in French, audition colorée, is based on this observation. By about the mid-1920s in Germany, Georg Anschütz amongst others put synesthesia research on a scientific footing. One further example would be the book entitled Das Phänomen der Synopsie, written by French scientist Théodore Flournoy and published in 1893, in which, in addition to nature, he closely observes sensory origins and intensities of synopsia. This knowledge, which can today be verified through neurophysiological findings, functions as the basis for Serres’ explication of exo-darwinism.

In the 1990s neuroscientists managed to undertake precise observations of the processes that occur within the brain by means of computer-aided machines called functional magnetic resonance imaging. The neuroscientist Richard E. Cytowic postulated that from birth onwards, all human beings dispose over nerve connections between that sensory system responsible for processing the initial stimulus, and those in which additional sensory impressions subsequently emerge. The implication is that the capacity for synesthesia is innate and not learnable. What is of interest here is that by means of these studies Cytowic discovered that actually everyone, from birth on, disposes over nerve connections between the sensory systems. Hence, essentially, all infants possess connecting lines between the areas, between diverse nervous systems, whether the eye, or the ear. Unfortunately, after a period of three months these nerve pathways begin to degenerate and slowly disappear. In the case of some human beings, however, these connections remain, thus not only facilitating the visual but also the auditory perception of colors and vice versa, that is, the capacity to see sounds, as well as hear them. Moreover, they are also capable of developing an olfactory sense: they are not only able to precisely determine and identify the smell of objects, but also to some extent, perceive and process other sensory stimuli. [2] One may conclude, therefore, that it is normal for all human beings to grow nerve pathways that erect a bridge between the nervous systems. In reality, the paranormal is not an ability which a person retains once having become an adult, but the paranormal is rather the fact that one loses it. The question, which now poses itself, is therefore why we are not concerned that this ability, with which every human being is born, does not degenerate, but instead enhance this natural capability.

To further explicate this continuation of the reprogramming of our sensory life, one can even draw on Isaac Newton, who pointed out in his famous work Lectiones opticae of 1666, that the frequency of sound waves are comparable with the frequency of light waves. To this end, he drew up a complicated chart of coefficients by means of which he was able to increase the frequencies of sounds up to the unevenly higher number of frequencies of light sources. Thus, theoretically, he was able to increase the frequency of sound above that of light, and thereby transform sound impulses into light impulses. In this case, all that was missing was the technology; the theory already existed. His student, Louis Bertrand Castel, had produced the famous Castelian color piano from these considerations towards the end of the seventeenth century. There are countless examples of how the aesthetics of synesthesia, the aesthetics of color and sound, was celebrated in art and in music, from the poem Vocals (1870) by Arthur Rimbaud to Alexander Scriabin’s Promethée. Le Poème du feu (1911) or the work of Josef Matthias Hauer, the inventor of twelve-tone music, or the work of painters like Enrico Prampolini or Francis Picabia’s La musique est comme la peinture (1917).

Another well-known pioneer of synesthesia, who introduced the term “visual music,” was Mary Ellen Bute who published the work Visual Music Synchronized in Abstract Film by Expanding Cinema in 1936, from which expanding cinema and the famous slogan, seeing sound, were to develop later.

Thus, if we admit that it is, indeed, possible that the stimulus in a given brain area co-activates neighboring brain areas, then one is able to see the process and the model of natural evolution in another light for the first time. This is because this insight prompts the idea that one is able to program this intervention in sensory life, namely, to steer this connection with the aid of external machines or media.

In 1872, about the same time as the publication of the first article on synesthesia, this notion of machines from a Darwinian standpoint was described in greater detail by the famous English writer and scholar Samuel Butler in his novel Erewohn, or, more precisely, in the chapter entitled The Book of the Machines. His idea was indeed so radically new and such a provocation to Darwinian thought that he was forced to publish the book under the pseudonym Zelavius. Here, for the first time, evolution was shifted from nature and transferred to machines, and the concepts of natural evolution were transferred to the mechanical world; “we find ourselves almost awestruck at the vast development of the mechanical world, at the gigantic strides with which it has advanced in comparison with the slow progress of the animal and vegetable kingdom.”[3] In other words, the point here is that natural evolution progresses too slowly, when compared to the world of the machines, which develops at a far more rapid rate:

We shall find it impossible to refrain from asking ourselves what the end of this mighty movement is to be […]; the machines are gaining ground upon us; day by day we are becoming more subservient to them; more men are daily bound down as slaves to tend them, more men are daily devoting the energies of their whole lives to development of mechanical life. [3]

This is the first time a mention has been made to »mechanical life,« namely, that one can transfer the laws of evolution to that of machines. That is to say we are able to envision a mechanical, mechanistic, and artificial evolution as a model.

Georg Dyson, the son of the celebrated physicist Dyson, further developed this idea in his book Darwin Among the Machines with the subheading The Evolution of Global Intelligence by citing Butler: »Is it the man’s eyes or is it the big seeing engine which has revealed to us the accidents of worlds beyond worlds into infinity?« [4] Were these our natural eyes as developed in evolution, or was it the seeing machines we developed, which facilitated the gaze into the hitherto unseen world? We are able to see most of what we see today only by virtue of machines. Our natural eyes have long-since been superseded, since we see everything through cameras, through satellites. In short, machines do indeed provide us with a world, which, with our natural eyes, we would have otherwise been unable to see. The question that consequently presents itself is the following: «May not man himself become a sort of parasite upon the machines?« [5]

After lengthy detours, such considerations—namely, the idea that natural evolution, as well as the evolution of our organs, will be shifted to and superseded by machines—have caused neurology also to adopt this theme. The 1972 winner of the Nobel Prize for Medicine, Gerald Maurice Edelman, published the book Neural Darwinism[^](Gerald Maurice Edelman: Neural Darwinism: The Theory Of Neuronal Group Selection. New York 1987.) in 1987, in which he, to some extent, transferred the idea of Darwinism to the neuronal net. In doing so, he saw that the categorization of various sensory stimuli—always due to our sense organs, which determine human behavior—consistently shows itself as a dynamic process of re-categorization. In short, if one were to draw a map over the brain area, then the claim, namely, that this and that area is responsible for this and that, and that, in certain places certain stimuli would be processed, would be invalid: one would then recognize that these single fields are incomplete, and are indeed stimulated by other fields. In his book Edelman even advanced the thesis that the brain is itself a dynamic process in which, in turn, the map of the brain perpetually invents itself anew. In other words, a dynamic process of re-categorization repeatedly occurs, and those parts that carry out this process to the optimum, survive. Thus, Edelman had, indeed, transferred the notion of Darwinism in the competition of the neuronal networks to the human brain.

Around this time, roughly at the beginning of the 1980s, computer scientists had similar ideas, such as the researcher Daniel Hillis, and developed so-called »connection machines.« These were gigantic calculating machines containing thousands of programs competing in a sort of evolutionary process, whereby the program providing the optimum solution asserted its superiority. This experience of networking computers was, in turn, taken up by neurologists under the term »connectivism.« The assumption here is that the neuronal networks, as Edelman understands them, are not connected to historically acquired capacities, but that the neuronal networks rewrite themselves. This re-programming of our sensory life, therefore, is something that the brain has long-since achieved; it reprograms our sensory life, as well as the relationships of our sense organs and sensory stimuli to one another. It is, in a way, the mode of operation of the human brain. It was this dialogue between neuroscientists and computer scientists which in turn led Terry Sejnowski to build his famous NETtalk computer, a speaking computer composed of networks, which made it possible for the first time to allow a computer read a written text out loud. Sejnowski was thereby capable of transferring the sensory perception of the eye, which is able to read, to the mouth, so that it was able to speak the text that it read as well. This is exactly what is meant by synesthetic experience: the transfer of information from a net to another net. The network of the brain, which is responsible for reading, transfers the information to the network that is responsible for speech. The computer already succeeded in accomplishing this task. These cognitive abilities of human beings have already been exterritorialized into machines and computers. This is what is meant by the term exo-darwinism. My name for this phenomenon of exteriorization of human faculties into machines and systems is »exo-evolution.«

Thus, the history of the sensory substitution began with synesthesia, further developing itself with the aid of machines, for example calculating machines. In this case a step has been made from the biological Darwinism of living beings to the Darwinism of programs and thus the evolution was outsourced in programs and machines. The continuation of this development was achieved with the development of the Josephson Junction with which it became possible to measure extremely small oscillations in magnetic fields. This discovery was based on an essay on the theme of superconductivity written by the then 23-year old physicist Brian David Josephson, who went on to be awarded with a Nobel Prize in 1973. The theory of superconductivity was later verified in experiments and Josephson transferred the results to brain research. Thanks to certain insights into the Josephson Junction, he discovered the ideal instrument for carrying out electromagnetic measurements.

In other words, these extremely fine measuring instruments for magnetic oscillations in the brain were capable of contributing in establishing a contact between natural nerve cells and artificial nerve cells—hence the name »junction,« the bridge. The functional transfer from natural to artificial nerve cells would normally not work, but once fine-tuned the communication between artificial and natural nerve cells was possible. The thesis of sensory substitution Josephson further pursued by way of an experiment with a Chinese girl in whom the ability to establish connections between different brain areas was still intact. To this end, he constructed a wall, placed the girl behind it and under observation wrote the word »Wittgenstein« on a small piece of paper. He then crossed out the word »Wittgenstein« before writing it out once again. He requested that the girl chew the text before writing out the content of what she had on her tongue. The result of this experiment was that the girl wrote the name Wittgenstein, precisely in the way Josephson had written it, namely, crossed out on a sheet of paper.

This work, which Josephson carried out as a certain supposition, has since become reality. One example of this realization is the work of North American neurophysiologist Paul Bach-y-Rita, who, above all, focused on the question of rehabilitation after the brain had been damaged. Based on the theoretical assumption of the neuroplasticity of the brain—by which one understands the capacity of synapses, nerve cells or entire brain areas, to change their functions—in 1969 he presented ideas for a machine which would translate the pictures of a camera into vibrational signals; these would then be capable of aiding blind persons in orienting themselves. This instrument, also named Brainport, was developed by Bach-y-Rita in 1998. A small computer transferred the signals of a video camera on top of glasses onto a small grid that is placed in the mouth, conveys electronic impulses and thus excites the 450 point resolution field of the tongue and thereby creates an image. Thus, it became possible for blind persons to see (so to speak) with the tongue.

In short, enormous developments have been made in the area of reprogramming sensory life in the processing of data by the brain. In his essay »What Do Brodmann Areas Do?«—Brodmann areas are areas of cerebral cortex in the brain—Dan Lloyd, a famous neuroscientist at Trinity College Hartford, even went so far as to enquire whether the sum of actions in various brain areas behave like a »neuro-democracy«—a thesis opposed to the assumption that the processing of certain sensory stimuli can only be allocated to certain areas of the brain, namely, to a ‘neuro-bureaucracy.« [6] Lloyd was also able to verify this thesis of a multiple wiring of functions of the brain areas by way of several tests.

Consequently, synesthesia has today transformed itself into the consciousness that we live in neural networks, which do, indeed, repeatedly reprogram themselves and which facilitate the transference of one sensory experience to the other. We not only live with one brain, we live with thousands and thousands of brains, since the brain perpetually rewrites itself. This experience of the neurosciences, together with the computer sciences, had already been previously made by philosophers. One of the first was the French Gilbert Simondon, who in 1958 wrote about the ways of existence of technical objects. [7] Before the appearance of technology the relationship between human beings and their surroundings, the environment, nature, was a binary function. There only was a dialogue between the natural, nature-endowed body and the equally naturally provided environment: a binary relationship. Technology was, in a way, a tertium comparationes, a third medium interceding between the environment and humankind. The main thesis of Simondon therefore states that through the development of technology the relationship between human beings and their environment, between man and nature, became a trifold relationship, and no longer a binary relation. That is to say, there is a mediator between the organism and the environment. The word »medium« is a term that was introduced by the experimental psychologist Fritz Heider in the 1920s in his essay »Ding und Medium« in which he expounds the notion that something new is standing between humankind and its environment. If one accepts this idea of a threefold relation between humankind, environment and technology then there are only two alternatives: one can look on how the natural environment and mankind advance, slowly as before; or, one can try to transfer the evolutionary approach into technology. That is, if a man should lose his hand, science could try to build a prosthesis that it could develop even further. Another French philosopher who took up this specific problematic was André Leroi-Gourhan in his book Le geste et la parole in 1964 in an essay about the fate of the hand. [8] With the advancement of civilization the importance of the fate of this organ decelerated. He cites the very well known sentence that would have been unimaginable 1,000 years ago, that one does not know what to do with his hands. This is a warning sign so to say, here the apocalypse is standing directly in front of us: humankind does not know what to do with its hands, that it only knows how to think. There is a tendency to forge links between the analysis of civilization to the analysis of culture saying that when the hand loses its importance, that craftsmanship loses its importance as it is carried out by machines, with media, and no longer with the hand, that this de-manualized art becomes a sign of decay.

Contrary to this position, technical existence, like the philosopher Max Bense once said at approximately the same time as Simondon, technical objects are the beginnings of exo-darwinism. Leroi-Gourhan has explicitly said that technology has to be interpreted as an exo-skeleton, as an instrument that advances with one’s natural sensory organs. If a physical organ is no longer capable of sufficiently mediating between the body and the environment, then it is possible to transform this organ by the creation of exterior organs. Hence, this harmless thesis of synesthesia not only consistently led to the possibility of sense organs being newly programmable and capable of being rewritten, but also that they are replaceable. In other words, if we work on the assumption that one sense organ, at least partially, is capable of also stimulating other sense organs and brain areas, then one might also introduce the assumption that it can, at least to some extent, take on their tasks. If a given area of the brain can, in part, take on the function of another area of the brain or, simply put, if one organ is capable of partially adopting the function of other organs, then we may conclude that the entire organ may take on the function of the other organ. This means that an entire brain area, which functions according to a dynamic neuronal modal, is capable of adopting the function of another brain area to supplement it. The tongue is thus capable of adopting the function of the eye and the eye the function of the tongue and so on—not, however, in terms of a natural evolution, but only with the aid of technology in the process of exo-evolution.

Published first in 2009. More about this topic in the publication Design of the In/Human.

  1. Jump Up Marshall McLuhan, David Carson: The Book of Probes. Corte Madera 2003, pp. 162 f.
  2. Jump Up See Richard E. Cytowic: Synesthesia; A Union of the Senses. Cambridge, MA 2002, pp. 242–246.
  3. Jump Up Ibid.,pp. 180, 184 f.
  4. Jump Up Idem: Erewohn: or, Over the Range. London 1872, p. 157.
  5. Jump Up Ibid.
  6. Jump Up Dan Lloyd: Waht Do Brodmann Areas Do? Or: Scanning the Neurocracy. Status: 12/04/2011, http://www.trincoll.edu/~dlloyd/brodmann.html.
  7. Jump Up See Gilbert Simondon: Du mode d’existence des objets techniques. Paris 1958.
  8. Jump Up André Leroi-Gourhan: »The Fate of the Hand,« in: idem: Gesture and Speech. Cambridge, MA 1993, pp. 254 f.