Is human language underpinned by gestures?
In this Psychology Report, I am interviewing renowned psychologist Emeritus Professor Michael Corballis. He has recently won the Rutherford Medal from the Royal Society of New Zealand for his work on asymmetry of the brain, but also his work on language, and that's what I'm focusing on today in our interview. One view of language is that it emerged as a single big bang event in one of our ancestors and from that point on the linguistic capability and ability for the human species somehow exploded from that point. Another idea, of which Michael is a keen supporter of, is that somehow gesture and our capacity to position ourselves and move in spacial habitats underpins language, but also this idea of mental time travel.
M Corballis: Well, I guess it began really with an interest in brain asymmetry and handedness, and sort of relation between the two. Of course, the brain is asymmetrical in two striking ways, one is the representation of language in the left hemisphere, and the other is handedness, and I guess that led me to the idea that there ought to be a link, and that maybe handedness holds the key to where language came from, and that led me to the idea of the gestural theory of the origins of language. So, language begins in the hand, not the mouth, so to speak.
Sarb Johal: Yes, because there are competing theories around where language comes from…
MC: Well yes, I think most people take it for granted that language probably evolved from animal calls. In other words it was vocal to begin with. So I then began to develop the alternative idea that really it probably started with manual gesture, and that vocalisation came later, and I think the opinion is probably about even now, there are still people who insist to me that language must have come out of animal calls. So I've been trying to build a case for gesture for some time now.
SJ: And even within the idea that language came from animal calls, there's even some difference there around how this may have come about, whether this was gradual evolutionary process, or if there was almost some kind of big event that led to this sudden development of language.
MC: Oh, absolutely, I also find myself trying to counter that Chomsky notion, in fact the Biblical notion, if you like, that language was suddenly bestowed on us as a kind of gift, so that there are no precursors to language, language sort of happened. And even Chomsky's quite explicit about that. He's got the idea that language appeared suddenly within the last 100,000 years, as a singular event, even in a singular person. The Bible calls that person Adam, but Chomsky, tongue in cheek, perhaps, calls that person Prometheus, as there was suddenly a bestowal of language on the single individual. So partly my message is Darwinian, trying to argue that language probably evolved gradually, and the gestural argument is part of that, part of the argument that it was a gradual evolution, not a sudden emergence.
SJ: So if we just focus in on this part of the story at a neuropsychological level, and how the hippocampus is really critical in understanding space and our relationship to space. How does the hippocampus fit into this idea that there may be a gestural underpinning to what this vocal language that we engage in?
MC: Well, really the mental time travel argument, the hippocampal argument is a little bit separate. The idea that what language evolved for, whether it's vocal language or gestural language, language evolved to be able to communicate about the non-present. What happened yesterday, or what might happen tomorrow, or even things that are in your head, experiences. So that's a somewhat separate argument, I think, the hippocampal argument. It's known for a long time that the hippocampus is a cognitive map, so there was a famous book written I think in the 1980's called "The Hippocampus as a Cognitive Map", so if you do, even in the rat, if you take recordings from single cells in hippocampus, they seem to represent where the animal is located in space. Okay, so it's almost like a GPS system, the hippocampus, so it also transpired that if you take an animal out of a maze, it keeps on giving recordings for locations where it has been, and even where it might go, so I think it's now understood the hippocampus is kind of mapping out past trajectories in space and future ones, and that I think is in a way, the basis of mental time travel in humans. So, we also know from hippocampal recordings in humans, through fMRI, that when we think about a past event or a future event, the hippocampus lights up in very specific ways. So that has to do with the role of language in ... Well this provides the basis, if you like, for experiences that are not involved in the present, and language has emerged in order for us to tell other people about that. Or people about travels in Europe, or whatever.
SJ: So really, it's back to the common sense idea that language evolved from a basic need to communicate, and that communication is about, perhaps, where we may have found food in the past, or where it is that we may want to go in the future, depending upon our experiences and our plans for the future, enables us, as you say, this time travel.
MC: Absolutely, and I think that's what language is really all about. Communicating about things that are not present. If you want to communicate about things in the present, you can point to them, and share experiences, but once you get away from the present, then you need a system of communication, to tell people what you did yesterday, or what you're going to do tomorrow. And I think that probably emerged in a gestural form of communication, rather than a vocal one, because vocalisations not very good at that, if you think about it. Vocalisation doesn't really easily address locations in space, or things that happened at different points in time, whereas gesture and the use of space, through gesture, is a more obvious way to do it.
SJ: So what are the sorts of things that are a prerequisite, then, in order to be able to communicate through gesture, our experiences of the past, and then the future? If language, as in vocalised language doesn't necessarily seem to be a necessary format, that can be achieved through gesture, what other things would have to be in place for that to be effective, for that to work?
MC: Well, I think just having experience of space is one. That's the first requisite, because that's what you're trying to communicate about. And I think the ability to construct space in your head and then use gesture to communicate is a natural way to go about it. Sign language is a good illustration of what is necessary. People using sign language actually construct kind of virtual spaces in front of them where they locate objects and then point to them in a virtual fashion, right? So I think that sort of system grew naturally from the understanding of space itself, and the use of gesture to address that space, and then you can begin to tell stories by constructing virtual objects in front of you that the person that you're communicating with can see. So, just the use of space, the understanding of space, and the role of gesture and manipulation in space kind of puts the story together.
SJ: And as you're talking, I'm put in mind of children, and in their development, and how they learn to communicate quite early on through gesture if you're sensitive enough to pick it up. Some of them are quite obvious, but some of them can be quite sensitive, you'd have to be quite sensitive to pick it up. What are your thoughts around that, because this is often way pre-verbal, any kind of articulation of language that children are able to communicate in this way.
MC: Yeah, I think development kind of maps on to evolution, kids naturally begin by communicating with pointing. Young kids ... I think they understand language through pointing before they understand it through words, so they will point to things to communicate their interest in something, and that sort of precedes the actual naming of things with words. And of course, kids with sign language, kids brought up in a sign language environment actually babble in gesture, so it's much more natural, and earlier, I think, to communicate through pointing and using your body than it is through using sound.
SJ: So one of the other things that we think about with children as well, and some of the things that you've talked about, is this idea or theory of mind, so in order to be able to effectively communicate with someone, we have to have some idea as to what's going on in their minds as well.
MC: That's absolutely critical, I think. Language is really ... People say it's under represented. Language is hopeless really, it doesn't work unless you know what another person is thinking. So really what you're doing is elaborating on their own thoughts by giving hints. So, language is very seldom fully explicit. It's underdetermined, so I think you've got to know what's going on in the other person's mind in order to even generate a meaningful sentence that they will understand. I think it's probably easier with respect to gesture, because then you are directly accessing space, but even when we're speaking, I think we have to have an intuitive understanding of what's in the other person's mind. Like I can say to you, "Hey, that was a great game yesterday, wasn't it?" I'm talking about cricket, in case you don't watch it.
SJ: I'm still thinking, is it the women's cricket, or the men's cricket he's talking about?
MC: Well, I was actually watching the men's cricket, I'm ashamed to say. But we normally talk in a kind of telegraphese, we don't say explicitly everything we want to communicate about, so I'm just going to say that's a great game, and what's going through your mind, I think, is images of that game, so we've gotta have this communal imagery, this communal understanding that lies behind what we're trying to communicate. So, language itself just scratches the surface.
SJ: Yes, and there's an interesting example that you've written about and quoted other people, where people who know each other quite well can just say a few words, and there is an implicit awakening of a shared understanding comes up with those words.
MC: Exactly. Yeah, a couple of grunts will do it.
SJ: So how does gesture fit into that? How does it underpin that, or how do we think differently about how gesture might achieve that?
MC: Well I think simply that gesture does it more naturally, and more obviously somehow. A shrug, or a raise of the eyebrow, or whatever, all of these things are kind of ... They're almost like sentences, but they're more direct. So somehow you've got to understand the words of a language in order to use spoken language to do these things, but gesture's more natural, isn't it? Although different cultures gesture it somewhat differently, but I think there's an awful lot of shared gesture that simply illustrates the naturalness of gesture as distinct from the naturalness of vocalisation.
SJ: Because, I guess, you have the many different languages that are spoken, where some of them share common roots, and many of them do not, and I guess part of that is because of this regional and physical separation, which meant that languages developed differently; but partly there are reasons why languages are quite different from each other as well, and it's interesting that you're saying that perhaps there is more of a shared common core within a gestural set across the human species.
MC: Yeah, I think so. At one time it was thought that all sign languages were the same. There was just one sign language. That's nonsense, too. Sign languages differ, but people who use different sign languages communicate much better than people who use different spoken languages. There's something like 6,000 different spoken languages, and most of them are mutually incomprehensible. Whereas people who have different sign languages communicate moderately well, and we all resort to sign language anyway, if we're in a country where they don't speak the language that you speak. My favourite story is trying to ask for a bottle opener in Russia. The only way I could do that was mimicking the idea of opening a bottle with a bottle opener and pretending to glug it down, they understood that instantly, but I'd never been able to get the idea across using any kind of spoken language.
SJ: And as you're speaking, I sat there thinking about Washoe, the primate. Was Washoe a chimpanzee? I seem to remember Washoe was a chimpanzee, communicating...
MC: Washoe was, yeah.
SJ: Yeah, communicating in this kind of sign language, or pointing to symbols that they'd been taught to use, in what way does that map on to our human experience and understanding? How much do we know about that?
MC: Well I think the language that was taught to Kansi in particular, and Washoe, I think, was based in part on sign language, on the grounds that that was sort of a natural way to communicate. They of course, early on, had tried to teach chimps to speaks, but that got nowhere at all, and I supposed, I'm not quite sure I can answer that question very well, they also use symbols on a kind of giant iPad kind of thing with Kansi, and communicate by pointing to the symbols. So, it's sort of natural to point to things, point to things to draw attention to them, and then use them as communicative symbols.
SJ: And did they show evidence that they could mentally time travel as well, in the way that they used those symbols, pointing at symbols and also their gestures?
MC: Yes, I think this ... It's not very good evidence, I don't think, but sometimes Kansi ... Kansi is the champion, I think. He was a Bonobo, actually, not a chimp, but he will sort of point to things and then lead the keeper as it were, to somewhere else in space where something is hidden, so that suggests that when he points to the symbol representing what it is that's hidden, he has in mind having hidden it a day earlier, or something like that. So there are, they're a little bit anecdotal I think, cases of communication by chimps using gesture, that refer to something that happened a day before, or something that's about to happen. "I'm about to take you to where I know something is."
SJ: Yes. I guess one of the things I'm thinking about, the difference, perhaps, between those gestures that primates and humans may make, and language, is that perhaps people think of, or conceptualise language as having many different building blocks that we can construct in many different ways to produce complex ideas and long strings of narrative that come together to enable us to tell stories, and through these stories, we communicate who we are, where we've been, who we might be in the future, and how we relate to others. How does that idea that language is unique in enabling us to do that, verbal utterance of language, how does that map on to the idea that perhaps gestures can do this, too, or perhaps gestures were the genesis of this?
MC: Well. I think the crunch question here is how much stories depend on language, and how much stories depend on what is being communicated. It may be that chimps, and even rats have stories in their heads, but don't have the ability to communicate them, so one of the precursors then, is having this mental time travel, this record in your head of things that have happened. This is where the hippocampal recordings come in, it looks like the rat can replay in its hippocampus a trajectory that it undertook in a maze a short while ago. And that in a sense is a kind of primitive story, and it's also known from some of the rat experiments, amazingly, hippocampal experiments, that these locations are also associated with smells or with simple happenings. So in a way, that's a precursor to a story, but it's a story in the head, it's not a story that's told, so at some point in evolution, I think it became adaptive not just to have records of the past or plans for the future, or things that you're gonna do, or things that you're going to relive in your mind. It became adaptive to communicate them. So, I think stories are part what's happening in your mind, but they're also part the communication of that. Now the Chompskian view of language is you need ... The generative aspect, it comes from language itself. What I'm trying to argue is that the generative language is there as a precursor to language. The generative aspect is the experience itself, the memories, the plans, and all the rest of it. So, you've got the stuff in your head, you want to tell somebody else about it, I think the natural way to do it was probably through gesture, because then you can link directly to space by the use of pointing and by the use of mimicking shapes with your hands, and movements of your hands and so forth. Okay, am I being clear?
SJ: I think so. I think taking on from your ideas, and correct me if I'm wrong, is that you have these gestures that you put forward, and somehow, through need or process of becoming familiar, that these gestures became minimised or shrunken, so you didn't have to go through the whole gesture, because people were familiar enough with it, "Okay, if you do that, then you mean all of this instead."
MC: Right. Gestures in a way are more elaborate than they need to be, probably, and if you're going to ... To mime a story is quite an effort, it takes energy, it takes movement, it's probably inefficient, so if you can cut it down, minimise it, you can still create the story by using much simpler devices like spoken words. So, you have to have a system whereby you can map your communicative device onto the story you're trying to tell, but the important thing is the story itself, that's where all the generativity is, that's where all the complexity is, and so forth. And that's where language itself is quite complex, but the complexity really derives from the story, and not from the language, that's where I think I part company from Chomsky.
SJ: Okay, so the story then becomes the motivator to communicate, because I've got something to communicate about, and then the vocal utterance, the language, becomes a shortcut for the extended mime that tells the story of my physical relationship to this experience that I've had.
MC: Exactly. So you're using these funny little things called words and sentences to try and get the other person's mind moving in the same direction as your own.
SJ: Right, right. So, as we're talking, I don't know why, but Donald Trump is coming to my head, and I'm thinking about his gestural use, and his use of language, and this idea that actually he has a very, how shall we say, unique take on events and things that may have happened in his recounting, and he is very noticeable in his gestural use when you see him talk, he has particular ways of being in his physical self while he's talking. What do we know about how our non-verbal communication maps onto our verbal communication, given your point of view? Both of these can be happening at the same time.
MC: Yeah, I'm never that convinced of the sharp distinction between verbal and non-verbal. Because people gesture all the time, and they speak anyway, and I think it's part of the language itself, so we make use of shrugs and pointings, and waving our arms around, which is actually part of the communication system. I don't think there's a sharp distinction between what's verbal and what's non-verbal. Perhaps to some extent, gestures add emotion that's not part of the language, but I think the fact that we gesture all the time as we're talking derives from the fact that language probably was gesture initially, in the first place.
SJ: It's interesting how we socialise our children, and often you hear people saying things like use your words.
MC: Yeah, don't point.
SJ: Yes, yes.
MC: Some cultures disallow pointing, I think, it's very rude to point.
SJ: That's right, and you use a different sort of a hand gesture instead.
MC: So somehow we're being taught, perhaps in different cultures in different degrees, taught to suppress their pointings and their movements, and use words, but I think in a way, the pointing is natural. And at some point in evolution, was probably more pointing than words, then vocalisations.
SJ: So if it makes ... If it doesn't make sense, we need to separate out this idea of this division between non-verbal and verbal behaviour, actually they're all part of the same system, and you need to understand people, you look at that in a coherent way, a cohesive way. What's the point of this line of thought? Who should care about this, Michael? When we're thinking about the implications of what you're saying, how does this move us forward in different ways? How should we be thinking about communicating with people, with each other?
MC: Now that's a trick question. I don't know, I think simply recognising the importance of gesture and the total communication system is important, whether it's important to consider whether we should suppress it or not. Our culture perhaps does suppress the kind of gestural components. There are big cultural differences here for a start. The Italians are famous for gesturing as they speak, so gesturing's a very important part of the culture in Naples. There's something about buttoned up English culture that sort of suppresses gesture, but I don't know that I have a moral position on this in particular, it's just there, isn't it? It's something that some cultures do and some don't.
SJ: It is, it is just there, it's an interesting ... It would be an interesting question to understand a little bit more about why some tended to ... I don't know if suppress is the right word, but discourage gestures as opposed to language, or whether one is seen as superior to the other.
MC: Yeah, I think we're taught to think that spoken language is superior and the mark of sophistication and all the rest of it. I think sign language, for example, for a very long time got a very bad rap, as though it was somehow crude and it was not proper language. One of the things that's come out over the last 50 years or so is that sign language is real, it's a real language. It has grammar, it has sophistication, and it can be quite beautiful. So I think this partly, a kind of prejudice against gesture, as if people who gesture are not using language properly. I think they probably are, and I think we have to try to understand language in that broader context. Gestures are a natural part of spoken language, of our own language, but it's also all of sign language, so I think just the recognition that this is what language is, rather than some sort of extremely complex high level thing that's purely spoken and just uses spoken words. So, I think that's part of the deal, trying to see language as broader than just the rather pretentious thing that linguists have built up. I got a bit of a thing about linguists, maybe. There's a sort of snobbery associated with speech, as though embellishing it with gestures is kind of crude, and unsophisticated.
SJ: … I was just thinking about where we started off this conversation.
MC: Yeah, goodness knows where all that came from.
SJ: No, I think it's been a really fascinating conversation, a little bit wide-ranging, but I'll just bring you back to the idea that you were saying that perhaps it's about 50/50 at the moment, as to the degree of support as to whether people think that we have this gestural basic underpinning of where language comes from, or as a base of language, as opposed to this kind of big bang, one particular moment in the last 100,000 years, where this one individual came along that was able to do this, and all languages come from that. Where is this debate at the moment? In what direction is it moving?
MC: One gets a bit biased by the people you talk to. I go to conferences, and I do the stuff, and I get ... Probably most people are reasonably sympathetic, but then I realise there's a bunch of hard-core people out there who won't buy it at all. One of the problems is the notion that switching from a visual mode to an auditory one just seems too extreme. Going from sight to sound just doesn't seem plausible, so that's probably the most compelling argument, that somehow, in the course of evolution we shifted from a visual modality to an auditory one, so some people simply can't buy that in neurophysiological or neurological terms. So, then I have to argue that in fact there's a very close connection between what we do with our hands and what we do with our mouths. So part of my argument is that speaking itself is gestural. What we are doing, we're sort of making gestures with our mouths, with our larynxes, with our tongues, and there's a natural connection between the hand and the mouth anyway, it comes from eating, among other things. So, really it's not so much a switch from vision to audition, from sight to sound, as it is a blending from one form of gesture to another. Gesture that was with the hands and face, but then increasingly became the face and less the hands. That's one of the critical questions, I think. Why would language shift so radically from the visual system to an auditory one, and so that's one of the sore snags in the gestural argument.
SJ: Speaking as a clinician, and I guess this is why I thought it would be good to talk to you, is that this resonates with me, and that I remember as a young learner ... Well not so young, but as a learner clinician, one of the things that I got fed back on was that I tended to use the whole of my body to communicate, and that I was, when I was in a clinical situation, I was somehow switching this off, and I was becoming very verbal, and this was affecting my effectiveness as a clinician. Some of my supervisors who would sit in with me would notice this. So there was some kind of value, particularly when we're talking about talking therapies that is given just to talking, and I was starting to change who I was because I was prioritising talking, whereas actually, if I allowed myself to bring a bit more of the whole of me to any kind of interaction in the clinical setting, then I became more effective. So I wonder how much I do screen myself, and be very matter about it and stick to language, whereas actually I might be much more of an effective communicator if I let go of that a little bit.
MC: Yeah, there's something a little bit buttoned up about speech, isn't there, really? It happens in the mouth, it's all ...I like to think of it ... Speech is minimalist. It's miniaturisation, so it buttons you up, alright. Whereas if you use gesture along with your speech, which is perfectly natural, you're revealing more, you're perhaps getting at more, aren't you? People talk about this as non-verbal communication, as though it's separate, and you should watch what people do, but I think it's part of the communication process in general. So, by just restricting yourself to speech, you're kind of buttoned up and slightly closed down.
SJ: Yeah, and I think that probably also has implications to how we hear other people, too, by only hearing the vocalisation, rather than attending to the whole, rather than, as you say, I'm not bifurcating this and that. I'm looking at the whole and experiencing a person as the whole, rather than just their speech, or just their gestures. Then I'm getting a different picture.
MC: There are a number of people, by the way, who kind of argue that gesture and speech were there all the time. My argument is that it began with manual gesture, and became increasingly miniaturised and located in the mouth, but there are other who argue quite plausibly, I think sometimes, that it always was this way. In other words, there was vocalisation and gesture, and they belong naturally together. That's one of the arguments I sometimes get, that it's not gesture first at all, it's that gesture was always there and always will be.
SJ: And what's their premise for that argument, that they both existed at the same time, they always have?
MC: It's not a very good argument in some sense, I think they base it primarily on studies of people gesturing while they're speaking, and the close connection in terms of timing between the way the hands move and the way vocal utterances are made. I don't think they really look back at what chimpanzee communication is like, because clearly in a chimpanzee you can't teach a chimp to talk, but you can teach it to gesture. And all of the success with chimpanzee communication has been related to gesture and not to vocalisation. So, I think they're really trying to extrapolate backwards for what they see people doing in natural speech using gesture, so I try to educate the people that argue with me on those grounds, on the basis of what you can do with a chimpanzee.
SJ: So, I guess I'm just thinking about our conversation and winding us up a little bit now. One of the things I learnt from reading some of your work, Mike, is that perhaps language isn't as special as we think it is. Perhaps the experience, at least the internal experience of what the motivator for language is may be more common than we think.
MC: Absolutely, I think that's what I'm trying to say. I'm trying to argue against the specialness of language, that it comes out of natural human behaviour, natural things like memory, like time travel, like bodily movement. So what I'm indeed trying to argue for is evolutionary continuity, that this has all got to do animals that move, and animals that move have experiences in time and experiences in place, and gradually, over the course of the millennia, we've developed ways to communicate this with each other, but it's still presently. People have done studies of chimpanzees in the wild. They sort of tell stories, and they play. Animals play. They generate sequences of events, they communicate to some extent through movements, their own movements and their own gestures. There was probably a bit of discontinuity with things like bipedalism, which frees up the body a bit more, whereby you can get more elaborate kinds of communications of what's in your mind, but I don't buy the argument that all this happened 100,000 years ago. I think it's sort of there in animal behaviour, and in particular in primate behaviour, because primates are fairly capable of making expressive movements with their hands and bodies more than, say, a cow is. So I kind of like to see this, as you say, as it's been there all the time, really. We've kind of built it up to a fairly sophisticated degree in our own species, although we don't really know much about what happened in the last 6 billion years, since humans and chimpanzees went their different ways, but the notion that it all happened 100,000 years ago seems to me to be fairly minimal. It just doesn't make sense to me. I'm not really terribly sure about any of it, to be honest. You keep trying to make the case, and people have different arguments, but I think there's a huge sort ... It's partly religion, I think, it's all got to do with the notion or the attempts to prove that humans are somehow special. And I think that's kind of important, because we like to put down animals, we like to put down anything that's not sophisticated in the sense that human language is, so we kind of ... From the Bible on, really, language is the definition of what is special about humans. The one thing that we can be sure that we seem to ... Or we think we're sure about that we seem to do differently, is talk. It goes back to Descartes, it goes back to the Bible, it goes back to language being given to Adam as a special privilege, and so I think that overrides a hell of a lot of this. The notion that language makes us human, other animals don't have it, that makes us superior. It's probably got to do a lot with the fact, with a kind of guilt over the way we treat other animals. We eat them, and we ride on them, and we exploit them in goodness knows how many ways, so we have to have the notion that we are superior, that they don't have feelings, and all the rest of it. So I think that kind of overrides the whole thing in a way, and language is right at the centre of it.
SJ: It sounds almost like a post-hoc rationalisation…
MC: Yeah, exactly, it's a kind of rationalisation. It's sort of hard to break down, I think.