Theme session on conceptual integration at ICLC 2001

Mark Turner, organizer of the theme session
Professor
Department of English language and literature
Doctoral Program in Neuroscience and Cognitive Science
University of Maryland
markt@umd5.umd.edu

Seana Coulson
Assistant Professor
Department of Cognitive Science
University of Califorrnia, San Diego
coulson@cogsci.ucsd.edu

Gilles Fauconnier
Professor and Chair
Department of Cognitive Science
University of California, San Diego
faucon@cogsci.ucsd.edu

Shweta Narayan
Doctoral student
Department of Linguistics
University of California, Berkeley
shweta@icsi.berkeley.edu

Rafael Núñez
Faculty
Department of Psychology
University of Freiburg
nunez@cogsci.berkeley.edu

Tim Rohrer
Postdoctoral fellow
Institute for Neural Computation
Department of Cognitive Science
University of California at San Diego
rohrer@cogsci.ucsd.edu

Eve Sweetser
Associate Professor
Department of Linguistics
University of California, Berkeley
sweetser@cogsci.ucsd.edu

Conceptual integration theory has developed rapidly in the last year. There are rich new theoretical analyses of its constitutive and guiding principles. Several new lines of research have been opened into its operation in the development and activation of grammatical constructions. There are new studies of blending in form-meaning pairing in both spoken and signed languages, and in the relationship of this kind of grammatical pairing to other systems of communication, including gesture and the use of culturally significant material anchors. Several new researchers have begun to contribute to the field. A presentation of this community of research is available at <markturner.org/blending.html>.

Until now, principal researchers in the field have been restricted in their presentations to more or less entry-level material. But by now the community of adepts in this field is large, and we propose a workshop pitched to their level. This theme session will serve not only as a forum for presenting research but also as a social focus where many researchers who have stayed in touch through electronic means and through attendance at a single lecture in some major city will be able to gather for personal conversation and extended informal sharing of ideas, in a group or groups.

SCHEDULE

We propose to conduct a half-day workshop, offering six coordinated talks, each lasting 20 minutes, each followed by 10 minutes of discussion.

TITLES AND ABSTRACTS

Seana Coulson. "I literally ate till I exploded: Literal and nonliteral meanings in conceptual integration networks"

In this talk I outline the meaning construction operations involved in the comprehension of figurative language, and address the implications of conceptual integration theory for the relationship between literal and nonliteral meanings. The focus is on contrasting predictions about on-line comprehension of figurative language made by two models of high-level language processing. One model is the standard pragmatic model (Grice, 1975), that posits distinct mechanisms for literal and nonliteral language processing. The other model is the space structuring model, and is based on the theory of conceptual integration, also known as blending (Coulson, in press; Fauconnier & Turner, 1998). In the space structuring model, literal and nonliteral language comprehension both proceed via the construction of simple cognitive models and the establishment of various sorts of mappings, or systematic correspondences between elements and relations in each.

The meaning construction operations involved in processing figurative language are discussed, and empirical results that support the space structuring model are reviewed. Experiments addressed three issues: (i) whether there is a qualitative difference in the processing of metaphors and more literal language; (ii) whether the continuum of metaphoricity suggested by blending theory predicted on-line comprehension difficulty; and, (iii) whether the right hemisphere is specialized for the processing of figurative language. We utilized a direct measure of brain activity that occurs in on-line language comprehension: event-related brain potentials (ERPs). Besides assessing whether literal and metaphoric stimuli elicited qualitatively different ERPs, we utilized properties of an ERP component, the N400, whose amplitude varies with the difficulty of semantic integration. By measuring the amplitude of this component, we were able to assess the difficulty of comprehending literal and figurative language in various conditions. Results argue against a firm dichotomy between literal and nonliteral language, and suggest that qualitatively similar processing operations underlie the comprehension of both sorts of meanings.

Although it represents a departure from the standard pragmatic model, the space structuring model does suggest that readers construct a "literal" interpretation automatically as part of the parsing process. However, this grammatically cued meaning construction occurs more-or-less in parallel with the structuring of other spaces in the network. Consequently, the space structuring model can explain evidence that "literal" meanings are activated in the course of metaphor comprehension. Such activations are argued to reflect the construction of cognitive models in various spaces in the conceptual integration network. As such it is crucial for establishing the overall meaning, a more comprehensive construal of the relationship between cognitive models in the source input, the target input, and the blended space.

The traditional characterization of nonliteral language comprehension as involving a single literal and a single nonliteral interpretation thus misses the extent to which meaning construction requires the apprehension of numerous relationships among models in multiple spaces. In sum, the space structuring model provides a way of accommodating empirical data that highlight the importance of "literal" meanings, as well as data that argue for the centrality of conceptual metaphor, and the imaginative processes of blending.

Rafael Nuñez and Eve Sweetser. "Spatial embodiment of temporal metaphors in Aymara: Blending source-domain gesture with speech."

Even a single language normally has multiple spatial construals of time, but basic human experiential correlations do constrain mappings between the domains of time and space (Lakoff [1993, 1999], Ahlers [1997], Nuñez [1999], Moore [2000]). In contradiction of some suggested universal constraints on time metaphors, Aymara (spoken in the Andean high plains of Bolivia, Peru, and Chile) appears to be the first well-supported case of a language which does use the mappings FUTURE IS IN BACK OF EGO and PAST IS IN FRONT OF EGO. In other proposed cases, "forwards" and "ahead" language refers in fact to relative past (before some other event), not to time prior to the speaker's present (Dunkel [1983], Moore [2000]) - common mappings, attested in Aymara too. But converging evidence (lexical, metaphorical, gestural) shows that Aymara also genuinely describes the past as "in front of" EGO. We examine Aymara models of time, using gestural and linguistic data videotaped in Nuñez and Neumann's ethnographic interviews with speakers from northern Chilean Aymara communities.

Speech-accompanying gesture is universal (Iverson & Thelen 1999) and provides a remarkable "back door" to linguistic cognition: a less consciously monitored channel than language, yet co-timed and co-produced with speech in a strongly language-specific processing package (McNeill 1992). Linguistic metaphorical mappings are paralleled systematically in gesture (Cienki[1998], Sweetser [1998]). In Aymara, forward gestures accompany past time reference; greater distance in front of the speaker maps onto increased remoteness of the time reference. Lexical analysis grounds the orientation of this mapping in the eyes' location on the front of the head and in correlations between the visual and epistemic domains (Miracle & Yapita, 1981). Aymara nayra "eye, sight, front," means "past" in nayra pacha "past times"; q''ipa "behind, back" means "future" in temporal expressions. The general motivation adduced for PAST IS IN FRONT OF EGO metaphorical systems is the KNOWING IS SEEING metaphor (Sweetser 1990), here grounded in the division of a visual scene into a visible and known area in front of EGO and a non-visible/unknown area behind EGO. Systems treating FUTURE as IN FRONT OF EGO, PAST as IN BACK are crosslinguistically the norm, and appear to reflect instead a basic experiential blend of motion along a linear path with a temporal sequence of visual experiences of locations passed through. This blend also involves KNOWING IS SEEING, but specifically KNOWN IS SEEN: areas behind a forward-facing moving EGO have been seen, while areas in front remain unseen and hence unknown.

For EGO-based mappings, Aymara speakers seem largely restricted to static linguistic and gestural descriptions of time. Temporal uses of motion verbs (COME, GO, PASS) are very rare with EGO-based mappings, and occurrences correlate with use of Spanish loan-words. This is coherent with Aymara mappings of a static back/front visual field onto temporal structure; any moving-EGO-based model would provide some basis for mapping FRONT to FUTURE, BACK to PAST.

Only further cultural research can assess the effect of such linguistic differences on nonlinguistic conceptualization of time. The Aymara static-Ego-based temporal metaphor system is consistent and experientially based. The rarity of such systems may stem from the equally strong experiential basis for the (inferentially more flexible) dynamic-EGO-centered models of time.

Shweta Narayan. "Conceptual integration in multimodal narration"

Comics are multimodal narratives requiring dynamic integration of visual and linguistic input both within and across panels. Normal linguistically-cued mental space construction is multimodal and dynamic (cf McNeill 1992, Sweetser 1998). As Oakley (1998) has shown, comics prompt complex multimodal conceptual integration which reflects subtle aspects of our general conceptualization of natural language narrative in context. I will suggest that the analytic tools of Mental Spaces theory (cf. Fauconnier 1997) may require extension to handle the full complexities of viewpoint and focus distribution in such multimodal conceptual blends.

Comics contain up to four different types of input in one frame: pictures, narrative boxes, speech bubbles, and onomatopoeic sound effects. These are integrated into a static multimodal whole. Linked panels form sequences, with new panels normally integrated into the development of a previously established dynamic space. Spatially distinct panels are thus being understood as temporally distinct parts of an action sequence. If a new panel shows the same landscape in a different lighting, or the same characters in different poses, it is understood as a later "time slice" of the dynamic space. If the panel also contains new characters or objects, these are understood as being added to the dynamic space.

Creating new narrative spaces requires overt space builders. Linguistic space builders often appear in narrative boxes, (e.g. ‘later that day…’ or ‘at that same moment, far away…’). Visual space builders include panel framing and changes in art style, color palette, and panel shape; these techniques can also re-evoke a previously created space and bring it into focus. The default interpretation of new panels, however, is integration into the currently focused narrative space.

One double-page spread in Sandman (Gaiman, 1994) shows a storyteller, images of his narrative content, the linguistic form of his narrative (as "voice-over" narrative boxes), and an image of an animal which, while not present in either the story-space or the narrator’s space, adds emotional tone to both. Focus is in the story, created jointly by images and "voice-over". Viewpoint apparently resides in the narrator’s distinctive voice, but each image also necessarily portrays visual viewpoint. The other images are outside the panels, and thus also in a different conceptual space, ‘outside’ the story. The narrator’s space acts like a temporary base in several ways. On another level, however, it is simultaneously an embedded space in the larger narrative. Analyses involving only shifting viewpoint and focus spaces, and a single base, seem insufficient to explain this space’s relation to the narrative. This suggests the existence of a previously unremarked phenomenon, in which for different purposes in the same situation, access is different.

Comics’ systematic use of static conventions to prompt construction of dynamic spaces forces relatively overt markings of inter-space transitions and relations between spaces. The conceptual integration processes involved are not a simple extension of those involved in static images or purely linguistic narratives; this makes comics a valuable resource for research in conceptual integration.

Tim Rohrer. "Spatial Relations Terms, Frames of Reference, and Blending Among the Micronesian Islanders"

The Micronesian islanders are renowned for their ability to navigate small seagoing outrigger canoes across vast stretches of ocean and reliably make landfall between tiny South Pacific atolls. What makes their skill more remarkable is that the traditional navigators accomplish this feat without conceptualizing the boat as moving along a Western-style chart and the geocentric frame of reference thus afforded; instead, the islanders use a viewer-centered frame of reference in which the world moves around the boat-which remains stationary for the duration of the journey. Navigation in the viewer-centered frame of reference relies on an intricate system composed of oral chants taught to fledgling navigators, simple sidereal (i.e. non-magnetic) compasses, and intimate experience with the effects of current, weather and season on the course chosen. In this presentation I explain how and why conceptual integration (blending) is crucial to this navigational process. For example, in the teaching of Caroline Islander chants, the process of naming off etak islands (islands used to make sidereal bearings) for a particular navigational route is blended with the process of using the breadfruit picker tool to twist off a breadfruit and catch it in a basket. Similarly, parallel navigational routes with comparable etak bearings are called sea-lane brothers, and when students are unable to remember the specifics of one route they are encouraged to run through the family of chants. These chants are examples of conceptual blending operating at both a cultural and a linguistic level, as enculturated knowledge from everyday life in Micronesia is blended with linguistic forms to make possible a technique of navigation whose mechanics have long baffled Western anthropologists. As such, it represents a rich case study to assess in terms of the Sapir-Whorf hypothesis, and I conclude this talk by discussing the relationship of the Micronesian terms for spatial relations and their favored frames of reference with respect to the Whorfian claims advanced by Pedersen, Levinson, Brown, et al. in their cross-cultural experiments on frames of reference in cognition and in their cross-linguistic typology of frames of reference in language.

Gilles Fauconnier. "Successive blends in material culture and scientific discovery"

Some of the things that we often take to be the most basic in everyday living and thinking are the result of creative successive blends evolved by cultures over hundreds of years. I will highlight the case of Time, Money, and Numbers by analyzing some of the intricate successive blending that leads to watches, coins or bills, and rational numbers.

Watches are material anchors (in Hutchins' sense) for a powerful conceptual blend that compresses an outer-space linear ordering of successive days that can go on to infinity into an inner-space cyclical ordering of repeated motion through the same unique day. We will see how this integration network provides one of the inputs (the Cyclical Blended Day) to another integration network, the Timepiece network, from which our modern everyday conception of time emerges. The seemingly obvious division of time into hours, minutes, seconds, ... turns out to be an awesome cognitive and cultural achievement taking place over many generations.

Money is a material anchor that also reflects the construction of successive intricate blends. The basic notion of exchange leads to an implicit and abstract value scale. Blending the input of Values with an input of Goods augmented by material objects like bills and coins yields a blended space of Goods and Money with Values in which the elementary structure of buying and selling as we know it can emerge. And further successive blends will lead to ever more complex emergent financial and economic dynamic organizations. A major goal of a social science like economics is to understand the emergent structures that result.

Learning by new generations is possible because much of the manipulation of the blended space can be learned without having to consciously apprehend the full networks of which it is part. Although buying and selling, or telling the time are activities that take time for children to learn, it is not the case that the children need to go through the long explorations and the many garden paths that the culture went through before it came up with watches or money.

Although the concept of number seems more abstract than that of day or money, its history consists of many successive blends quite similar to the ones above. I will illustrate this with the rational numbers, a surprisingly complex double-scope blend of the mental space of counting numbers with the mental space of proportions. Once the many clashes between the input spaces are overcome, and a viable blended space is achieved, rich emergent structure develops (and becomes an area of mathematical research). As in the cases of watches and money, the resulting dynamic organization (manipulation of rational numbers) can be learned fairly fast by members of a new generation.

Mark Turner. "Successive blends in material culture, Part Two: Writing, Speech, Sign Language, Gesture"

WRITING. Writing hardly seems the same kind of thing as a watch, a coin, or a cathedral. Yet when we look at it, we see physical marks on stone or paper or a computer screen, and these marks are circulated through the community. By themselves, these marks are meaningless: if we could send a sheet of writing back ten thousand years to a tribe of cognitively modern human beings, they would not have the slightest idea what to do with it, although the sheet would be a marvel. But we have elaborate conceptual and linguistic mental systems that can use these marks in culturally-supported ways. Just as we look at the watch to see what time it is, we look at a sentence in a letter to see what someone is saying to us. The blend seems natural to us even if it is immensely rich in its projections and elaborations.

A proficient reader ends up with a general blending template for writing and reading. In it, one input has someone talking and the other has some medium with marks, and in the blend, the marks and the speech are fused in impressive ways. This blending template does not say what is being said in the blend. For that, we need a particular material anchor: a particular letter, book, or inscription.

SPEECH. Speech may seem to be immaterial. But when one person speaks to another, what is happening, from one perspective, is that longitudinal waves in the air are striking one person's ear drum, and she is aware of this. Yet from the same perspective, a horse or a pig is doing just the same thing, and the human hearer is clearly doing something the animals are not. For the human hearer, the longitudinal waves give rise to "sounds" that are like physical objects. The human hearer knows a complex mapping that connects particular equivalence classes of sounds to particular linguistic structures like words and clauses that are publicly shared and mentally represented.

SIGN LANGUAGE. Scott Liddell, Karen von Hoek, Christine Poulin, and others have studied the ways in which connections between mental spaces are reflected and prompted for in the modality of sign. Scott Liddell has explicitly studied blended spaces in sign language in "Grounded blends, gestures, and conceptual shifts" (1998). Liddell shows that mental representations of one's immediate surroundings constitute a special type of blended space. Liddell points out that similar blends are also found in systems of gesture that accompany spoken languages, and indeed we find such blends involving gestural material anchors so natural that we might have to think twice to see how complex these performances really are. They bring in the power of double-scope blending, which is an astonishing capacity relative to the rest of the biological world, but one which human beings take for granted because every one of them can use it easily, beginning from the earliest age.