Research Project
Multisensory integration in cognitive development and learning

In order to make sense of the world, infants, children and adults integrate information from multiple sources, for example the visual and auditory properties (or names) of objects and animals. My project at the Lichtenberg-Kolleg will empirically investigate how the interrelationship between information from different domains affects learning.

One case of multisensory integration that has recently elicited renewed interest is sound symbolism. Sound symbolism describes the phenomenon that what a thing is called is linked to what it is like, contrary to the commonly held belief of the ›arbitrariness of the sign‹. For example, when seeing a novel roundshaped object people are more likely to give it a name containing a rounded vowel such as ›bom‹ (rather that ›zak‹). A popular explanation for sound symbolism has been that the rounded shape of the mouth whilst articulating round vowels such as ›o‹ is matched to the rounded features of the object. However, it is not clear that this explanation holds universally, and German offers the ideal test case to evaluate this explanation: here we have umlauts (e.g., ü) that are produced with rounded lips but sound less rounded than, for example, ›o‹. In collaboration with Prof. Nivedita Mani (Psychology) we will conduct a range of empirical studies with German infants and adults to investigate how this ›synesthetic mismatch‹ influences sound symbolic judgments. This project will allow us to gain a better understanding of the mechanisms underlying sound symbolism and multisensory integration in general.

A second domain that we will address in collaboration with Prof. Markus Steinbach and Prof. Nivedita Mani concerns the acquisition of sign language in normal and hearing impaired children and adults. Young children often prefer to have a single name for an object, but with the increase of baby signing an object can be referred to both by a sign and a word. We will therefore investigate whether the acquisition of a sign as a ›label‹ for an object influences infant and adult acquisition of a word as another label for the same object. The question we will address in this way is in how far a visual and an auditory label compete to represent the same visual object. Together, these studies will enhance our knowledge of the way in which multisensory information is combined in children and adults.

Selected Publications

Westermann, G. and Ruh, N. (in press). A neuroconstructivist model of past tense development and processing. Psychological Review.

Westermann, G. and Mareschal, D. (in press). Mechanisms of developmental change in infant categorization. Cognitive Development.

Westermann, G., N. Ruh and K. Plunkett. 2009. Connectionist approaches to language learning. Linguistics 47: 413-452.

Westermann, G., D. Mareschal, M. Johnson, S. Sirois, M. Spratling and M. Thomas. 2007. Neuroconstructivism. Developmental Science 10: 75-83.

Westermann, G. and D. Mareschal. 2004. From parts to wholes: mechanisms of development in infant visual object processing. Infancy 5: 131-151.