Overview
I study the interaction of language and cognition with an emphasis on multimodality. My research focuses on how our interaction with and perception of space influences our language, and in turn how our language use reflects and construes how we think about spatial relationships and events. I work on both co-speech gesture and signed language, and make use of a variety of research methods including experimental and computational, corpus-based approaches. As a cognitive semanticist, I work within the cognitive linguistics framework, drawing on frame semantics, conceptual metaphor theory, and Embodied Construction Grammar.
Metaphoric motion and Multimodal Embodied Construction Grammar
My current research, dating back to my dissertation, is on the use of manner and path metaphoric motion verbs in co-speech gesture. In particular, I make use of both gesture data collected from naturalistic sources (online video archives) and lab-elicited gesture production to understand how this family of metaphors interacts with both grammatical structures and modalities, and how it compares to how we use the same words in literal contexts. This research incorporates formalisms from Embodied Construction Grammar, along with the metaphor analysis formalisms developed at MetaNet, both to analyze data and as a tool for collecting metaphoric language in spoken and written English. Ultimately the goal of this project is to develop a multimodal variant of Embodied Construction Grammar which incorporates gestural content into the analysis of linguistic form and meaning.
MetaNet: Computational approaches to metaphor analysis
MetaNet is a very large-scale project with two main goals: (a) develop a system to automatically identify and extract metaphoric language from texts for NLP and translation purposes; (b) develop a rigorously analyzed database of conceptual metaphors and schemas to serve as a research and teaching tool for cognitive linguistics. I am primarily involved in the second goal and work on the linguistic analysis needed for the database. I also work on incorporating MetaNet and Embodied Construction Grammar formalisms. As a result of this project, we have developed a robust theory of metaphor compositionality that moves beyond the typical “lists of metaphors” approach. As this project continues and the database grows we will be able to make falsifiable predictions regarding the internal structure of conceptual metaphors. This approach makes use of corpus linguistics techniques to analyze usage data in large written and spoken texts.
Gesture, metaphor, and communication modality
This joint project with Tasha N. Lewis is an investigation of the interaction between metaphoric gestures and cognition. While we know that gestures can be metaphoric, and that they often convey information complementary to their coordinated spoken language, we don’t know as much about how the perception of metaphoric gesture influences cognition, or how communication modality influences our perception of gesture. Our work shows that metaphoric gestures, even when not overtly iconic, are not only communicative but prime metaphoric reasoning in the addressee.
Gesture production and mental imagery
I am currently pursuing a new line of research into gesture and cognition with Tasha N. Lewis and Matthew W. Kirkhart on the relationship between variation in gesture production and mental imagery skills. It has been previously shown that high spatial reasoning skills are correlated with increased iconic gesture production rates; we will investigate if mental imagery skills are also correlated with gesture production, and if there is an interaction between spatial reasoning, mental imagery, and gesture viewpoint.