Universität zu Köln at the Cologne Aachen Gesture Colloquium
on "Multimodal stance and emotion in the oldest-old"
on November 24, 2016, 4:00 - 5:30 p.m.
at the Senatssaal der Deutschen Sporthochschule Köln,
Am Sportpark Müngersdorf 6, 50933 Köln.
The field of pragmatics is a multidisciplinary domain that explores the underlying processes and realization of meaning in use, taking into account available evidence provided by the context within which the interaction takes place. Aiming at understanding language in the most integrative way possible, the CorpAGEst project (http://corpagest.org) strives to establish the verbal and gestural profile of very old people, looking at their pragmatic competence in interaction, that is, at their ability to use language resources in a contextually appropriate manner (Kasper & Rose, 2002). To reach this goal, a multimodal corpus of audio- and video-recorded interactions was created in order to test several hypotheses: (i) discourse markers (e.g., tu sais/vous savez ?you know?) and pragmatic gestures (e.g., an exaggerated opening of the eyes) are relevant indicators of the overall pragmatic competence of the aging subject; (ii) a change in the concurrent use of these (non)verbal pragmatic cues (incl. speech, facial expression, eye gaze, hand gesture, and body gesture) could be an indicator of an adaptive strategy used, with advancing age, to reduce the cost of language production during interaction. Contextual independent variables were also included in the corpus design, such as the environment type (private vs. residential home), the social tie between the participants (familiar vs. unknown interviewer) and the task type (focusing on past events vs. present-day life). As recently stated by Keisanen & K?rkk?inen (2014), stance taking in the embodied interaction is concerned with the study of multimodal practices (including language, prosody, gesture, body posture, as well as sequential position and timing, activity and situation settings). In this talk, particular attention will be paid to pragmatic markers of stance in speech and gesture, in order to shed some light on pragmatic cues which are specific to one or the other modality and, most importantly, on the pragmatic functions and/or forms possibly shared by the concurrent systems. After an introduction of the general basics and assumptions that are at the core of the project, the annotation principles adopted to analyze the functions of pragmatic markers in speech and gesture will be presented, and some results will be highlighted. Preliminary results indicate that the use of nonverbal resources is highly idiosyncratic: for instance, no clear physiological pattern seems to be emotion-specific when decoding emotions from old persons? face (Bolly & Thomas, 2015). Yet, some regularity emerges when looking at the way discourse markers and hand gestures combine in the speech of one single speaker, with respect to their pairing of form(s) and function in context of use (cf. the notion of ?multimodal pragmatic constructions? ? Bolly & Boutet, forthc.). Even though providing only one part of the big picture, the approach allows for a better understanding of the way people communicate in later life, and thereby deserves urgent attention and research to move from experiments in the laboratory towards empirical studies ?into the wild?.
Bolly, C. T., and D. Boutet (forthcoming). The multimodal CorpAGEst corpus: Keeping an eye on pragmatic competence in later life. Corpora 13 (2). Draft paper available at: https://corpagest.files.wordpress.com/2016/07/bollyboutet_corpagest_corpora-draft.pdf.
Bolly, C. T., and A. Thomas (2015). Facing Nadine?s speech: Multimodal annotation of emotion in later life. In K. Jokinen and M. Vels (eds), Proceedings of the 2nd European and the 5th Nordic Symposium on Multimodal Communication August 6-8, Tartu, Estonia, pp. 23-32. Link?ping: Link?ping Electronic Conference Proceedings 110, http://www.ep.liu.se/ecp_home/index.en.aspx?issue=110.
Kasper, G., and K. R. Rose (2002). Pragmatic development in a second language. Mahwah: Blackwell (Also Language Learning: Supplement 1, 52).
Keisanen, T., and E. K?rkk?inen (2014). Stance. In K. P. Schneider and A. Barron (eds.), Pragmatics of discourse (Handbooks of Pragmatics 3), pp. 295-322. Berlin, Boston: Walter de Gruyter.
The Cologne Aachen Gesture Colloquium is organized by Univ.-Prof. Dr. Hedda Lausberg (Berlin Gesture Center (BGC Cologne), Dept. of Neurology, Psychosomatic Medicine, and Psychiatry of the German Sport University Cologne) und Univ.-Prof. Dr. Irene Mittelberg (Natural Media NeuroPeirce, HumTec of the RWTH Aachen; Competence Center for Sign Language and Gesture SignGEs).