2022
pdf
bib
abs
Mutual Gaze and Linguistic Repetition in a Multimodal Corpus
Anais Murat
|
Maria Koutsombogera
|
Carl Vogel
Proceedings of the Thirteenth Language Resources and Evaluation Conference
This paper investigates the correlation between mutual gaze and linguistic repetition, a form of alignment, which we take as evidence of mutual understanding. We focus on a multimodal corpus made of three-party conversations and explore the question of whether mutual gaze events correspond to moments of repetition or non-repetition. Our results, although mainly significant on word unigrams and bigrams, suggest positive correlations between the presence of mutual gaze and the repetitions of tokens, lemmas, or parts-of-speech, but negative correlations when it comes to paired levels of representation (tokens or lemmas associated with their part-of-speech). No compelling correlation is found with duration of mutual gaze. Results are strongest when ignoring punctuation as representations of pauses, intonation, etc. in counting aligned tokens.
2018
pdf
bib
Modeling Collaborative Multimodal Behavior in Group Dialogues: The MULTISIMO Corpus
Maria Koutsombogera
|
Carl Vogel
Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)
2016
pdf
bib
abs
Multimodal Resources for Human-Robot Communication Modelling
Stavroula–Evita Fotinea
|
Eleni Efthimiou
|
Maria Koutsombogera
|
Athanasia-Lida Dimou
|
Theodore Goulas
|
Kyriaki Vasilaki
Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)
This paper reports on work related to the modelling of Human-Robot Communication on the basis of multimodal and multisensory human behaviour analysis. A primary focus in this framework of analysis is the definition of semantics of human actions in interaction, their capture and their representation in terms of behavioural patterns that, in turn, feed a multimodal human-robot communication system. Semantic analysis encompasses both oral and sign languages, as well as both verbal and non-verbal communicative signals to achieve an effective, natural interaction between elderly users with slight walking and cognitive inability and an assistive robotic platform.
2014
pdf
bib
abs
The Tutorbot Corpus — A Corpus for Studying Tutoring Behaviour in Multiparty Face-to-Face Spoken Dialogue
Maria Koutsombogera
|
Samer Al Moubayed
|
Bajibabu Bollepalli
|
Ahmed Hussen Abdelaziz
|
Martin Johansson
|
José David Aguas Lopes
|
Jekaterina Novikova
|
Catharine Oertel
|
Kalin Stefanov
|
Gül Varol
Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)
This paper describes a novel experimental setup exploiting state-of-the-art capture equipment to collect a multimodally rich game-solving collaborative multiparty dialogue corpus. The corpus is targeted and designed towards the development of a dialogue system platform to explore verbal and nonverbal tutoring strategies in multiparty spoken interactions. The dialogue task is centered on two participants involved in a dialogue aiming to solve a card-ordering game. The participants were paired into teams based on their degree of extraversion as resulted from a personality test. With the participants sits a tutor that helps them perform the task, organizes and balances their interaction and whose behavior was assessed by the participants after each interaction. Different multimodal signals captured and auto-synchronized by different audio-visual capture technologies, together with manual annotations of the tutors behavior constitute the Tutorbot corpus. This corpus is exploited to build a situated model of the interaction based on the participants temporally-changing state of attention, their conversational engagement and verbal dominance, and their correlation with the verbal and visual feedback and conversation regulatory actions generated by the tutor.
2006
pdf
bib
abs
Adding multi-layer semantics to the Greek Dependency Treebank
Harris Papageorgiou
|
Elina Desipri
|
Maria Koutsombogera
|
Kanella Pouli
|
Prokopis Prokopidis
Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC’06)
In this paper we give an overview of the approach adopted to add a layer of semantic information to the Greek Dependency Treebank [GDT]. Our ultimate goal is to come up with a large corpus, reliably annotated with rich semantic structures. To this end, a corpus has been compiled encompassing various data sources and domains. This collection has been preprocessed, annotated and validated on the basis of dependency representation. Taking into account multi-layered annotation schemes designed to provide deeper representations of structure and meaning, we describe the methodology followed as regards the semantic layer, we report on the annotation process and the problems faced and we conclude with comments on future work and exploitation of the resulting resource.