2023
pdf
bib
abs
System-Initiated Transitions from Chit-Chat to Task-Oriented Dialogues with Transition Info Extractor and Transition Sentence Generator
Ye Liu
|
Stefan Ultes
|
Wolfgang Minker
|
Wolfgang Maier
Proceedings of the 16th International Natural Language Generation Conference
In this work, we study dialogue scenarios that start from chit-chat but eventually switch to task-related services, and investigate how a unified dialogue model, which can engage in both chit-chat and task-oriented dialogues, takes the initiative during the dialogue mode transition from chit-chat to task-oriented in a coherent and cooperative manner. We firstly build a transition info extractor (TIE) that keeps track of the preceding chit-chat interaction and detects the potential user intention to switch to a task-oriented service. Meanwhile, in the unified model, a transition sentence generator (TSG) is extended through efficient Adapter tuning and transition prompt learning. When the TIE successfully finds task-related information from the preceding chit-chat, such as a transition domain (“train” in Figure fig: system-initiated transition from chit-chat to task-oriented.), then the TSG is activated automatically in the unified model to initiate this transition by generating a transition sentence under the guidance of transition information extracted by TIE. The experimental results show promising performance regarding the proactive transitions. We achieve an additional large improvement on TIE model by utilizing Conditional Random Fields (CRF). The TSG can flexibly generate transition sentences while maintaining the unified capabilities of normal chit-chat and task-oriented response generation.
2021
pdf
bib
abs
Context Matters in Semantically Controlled Language Generation for Task-oriented Dialogue Systems
Ye Liu
|
Wolfgang Maier
|
Wolfgang Minker
|
Stefan Ultes
Proceedings of the 18th International Conference on Natural Language Processing (ICON)
This work combines information about the dialogue history encoded by pre-trained model with a meaning representation of the current system utterance to realise contextual language generation in task-oriented dialogues. We utilise the pre-trained multi-context ConveRT model for context representation in a model trained from scratch; and leverage the immediate preceding user utterance for context generation in a model adapted from the pre-trained GPT-2. Both experiments with the MultiWOZ dataset show that contextual information encoded by pre-trained model improves the performance of response generation both in automatic metrics and human evaluation. Our presented contextual generator enables higher variety of generated responses that fit better to the ongoing dialogue. Analysing the context size shows that longer context does not automatically lead to better performance, but the immediate preceding user utterance plays an essential role for contextual generation. In addition, we also propose a re-ranker for the GPT-based generation model. The experiments show that the response selected by the re-ranker has a significant improvement on automatic metrics.
pdf
bib
abs
Naturalness Evaluation of Natural Language Generation in Task-oriented Dialogues Using BERT
Ye Liu
|
Wolfgang Maier
|
Wolfgang Minker
|
Stefan Ultes
Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021)
This paper presents an automatic method to evaluate the naturalness of natural language generation in dialogue systems. While this task was previously rendered through expensive and time-consuming human labor, we present this novel task of automatic naturalness evaluation of generated language. By fine-tuning the BERT model, our proposed naturalness evaluation method shows robust results and outperforms the baselines: support vector machines, bi-directional LSTMs, and BLEURT. In addition, the training speed and evaluation performance of naturalness model are improved by transfer learning from quality and informativeness linguistic knowledge.
pdf
bib
abs
Blending Task Success and User Satisfaction: Analysis of Learned Dialogue Behaviour with Multiple Rewards
Stefan Ultes
|
Wolfgang Maier
Proceedings of the 22nd Annual Meeting of the Special Interest Group on Discourse and Dialogue
Recently, principal reward components for dialogue policy reinforcement learning use task success and user satisfaction independently and neither the resulting learned behaviour has been analysed nor a suitable proper analysis method even existed. In this work, we employ both principal reward components jointly and propose a method to analyse the resulting behaviour through a structured way of probing the learned policy. We show that blending both reward components increases user satisfaction without sacrificing task success in more hostile environments and provide insight about actions chosen by the learned policies.
2020
pdf
bib
abs
Similarity Scoring for Dialogue Behaviour Comparison
Stefan Ultes
|
Wolfgang Maier
Proceedings of the 21th Annual Meeting of the Special Interest Group on Discourse and Dialogue
The differences in decision making between behavioural models of voice interfaces are hard to capture using existing measures for the absolute performance of such models. For instance, two models may have a similar task success rate, but very different ways of getting there. In this paper, we propose a general methodology to compute the similarity of two dialogue behaviour models and investigate different ways of computing scores on both the semantic and the textual level. Complementing absolute measures of performance, we test our scores on three different tasks and show the practical usability of the measures.
2018
pdf
bib
Towards an Automatic Assessment of Crowdsourced Data for NLU
Patricia Braunger
|
Wolfgang Maier
|
Jan Wessling
|
Maria Schmidt
Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)
pdf
bib
abs
GHHT at CALCS 2018: Named Entity Recognition for Dialectal Arabic Using Neural Networks
Mohammed Attia
|
Younes Samih
|
Wolfgang Maier
Proceedings of the Third Workshop on Computational Approaches to Linguistic Code-Switching
This paper describes our system submission to the CALCS 2018 shared task on named entity recognition on code-switched data for the language variant pair of Modern Standard Arabic and Egyptian dialectal Arabic. We build a a Deep Neural Network that combines word and character-based representations in convolutional and recurrent networks with a CRF layer. The model is augmented with stacked layers of enriched information such pre-trained embeddings, Brown clusters and named entity gazetteers. Our system is ranked second among those participating in the shared task achieving an FB1 average of 70.09%.
pdf
bib
abs
GHH at SemEval-2018 Task 10: Discovering Discriminative Attributes in Distributional Semantics
Mohammed Attia
|
Younes Samih
|
Manaal Faruqui
|
Wolfgang Maier
Proceedings of the 12th International Workshop on Semantic Evaluation
This paper describes our system submission to the SemEval 2018 Task 10 on Capturing Discriminative Attributes. Given two concepts and an attribute, the task is to determine whether the attribute is semantically related to one concept and not the other. In this work we assume that discriminative attributes can be detected by discovering the association (or lack of association) between a pair of words. The hypothesis we test in this contribution is whether the semantic difference between two pairs of concepts can be treated in terms of measuring the distance between words in a vector space, or can simply be obtained as a by-product of word co-occurrence counts.
2017
pdf
bib
abs
Natural Language Input for In-Car Spoken Dialog Systems: How Natural is Natural?
Patricia Braunger
|
Wolfgang Maier
Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue
Recent spoken dialog systems are moving away from command and control towards a more intuitive and natural style of interaction. In order to choose an appropriate system design which allows the system to deal with naturally spoken user input, a definition of what exactly constitutes naturalness in user input is important. In this paper, we examine how different user groups naturally speak to an automotive spoken dialog system (SDS). We conduct a user study in which we collect freely spoken user utterances for a wide range of use cases in German. By means of a comparative study of the utterances from the study with interpersonal utterances, we provide criteria what constitutes naturalness in the user input of an state-of-the-art automotive SDS.
2016
pdf
bib
Proceedings of the Workshop on Discontinuous Structures in Natural Language Processing
Wolfgang Maier
|
Sandra Kübler
|
Constantin Orasan
Proceedings of the Workshop on Discontinuous Structures in Natural Language Processing
pdf
bib
Discontinuous parsing with continuous trees
Wolfgang Maier
|
Timm Lichte
Proceedings of the Workshop on Discontinuous Structures in Natural Language Processing
pdf
bib
SAWT: Sequence Annotation Web Tool
Younes Samih
|
Wolfgang Maier
|
Laura Kallmeyer
Proceedings of the Second Workshop on Computational Approaches to Code Switching
pdf
bib
abs
An Arabic-Moroccan Darija Code-Switched Corpus
Younes Samih
|
Wolfgang Maier
Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)
In this paper, we describe our effort in the development and annotation of a large scale corpus containing code-switched data. Until recently, very limited effort has been devoted to develop computational approaches or even basic linguistic resources to support research into the processing of Moroccan Darija.
2015
pdf
bib
LR Parsing for LCFRS
Laura Kallmeyer
|
Wolfgang Maier
Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
pdf
bib
Discontinuous Incremental Shift-reduce Parsing
Wolfgang Maier
Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)
2014
pdf
bib
abs
Discosuite - A parser test suite for German discontinuous structures
Wolfgang Maier
|
Miriam Kaeshammer
|
Peter Baumann
|
Sandra Kübler
Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)
Parser evaluation traditionally relies on evaluation metrics which deliver a single aggregate score over all sentences in the parser output, such as PARSEVAL. However, for the evaluation of parser performance concerning a particular phenomenon, a test suite of sentences is needed in which this phenomenon has been identified. In recent years, the parsing of discontinuous structures has received a rising interest. Therefore, in this paper, we present a test suite for testing the performance of dependency and constituency parsers on non-projective dependencies and discontinuous constituents for German. The test suite is based on the newly released TIGER treebank version 2.2. It provides a unique possibility of benchmarking parsers on non-local syntactic relationships in German, for constituents and dependencies. We include a linguistic analysis of the phenomena that cause discontinuity in the TIGER annotation, thereby closing gaps in previous literature. The linguistic phenomena we investigate include extraposition, a placeholder/repeated element construction, topicalization, scrambling, local movement, parentheticals, and fronting of pronouns.
pdf
bib
The IUCL+ System: Word-Level Language Identification via Extended Markov Models
Levi King
|
Eric Baucom
|
Timur Gilmanov
|
Sandra Kübler
|
Dan Whyatt
|
Wolfgang Maier
|
Paul Rodrigues
Proceedings of the First Workshop on Computational Approaches to Code Switching
pdf
bib
Language variety identification in Spanish tweets
Wolfgang Maier
|
Carlos Gómez-Rodríguez
Proceedings of the EMNLP’2014 Workshop on Language Technology for Closely Related Languages and Language Variants
pdf
bib
Parsing German: How Much Morphology Do We Need?
Wolfgang Maier
|
Sandra Kübler
|
Daniel Dakota
|
Daniel Whyatt
Proceedings of the First Joint Workshop on Statistical Parsing of Morphologically Rich Languages and Syntactic Analysis of Non-Canonical Languages
2013
pdf
bib
Data-Driven Parsing using Probabilistic Linear Context-Free Rewriting Systems
Laura Kallmeyer
|
Wolfgang Maier
Computational Linguistics, Volume 39, Issue 1 - March 2013
pdf
bib
Overview of the SPMRL 2013 Shared Task: A Cross-Framework Evaluation of Parsing Morphologically Rich Languages
Djamé Seddah
|
Reut Tsarfaty
|
Sandra Kübler
|
Marie Candito
|
Jinho D. Choi
|
Richárd Farkas
|
Jennifer Foster
|
Iakes Goenaga
|
Koldo Gojenola Galletebeitia
|
Yoav Goldberg
|
Spence Green
|
Nizar Habash
|
Marco Kuhlmann
|
Wolfgang Maier
|
Joakim Nivre
|
Adam Przepiórkowski
|
Ryan Roth
|
Wolfgang Seeker
|
Yannick Versley
|
Veronika Vincze
|
Marcin Woliński
|
Alina Wróblewska
|
Eric Villemonte de la Clergerie
Proceedings of the Fourth Workshop on Statistical Parsing of Morphologically-Rich Languages
pdf
bib
LCFRS binarization and debinarization for directional parsing
Wolfgang Maier
Proceedings of the 13th International Conference on Parsing Technologies (IWPT 2013)
2012
pdf
bib
Annotating Coordination in the Penn Treebank
Wolfgang Maier
|
Sandra Kübler
|
Erhard Hinrichs
|
Julia Krivanek
Proceedings of the Sixth Linguistic Annotation Workshop
pdf
bib
PLCFRS Parsing Revisited: Restricting the Fan-Out to Two
Wolfgang Maier
|
Miriam Kaeshammer
|
Laura Kallmeyer
Proceedings of the 11th International Workshop on Tree Adjoining Grammars and Related Formalisms (TAG+11)
2010
pdf
bib
Direct Parsing of Discontinuous Constituents in German
Wolfgang Maier
Proceedings of the NAACL HLT 2010 First Workshop on Statistical Parsing of Morphologically-Rich Languages
pdf
bib
Discontinuity and Non-Projectivity: Using Mildly Context-Sensitive Formalisms for Data-Driven Parsing
Wolfgang Maier
|
Laura Kallmeyer
Proceedings of the 10th International Workshop on Tree Adjoining Grammar and Related Frameworks (TAG+10)
pdf
bib
Data-Driven Parsing with Probabilistic Linear Context-Free Rewriting Systems
Laura Kallmeyer
|
Wolfgang Maier
Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010)
2009
pdf
bib
abs
Un Algorithme d’Analyse de Type Earley pour Grammaires à Concaténation d’Intervalles
Laura Kallmeyer
|
Wolfgang Maier
|
Yannick Parmentier
Actes de la 16ème conférence sur le Traitement Automatique des Langues Naturelles. Articles courts
Nous présentons ici différents algorithmes d’analyse pour grammaires à concaténation d’intervalles (Range Concatenation Grammar, RCG), dont un nouvel algorithme de type Earley, dans le paradigme de l’analyse déductive. Notre travail est motivé par l’intérêt porté récemment à ce type de grammaire, et comble un manque dans la littérature existante.
pdf
bib
Parsing Coordinations
Sandra Kübler
|
Erhard Hinrichs
|
Wolfgang Maier
|
Eva Klett
Proceedings of the 12th Conference of the European Chapter of the ACL (EACL 2009)
pdf
bib
An Incremental Earley Parser for Simple Range Concatenation Grammar
Laura Kallmeyer
|
Wolfgang Maier
Proceedings of the 11th International Conference on Parsing Technologies (IWPT’09)
pdf
bib
Synchronous Rewriting in Treebanks
Laura Kallmeyer
|
Wolfgang Maier
|
Giorgio Satta
Proceedings of the 11th International Conference on Parsing Technologies (IWPT’09)
pdf
bib
An Earley Parsing Algorithm for Range Concatenation Grammars
Laura Kallmeyer
|
Wolfgang Maier
|
Yannick Parmentier
Proceedings of the ACL-IJCNLP 2009 Conference Short Papers
2008
pdf
bib
TuLiPA: Towards a Multi-Formalism Parsing Environment for Grammar Engineering
Laura Kallmeyer
|
Timm Lichte
|
Wolfgang Maier
|
Yannick Parmentier
|
Johannes Dellert
|
Kilian Evang
Coling 2008: Proceedings of the workshop on Grammar Engineering Across Frameworks
pdf
bib
TuLiPA: A syntax-semantics parsing environment for mildly context-sensitive formalisms
Yannick Parmentier
|
Laura Kallmeyer
|
Wolfgang Maier
|
Timm Lichte
|
Johannes Dellert
Proceedings of the Ninth International Workshop on Tree Adjoining Grammar and Related Frameworks (TAG+9)
pdf
bib
Proceedings of the ACL-08: HLT Student Research Workshop
Ebru Arisoy
|
Keisuke Inoue
|
Wolfgang Maier
Proceedings of the ACL-08: HLT Student Research Workshop
pdf
bib
abs
Developing a TT-MCTAG for German with an RCG-based Parser
Laura Kallmeyer
|
Timm Lichte
|
Wolfgang Maier
|
Yannick Parmentier
|
Johannes Dellert
Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC'08)
Developing linguistic resources, in particular grammars, is known to be a complex task in itself, because of (amongst others) redundancy and consistency issues. Furthermore some languages can reveal themselves hard to describe because of specific characteristics, e.g. the free word order in German. In this context, we present (i) a framework allowing to describe tree-based grammars, and (ii) an actual fragment of a core multicomponent tree-adjoining grammar with tree tuples (TT-MCTAG) for German developed using this framework. This framework combines a metagrammar compiler and a parser based on range concatenation grammar (RCG) to respectively check the consistency and the correction of the grammar. The German grammar being developed within this framework already deals with a wide range of scrambling and extraction phenomena.
pdf
bib
abs
How to Compare Treebanks
Sandra Kübler
|
Wolfgang Maier
|
Ines Rehbein
|
Yannick Versley
Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC'08)
Recent years have seen an increasing interest in developing standards for linguistic annotation, with a focus on the interoperability of the resources. This effort, however, requires a profound knowledge of the advantages and disadvantages of linguistic annotation schemes in order to avoid importing the flaws and weaknesses of existing encoding schemes into the new standards. This paper addresses the question how to compare syntactically annotated corpora and gain insights into the usefulness of specific design decisions. We present an exhaustive evaluation of two German treebanks with crucially different encoding schemes. We evaluate three different parsers trained on the two treebanks and compare results using EvalB, the Leaf-Ancestor metric, and a dependency-based evaluation. Furthermore, we present TePaCoC, a new testsuite for the evaluation of parsers on complex German grammatical constructions. The testsuite provides a well thought-out error classification, which enables us to compare parser output for parsers trained on treebanks with different encoding schemes and provides interesting insights into the impact of treebank annotation schemes on specific constructions like PP attachment or non-constituent coordination.
2006
pdf
bib
Annotation Schemes and their Influence on Parsing Results
Wolfgang Maier
Proceedings of the COLING/ACL 2006 Student Research Workshop
pdf
bib
Is it Really that Difficult to Parse German?
Sandra Kübler
|
Erhard W. Hinrichs
|
Wolfgang Maier
Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing