2024
pdf
bib
abs
Measuring Bargaining Abilities of LLMs: A Benchmark and A Buyer-Enhancement Method
Tian Xia
|
Zhiwei He
|
Tong Ren
|
Yibo Miao
|
Zhuosheng Zhang
|
Yang Yang
|
Rui Wang
Findings of the Association for Computational Linguistics ACL 2024
Bargaining is an important and unique part of negotiation between humans. As LLM-driven agents learn to negotiate and act like real humans, how to evaluate agents’ bargaining abilities remains an open problem.For the first time, we formally described the Bargaining task as an asymmetric incomplete information game, defining the gains of the Buyer and Seller in multiple bargaining processes. It allows us to quantitatively assess an agent’s performance in the Bargain task.We collected a real product price dataset, AmazonHistoryPrice, and conducted evaluations of various LLM agents’ bargaining abilities. We find that playing a Buyer is much harder than a Seller, and increasing model size can not effectively improve the Buyer’s performance.To address the challenge, we propose a novel approach called OG-Narrator that integrates a deterministic Offer Generator to control the price range of Buyer’s offers, and an LLM Narrator to create natural language sentences for generated offers.Experimental results show that OG-Narrator improves the buyer’s deal rates from 26.67% to 88.88% and brings a ten times multiplication of profits on all baselines, even a model that has not been aligned.
pdf
bib
abs
SPEADO: Segmentation and Punctuation for Ancient Chinese Texts via Example Augmentation and Decoding Optimization
Tian Xia
|
Kai Yu
|
Qianrong Yu
|
Xinran Peng
Proceedings of the Third Workshop on Language Technologies for Historical and Ancient Languages (LT4HALA) @ LREC-COLING-2024
The SPEADO model for sentence segmentation and punctuation tasks in ancient Chinese texts is proposed, which incorporates text chunking and MinHash indexing techniques to realise example argumentation. Additionally, decoding optimization strategies are introduced to direct the attention of the LLM model towards punctuation errors and address the issue of uncontrollable output. Experimental results show that the F1 score of the proposed method exceeds the baseline model by 14.18%, indicating a significant improvement in performance.
2022
pdf
bib
abs
A Localized Geometric Method to Match Knowledge in Low-dimensional Hyperbolic Space
Bo Hui
|
Tian Xia
|
Wei-Shinn Ku
Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing
Matching equivalent entities across Knowledge graphs is a pivotal step for knowledge fusion. Previous approaches usually study the problem in Euclidean space. However, recent works have shown that hyperbolic space has a higher capacity than Euclidean space and hyperbolic embedding can represent the hierarchical structure in a knowledge graph. In this paper, we propose a localized geometric method to find equivalent entities in hyperbolic space. Specifically, we use a hyperbolic neural network to encode the lingual information of entities and the structure of both knowledge graphs into a low-dimensional hyperbolic space. To address the asymmetry of structure on different KGs and the localized nature of relations, we learn an instance-specific geometric mapping function based on rotation to match entity pairs. A contrastive loss function is used to train the model. The experiment verifies the power of low-dimensional hyperbolic space for entity matching and shows that our method outperforms the state of the art by a large margin.
2021
pdf
bib
abs
PAII-NLP at SMM4H 2021: Joint Extraction and Normalization of Adverse Drug Effect Mentions in Tweets
Zongcheng Ji
|
Tian Xia
|
Mei Han
Proceedings of the Sixth Social Media Mining for Health (#SMM4H) Workshop and Shared Task
This paper describes our system developed for the subtask 1c of the sixth Social Media Mining for Health Applications (SMM4H) shared task in 2021. The aim of the subtask is to recognize the adverse drug effect (ADE) mentions from tweets and normalize the identified mentions to their mapping MedDRA preferred term IDs. Our system is based on a neural transition-based joint model, which is to perform recognition and normalization simultaneously. Our final two submissions outperform the average F1 score by 1-2%.
pdf
bib
abs
A Neural Transition-based Joint Model for Disease Named Entity Recognition and Normalization
Zongcheng Ji
|
Tian Xia
|
Mei Han
|
Jing Xiao
Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)
Disease is one of the fundamental entities in biomedical research. Recognizing such entities from biomedical text and then normalizing them to a standardized disease vocabulary offer a tremendous opportunity for many downstream applications. Previous studies have demonstrated that joint modeling of the two sub-tasks has superior performance than the pipelined counterpart. Although the neural joint model based on multi-task learning framework has achieved state-of-the-art performance, it suffers from the boundary inconsistency problem due to the separate decoding procedures. Moreover, it ignores the rich information (e.g., the text surface form) of each candidate concept in the vocabulary, which is quite essential for entity normalization. In this work, we propose a neural transition-based joint model to alleviate these two issues. We transform the end-to-end disease recognition and normalization task as an action sequence prediction task, which not only jointly learns the model with shared representations of the input, but also jointly searches the output by state transitions in one search space. Moreover, we introduce attention mechanisms to take advantage of the text surface form of each candidate concept for better normalization performance. Experimental results conducted on two publicly available datasets show the effectiveness of the proposed method.
2015
pdf
bib
abs
Une méthode discriminant formation simple pour la traduction automatique avec Grands Caractéristiques
Tian Xia
|
Shaodan Zhai
|
Zhongliang Li
|
Shaojun Wang
Actes de la 22e conférence sur le Traitement Automatique des Langues Naturelles. Articles courts
Marge infusé algorithmes détendus (MIRAS) dominent modèle de tuning dans la traduction automatique statistique dans le cas des grandes caractéristiques de l’échelle, mais ils sont également célèbres pour la complexité de mise en œuvre. Nous introduisons une nouvelle méthode, qui concerne une liste des N meilleures comme une permutation et minimise la perte Plackett-Luce de permutations rez-de-vérité. Des expériences avec des caractéristiques à grande échelle démontrent que, la nouvelle méthode est plus robuste que MERT ; si ce est seulement à rattacher avec Miras, il a un avantage comparativement, plus facile à mettre en œuvre.
2013
pdf
bib
Improving Alignment of System Combination by Using Multi-objective Optimization
Tian Xia
|
Zongcheng Ji
|
Shaodan Zhai
|
Yidong Chen
|
Qun Liu
|
Shaojun Wang
Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing
pdf
bib
A Corpus Level MIRA Tuning Strategy for Machine Translation
Ming Tan
|
Tian Xia
|
Shaojun Wang
|
Bowen Zhou
Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing
2009
pdf
bib
Weighted Alignment Matrices for Statistical Machine Translation
Yang Liu
|
Tian Xia
|
Xinyan Xiao
|
Qun Liu
Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing
pdf
bib
abs
The ICT statistical machine translation system for the IWSLT 2009
Haitao Mi
|
Yang Li
|
Tian Xia
|
Xinyan Xiao
|
Yang Feng
|
Jun Xie
|
Hao Xiong
|
Zhaopeng Tu
|
Daqi Zheng
|
Yanjuan Lu
|
Qun Liu
Proceedings of the 6th International Workshop on Spoken Language Translation: Evaluation Campaign
This paper describes the ICT Statistical Machine Translation systems that used in the evaluation campaign of the International Workshop on Spoken Language Translation (IWSLT) 2009. For this year’s evaluation, we participated in the Challenge Task (Chinese-English and English-Chinese) and BTEC Task (Chinese-English). And we mainly focus on one new method to improve single system’s translation quality. Specifically, we developed a sentence-similarity based development set selection technique. For each task, we finally submitted the single system who got the maximum BLEU scores on the selected development set. The four single translation systems are based on different techniques: a linguistically syntax-based system, two formally syntax-based systems and a phrase-based system. Typically, we didn’t use any rescoring or system combination techniques in this year’s evaluation.