Fumiyo Fukumoto


2024

pdf bib
Reduction-Synthesis: Plug-and-Play for Sentiment Style Transfer
Sheng Xu | Fumiyo Fukumoto | Yoshimi Suzuki
Proceedings of the 17th International Natural Language Generation Conference

Sentiment style transfer (SST), a variant of text style transfer (TST), has recently attracted extensive interest. Some disentangling-based approaches have improved performance, while most still struggle to properly transfer the input as the sentiment style is intertwined with the content of the text. To alleviate the issue, we propose a plug-and-play method that leverages an iterative self-refinement algorithm with a large language model (LLM). Our approach separates the straightforward Seq2Seq generation into two phases: (1) Reduction phase which generates a style-free sequence for a given text, and (2) Synthesis phase which generates the target text by leveraging the sequence output from the first phase. The experimental results on two datasets demonstrate that our transfer strategy is effective for challenging SST cases where the baseline methods perform poorly. Our code is available online.

pdf bib
Enhanced Coherence-Aware Network with Hierarchical Disentanglement for Aspect-Category Sentiment Analysis
Jin Cui | Fumiyo Fukumoto | Xinfeng Wang | Yoshimi Suzuki | Jiyi Li | Noriko Tomuro | Wanzeng Kong
Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)

Aspect-category-based sentiment analysis (ACSA), which aims to identify aspect categories and predict their sentiments has been intensively studied due to its wide range of NLP applications. Most approaches mainly utilize intrasentential features. However, a review often includes multiple different aspect categories, and some of them do not explicitly appear in the review. Even in a sentence, there is more than one aspect category with its sentiments, and they are entangled intra-sentence, which makes the model fail to discriminately preserve all sentiment characteristics. In this paper, we propose an enhanced coherence-aware network with hierarchical disentanglement (ECAN) for ACSA tasks. Specifically, we explore coherence modeling to capture the contexts across the whole review and to help the implicit aspect and sentiment identification. To address the issue of multiple aspect categories and sentiment entanglement, we propose a hierarchical disentanglement module to extract distinct categories and sentiment features. Extensive experimental and visualization results show that our ECAN effectively decouples multiple categories and sentiments entangled in the coherence representations and achieves state-of-the-art (SOTA) performance. Our codes and data are available online: https://github.com/cuijin-23/ECAN.

pdf bib
RDRec: Rationale Distillation for LLM-based Recommendation
Xinfeng Wang | Jin Cui | Yoshimi Suzuki | Fumiyo Fukumoto
Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)

Large language model (LLM)-based recommender models that bridge users and items through textual prompts for effective semantic reasoning have gained considerable attention. However, few methods consider the underlying rationales behind interactions, such as user preferences and item attributes, limiting the reasoning ability of LLMs for recommendations. This paper proposes a rationale distillation recommender (RDRec), a compact model designed to learn rationales generated by a larger language model (LM). By leveraging rationales from reviews related to users and items, RDRec remarkably specifies their profiles for recommendations. Experiments show that RDRec achieves state-of-the-art (SOTA) performance in both top-N and sequential recommendations. Our code is available online.

2023

pdf bib
Aspect-Category Enhanced Learning with a Neural Coherence Model for Implicit Sentiment Analysis
Jin Cui | Fumiyo Fukumoto | Xinfeng Wang | Yoshimi Suzuki | Jiyi Li | Wanzeng Kong
Findings of the Association for Computational Linguistics: EMNLP 2023

Aspect-based sentiment analysis (ABSA) has been widely studied since the explosive growth of social networking services. However, the recognition of implicit sentiments that do not contain obvious opinion words remains less explored. In this paper, we propose aspect-category enhanced learning with a neural coherence model (ELCoM). It captures document-level coherence by using contrastive learning, and sentence-level by a hypergraph to mine opinions from explicit sentences to aid implicit sentiment classification. To address the issue of sentences with different sentiment polarities in the same category, we perform cross-category enhancement to offset the impact of anomalous nodes in the hypergraph and obtain sentence representations with enhanced aspect-category. Extensive experiments on benchmark datasets show that the ELCoM achieves state-of-the-art performance. Our source codes and data are released at https://github.com/cuijin-23/ELCoM.

pdf bib
Knowledge Injection with Perturbation-based Constrained Attention Network for Word Sense Disambiguation
Fumiyo Fukumoto | Shou Asakawa
Findings of the Association for Computational Linguistics: IJCNLP-AACL 2023 (Findings)

pdf bib
Learning Disentangled Meaning and Style Representations for Positive Text Reframing
Xu Sheng | Fumiyo Fukumoto | Jiyi Li | Go Kentaro | Yoshimi Suzuki
Proceedings of the 16th International Natural Language Generation Conference

The positive text reframing (PTR) task which generates a text giving a positive perspective with preserving the sense of the input text, has attracted considerable attention as one of the NLP applications. Due to the significant representation capability of the pre-trained language model (PLM), a beneficial baseline can be easily obtained by just fine-tuning the PLM. However, how to interpret a diversity of contexts to give a positive perspective is still an open problem. Especially, it is more serious when the size of the training data is limited. In this paper, we present a PTR framework, that learns representations where the meaning and style of text are structurally disentangled. The method utilizes pseudo-positive reframing datasets which are generated with two augmentation strategies. A simple but effective multi-task learning-based model is learned to fuse the generation capabilities from these datasets. Experimental results on Positive Psychology Frames (PPF) dataset, show that our approach outperforms the baselines, BART by five and T5 by six evaluation metrics. Our source codes and data are available online.

pdf bib
Intermediate-Task Transfer Learning for Peer Review Score Prediction
Panitan Muangkammuen | Fumiyo Fukumoto | Jiyi Li | Yoshimi Suzuki
Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics: Student Research Workshop

2022

pdf bib
Exploiting Labeled and Unlabeled Data via Transformer Fine-tuning for Peer-Review Score Prediction
Panitan Muangkammuen | Fumiyo Fukumoto | Jiyi Li | Yoshimi Suzuki
Findings of the Association for Computational Linguistics: EMNLP 2022

Automatic Peer-review Aspect Score Prediction (PASP) of academic papers can be a helpful assistant tool for both reviewers and authors. Most existing works on PASP utilize supervised learning techniques. However, the limited number of peer-review data deteriorates the performance of PASP. This paper presents a novel semi-supervised learning (SSL) method that incorporates the Transformer fine-tuning into the Γ-model, a variant of the Ladder network, to leverage contextual features from unlabeled data. Backpropagation simultaneously minimizes the sum of supervised and unsupervised cost functions, avoiding the need for layer-wise pre-training. The experimental results show that our model outperforms the supervised and naive semi-supervised learning baselines. Our source codes are available online.

2021

pdf bib
Abstract, Rationale, Stance: A Joint Model for Scientific Claim Verification
Zhiwei Zhang | Jiyi Li | Fumiyo Fukumoto | Yanming Ye
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing

Scientific claim verification can help the researchers to easily find the target scientific papers with the sentence evidence from a large corpus for the given claim. Some existing works propose pipeline models on the three tasks of abstract retrieval, rationale selection and stance prediction. Such works have the problems of error propagation among the modules in the pipeline and lack of sharing valuable information among modules. We thus propose an approach, named as ARSJoint, that jointly learns the modules for the three tasks with a machine reading comprehension framework by including claim information. In addition, we enhance the information exchanges and constraints among tasks by proposing a regularization term between the sentence attention scores of abstract retrieval and the estimated outputs of rational selection. The experimental results on the benchmark dataset SciFact show that our approach outperforms the existing works.

2020

pdf bib
DeepMet: A Reading Comprehension Paradigm for Token-level Metaphor Detection
Chuandong Su | Fumiyo Fukumoto | Xiaoxi Huang | Jiyi Li | Rongbo Wang | Zhiqun Chen
Proceedings of the Second Workshop on Figurative Language Processing

Machine metaphor understanding is one of the major topics in NLP. Most of the recent attempts consider it as classification or sequence tagging task. However, few types of research introduce the rich linguistic information into the field of computational metaphor by leveraging powerful pre-training language models. We focus a novel reading comprehension paradigm for solving the token-level metaphor detection task which provides an innovative type of solution for this task. We propose an end-to-end deep metaphor detection model named DeepMet based on this paradigm. The proposed approach encodes the global text context (whole sentence), local text context (sentence fragments), and question (query word) information as well as incorporating two types of part-of-speech (POS) features by making use of the advanced pre-training language model. The experimental results by using several metaphor datasets show that our model achieves competitive results in the second shared task on metaphor detection.

pdf bib
Multi-task Peer-Review Score Prediction
Jiyi Li | Ayaka Sato | Kazuya Shimura | Fumiyo Fukumoto
Proceedings of the First Workshop on Scholarly Document Processing

Automatic prediction on the peer-review aspect scores of academic papers can be a useful assistant tool for both reviewers and authors. To handle the small size of published datasets on the target aspect of scores, we propose a multi-task approach to leverage additional information from other aspects of scores for improving the performance of the target. Because one of the problems of building multi-task models is how to select the proper resources of auxiliary tasks and how to select the proper shared structures. We propose a multi-task shared structure encoding approach which automatically selects good shared network structures as well as good auxiliary resources. The experiments based on peer-review datasets show that our approach is effective and has better performance on the target scores than the single-task method and naive multi-task methods.

pdf bib
Multi-task Learning for Automated Essay Scoring with Sentiment Analysis
Panitan Muangkammuen | Fumiyo Fukumoto
Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing: Student Research Workshop

Automated Essay Scoring (AES) is a process that aims to alleviate the workload of graders and improve the feedback cycle in educational systems. Multi-task learning models, one of the deep learning techniques that have recently been applied to many NLP tasks, demonstrate the vast potential for AES. In this work, we present an approach for combining two tasks, sentiment analysis, and AES by utilizing multi-task learning. The model is based on a hierarchical neural network that learns to predict a holistic score at the document-level along with sentiment classes at the word-level and sentence-level. The sentiment features extracted from opinion expressions can enhance a vanilla holistic essay scoring, which mainly focuses on lexicon and text semantics. Our approach demonstrates that sentiment features are beneficial for some essay prompts, and the performance is competitive to other deep learning models on the Automated StudentAssessment Prize (ASAP) benchmark. TheQuadratic Weighted Kappa (QWK) is used to measure the agreement between the human grader’s score and the model’s prediction. Ourmodel produces a QWK of 0.763.

pdf bib
Semi-Automatic Construction and Refinement of an Annotated Corpus for a Deep Learning Framework for Emotion Classification
Jiajun Xu | Kyosuke Masuda | Hiromitsu Nishizaki | Fumiyo Fukumoto | Yoshimi Suzuki
Proceedings of the Twelfth Language Resources and Evaluation Conference

In the case of using a deep learning (machine learning) framework for emotion classification, one significant difficulty faced is the requirement of building a large, emotion corpus in which each sentence is assigned emotion labels. As a result, there is a high cost in terms of time and money associated with the construction of such a corpus. Therefore, this paper proposes a method of creating a semi-automatically constructed emotion corpus. For the purpose of this study sentences were mined from Twitter using some emotional seed words that were selected from a dictionary in which the emotion words were well-defined. Tweets were retrieved by one emotional seed word, and the retrieved sentences were assigned emotion labels based on the emotion category of the seed word. It was evident from the findings that the deep learning-based emotion classification model could not achieve high levels of accuracy in emotion classification because the semi-automatically constructed corpus had many errors when assigning emotion labels. In this paper, therefore, an approach for improving the quality of the emotion labels by automatically correcting the errors of emotion labels is proposed and tested. The experimental results showed that the proposed method worked well, and the classification accuracy rate was improved to 55.1% from 44.9% on the Twitter emotion classification task.

pdf bib
A Neural Local Coherence Analysis Model for Clarity Text Scoring
Panitan Muangkammuen | Sheng Xu | Fumiyo Fukumoto | Kanda Runapongsa Saikaew | Jiyi Li
Proceedings of the 28th International Conference on Computational Linguistics

Local coherence relation between two phrases/sentences such as cause-effect and contrast gives a strong influence of whether a text is well-structured or not. This paper follows the assumption and presents a method for scoring text clarity by utilizing local coherence between adjacent sentences. We hypothesize that the contextual features of coherence relations learned by utilizing different data from the target training data are also possible to discriminate well-structured of the target text and thus help to score the text clarity. We propose a text clarity scoring method that utilizes local coherence analysis with an out-domain setting, i.e. the training data for the source and target tasks are different from each other. The method with language model pre-training BERT firstly trains the local coherence model as an auxiliary manner and then re-trains it together with clarity text scoring model. The experimental results by using the PeerRead benchmark dataset show the improvement compared with a single model, scoring text clarity model. Our source codes are available online.

pdf bib
HSCNN: A Hybrid-Siamese Convolutional Neural Network for Extremely Imbalanced Multi-label Text Classification
Wenshuo Yang | Jiyi Li | Fumiyo Fukumoto | Yanming Ye
Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

The data imbalance problem is a crucial issue for the multi-label text classification. Some existing works tackle it by proposing imbalanced loss objectives instead of the vanilla cross-entropy loss, but their performances remain limited in the cases of extremely imbalanced data. We propose a hybrid solution which adapts general networks for the head categories, and few-shot techniques for the tail categories. We propose a Hybrid-Siamese Convolutional Neural Network (HSCNN) with additional technical attributes, i.e., a multi-task architecture based on Single and Siamese networks; a category-specific similarity in the Siamese structure; a specific sampling method for training HSCNN. The results using two benchmark datasets and three loss objectives show that our method can improve the performance of Single networks with diverse loss objectives on the tail or entire categories.

2019

pdf bib
Text Categorization by Learning Predominant Sense of Words as Auxiliary Task
Kazuya Shimura | Jiyi Li | Fumiyo Fukumoto
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

Distributions of the senses of words are often highly skewed and give a strong influence of the domain of a document. This paper follows the assumption and presents a method for text categorization by leveraging the predominant sense of words depending on the domain, i.e., domain-specific senses. The key idea is that the features learned from predominant senses are possible to discriminate the domain of the document and thus improve the overall performance of text categorization. We propose multi-task learning framework based on the neural network model, transformer, which trains a model to simultaneously categorize documents and predicts a predominant sense for each word. The experimental results using four benchmark datasets show that our method is comparable to the state-of-the-art categorization approach, especially our model works well for categorization of multi-label documents.

pdf bib
A Dataset of Crowdsourced Word Sequences: Collections and Answer Aggregation for Ground Truth Creation
Jiyi Li | Fumiyo Fukumoto
Proceedings of the First Workshop on Aggregating and Analysing Crowdsourced Annotations for NLP

The target outputs of many NLP tasks are word sequences. To collect the data for training and evaluating models, the crowd is a cheaper and easier to access than the oracle. To ensure the quality of the crowdsourced data, people can assign multiple workers to one question and then aggregate the multiple answers with diverse quality into a golden one. How to aggregate multiple crowdsourced word sequences with diverse quality is a curious and challenging problem. People need a dataset for addressing this problem. We thus create a dataset (CrowdWSA2019) which contains the translated sentences generated from multiple workers. We provide three approaches as the baselines on the task of extractive word sequence aggregation. Specially, one of them is an original one we propose which models the reliability of workers. We also discuss some issues on ground truth creation of word sequences which can be addressed based on this dataset.

2018

pdf bib
HFT-CNN: Learning Hierarchical Category Structure for Multi-label Short Text Categorization
Kazuya Shimura | Jiyi Li | Fumiyo Fukumoto
Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing

We focus on the multi-label categorization task for short texts and explore the use of a hierarchical structure (HS) of categories. In contrast to the existing work using non-hierarchical flat model, the method leverages the hierarchical relations between the pre-defined categories to tackle the data sparsity problem. The lower the HS level, the less the categorization performance. Because the number of training data per category in a lower level is much smaller than that in an upper level. We propose an approach which can effectively utilize the data in the upper levels to contribute the categorization in the lower levels by applying the Convolutional Neural Network (CNN) with a fine-tuning technique. The results using two benchmark datasets show that proposed method, Hierarchical Fine-Tuning based CNN (HFT-CNN) is competitive with the state-of-the-art CNN based methods.

2015

pdf bib
Learning Timeline Difference for Text Categorization
Fumiyo Fukumoto | Yoshimi Suzuki
Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing

2014

pdf bib
Annotating the Focus of Negation in Japanese Text
Suguru Matsuyoshi | Ryo Otsuki | Fumiyo Fukumoto
Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)

This paper proposes an annotation scheme for the focus of negation in Japanese text. Negation has its scope and the focus within the scope. The scope of negation is the part of the sentence that is negated; the focus is the part of the scope that is most prominently or explicitly negated. In natural language processing, correct interpretation of negated statements requires precise detection of the focus of negation in the statements. As a foundation for developing a negation focus detector for Japanese, we have annotated textdata of “Rakuten Travel: User review data” and the newspaper subcorpus of the “Balanced Corpus of Contemporary Written Japanese” with labels proposed in our annotation scheme. We report 1,327 negation cues and the foci in the corpora, and present classification of these foci based on syntactic types and semantic types. We also propose a system for detecting the focus of negation in Japanese using 16 heuristic rules and report the performance of the system.

pdf bib
Detection of Topic and its Extrinsic Evaluation Through Multi-Document Summarization
Yoshimi Suzuki | Fumiyo Fukumoto
Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)

pdf bib
The Effect of Temporal-based Term Selection for Text Classification
Fumiyo Fukumoto | Shougo Ushiyama | Yoshimi Suzuki | Suguru Matsuyoshi
Proceedings of the Australasian Language Technology Association Workshop 2014

2013

pdf bib
Text Classification from Positive and Unlabeled Data using Misclassified Data Correction
Fumiyo Fukumoto | Yoshimi Suzuki | Suguru Matsuyoshi
Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)

2012

pdf bib
Exploiting Discourse Relations between Sentences for Text Clustering
Nik Adilah Hanin Binti Zahri | Fumiyo Fukumoto | Suguru Matsuyoshi
Proceedings of the Workshop on Advances in Discourse Analysis and its Computational Aspects

2011

pdf bib
Cluster Labelling based on Concepts in a Machine-Readable Dictionary
Fumiyo Fukumoto | Yoshimi Suzuki
Proceedings of 5th International Joint Conference on Natural Language Processing

pdf bib
Identification of Domain-Specific Senses in a Machine-Readable Dictionary
Fumiyo Fukumoto | Yoshimi Suzuki
Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies

2010

pdf bib
Eliminating Redundancy by Spectral Relaxation for Multi-Document Summarization
Fumiyo Fukumoto | Akina Sakai | Yoshimi Suzuki
Proceedings of TextGraphs-5 - 2010 Workshop on Graph-based Methods for Natural Language Processing

2009

pdf bib
Classifying Japanese Polysemous Verbs based on Fuzzy C-means Clustering
Yoshimi Suzuki | Fumiyo Fukumoto
Proceedings of the 2009 Workshop on Graph-based Methods for Natural Language Processing (TextGraphs-4)

2008

pdf bib
Graph-Based Clustering for Semantic Classification of Onomatopoetic Words
Kenichi Ichioka | Fumiyo Fukumoto
Coling 2008: Proceedings of the 3rd Textgraphs workshop on Graph-based Algorithms for Natural Language Processing

pdf bib
Retrieving Bilingual Verb-Noun Collocations by Integrating Cross-Language Category Hierarchies
Fumiyo Fukumoto | Yoshimi Suzuki | Kazuyuki Yamashita
Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008)

2006

pdf bib
Using Bilingual Comparable Corpora and Semi-supervised Clustering for Topic Tracking
Fumiyo Fukumoto | Yoshimi Suzuki
Proceedings of the COLING/ACL 2006 Main Conference Poster Sessions

2005

pdf bib
Topic Tracking Based on Linguistic Features
Fumiyo Fukumoto | Yusuke Yamaji
Second International Joint Conference on Natural Language Processing: Full Papers

2004

pdf bib
A Comparison of Manual and Automatic Constructions of Category Hierarchy for Classifying Large Corpora
Fumiyo Fukumoto | Yoshimi Suzuki
Proceedings of the Eighth Conference on Computational Natural Language Learning (CoNLL-2004) at HLT-NAACL 2004

pdf bib
Correcting Category Errors in Text Classification
Fumiyo Fukumoto | Yoshimi Suzuki
COLING 2004: Proceedings of the 20th International Conference on Computational Linguistics

2002

pdf bib
Detecting Shifts in News Stories for Paragraph Extraction
Fumiyo Fukumoto | Yoshimi Suzuki
COLING 2002: The 19th International Conference on Computational Linguistics

pdf bib
Topic Tracking using Subject Templates and Clustering Positive Training Instances
Yoshimi Suzuki | Fumiyo Fukumoto | Yoshihiro Sekiguchi
COLING 2002: The 17th International Conference on Computational Linguistics: Project Notes

pdf bib
Manipulating Large Corpora for Text Classification
Fumiyo Fukumoto | Yoshimi Suzuki
Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing (EMNLP 2002)

2000

pdf bib
Extracting Key Paragraph based on Topic and Event Detection Towards Multi-Document Summarization
Fumiyo Fukumoto | Yoshimi Suzuki
NAACL-ANLP 2000 Workshop: Automatic Summarization

1999

pdf bib
Word Sense Disambiguation in Untagged Text based on Term Weight Learning
Fumiyo Fukumoto | Yoshimi Suzuki
Ninth Conference of the European Chapter of the Association for Computational Linguistics

1998

pdf bib
Keyword Extraction using Term-Domain Interdependence for Dictation of Radio News
Yoshimi Suzuki | Fumiyo Fukumoto | Yoshihiro Sekiguchi
COLING 1998 Volume 2: The 17th International Conference on Computational Linguistics

pdf bib
An Empirical Approach to Text Categorization Based on Term Weight Learning
Fumiyo Fukumoto | Yoshimi Suzuki
Proceedings of the Third Conference on Empirical Methods for Natural Language Processing

pdf bib
Keyword Extraction using Term-Domain Interdependence for Dictation of Radio News
Yoshimi Suzuki | Fumiyo Fukumoto | Yoshihiro Sekiguchi
36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics, Volume 2

1997

pdf bib
An Automatic Extraction of Key Paragraphs Based on Context Dependency
Fumiyo Fukumoto | Yoshimi Suzuki | Jun’ichi Fukumoto
Fifth Conference on Applied Natural Language Processing

1996

pdf bib
An Automatic Clustering of Articles Using Dictionary Definitions
Fumiyo Fukumoto | Yoshimi Suzuki
COLING 1996 Volume 1: The 16th International Conference on Computational Linguistics

1994

pdf bib
Automatic Recognition of Verbal Polysemy
Fumiyo Fukumoto | Jun’ichi Tsujii
COLING 1994 Volume 2: The 15th International Conference on Computational Linguistics