2023
pdf
bib
abs
Statistical Measures for Readability Assessment
Mohammed Attia
|
Younes Samih
|
Yo Ehara
Proceedings of the Joint 3rd International Conference on Natural Language Processing for Digital Humanities and 8th International Workshop on Computational Linguistics for Uralic Languages
Neural models and deep learning techniques have predominantly been used in many tasks of natural language processing (NLP), including automatic readability assessment (ARA). They apply deep transfer learning and enjoy high accuracy. However, most of the models still cannot leverage long dependence such as inter-sentential topic-level or document-level information because of their structure and computational cost. Moreover, neural models usually have low interpretability. In this paper, we propose a generalization of passage-level, corpus-level, document-level and topic-level features. In our experiments, we show the effectiveness of “Statistical Lexical Spread (SLS)” features when combined with IDF (inverse document frequency) and TF-IDF (term frequency–inverse document frequency), which adds a topological perspective (inter-document) to readability to complement the typological approaches (intra-document) used in traditional readability formulas. Interestingly, simply adding these features in BERT models outperformed state-of-the-art systems trained on a large number of hand-crafted features derived from heavy linguistic processing. In analysis, we show that SLS is also easy-to-interpret because SLS computes lexical features, which appear explicitly in texts, compared to parameters in neural models.
2021
pdf
bib
abs
Evaluation of Unsupervised Automatic Readability Assessors Using Rank Correlations
Yo Ehara
Proceedings of the 2nd Workshop on Evaluation and Comparison of NLP Systems
Automatic readability assessment (ARA) is the task of automatically assessing readability with little or no human supervision. ARA is essential for many second language acquisition applications to reduce the workload of annotators, who are usually language teachers. Previous unsupervised approaches manually searched textual features that correlated well with readability labels, such as perplexity scores of large language models. This paper argues that, to evaluate an assessors’ performance, rank-correlation coefficients should be used instead of Pearson’s correlation coefficient (𝜌). In the experiments, we show that its performance can be easily underestimated using Pearson’s 𝜌, which is significantly affected by the linearity of the output readability scores. We also propose a lightweight unsupervised readability assessor that achieved the best performance in both the rank correlations and Pearson’s 𝜌 among all unsupervised assessors compared.
pdf
bib
Readability and Linearity
Yo Ehara
Proceedings of the 35th Pacific Asia Conference on Language, Information and Computation
pdf
bib
abs
To What Extent Does Lexical Normalization Help English-as-a-Second Language Learners to Read Noisy English Texts?
Yo Ehara
Proceedings of the Seventh Workshop on Noisy User-generated Text (W-NUT 2021)
How difficult is it for English-as-a-second language (ESL) learners to read noisy English texts? Do ESL learners need lexical normalization to read noisy English texts? These questions may also affect community formation on social networking sites where differences can be attributed to ESL learners and native English speakers. However, few studies have addressed these questions. To this end, we built highly accurate readability assessors to evaluate the readability of texts for ESL learners. We then applied these assessors to noisy English texts to further assess the readability of the texts. The experimental results showed that although intermediate-level ESL learners can read most noisy English texts in the first place, lexical normalization significantly improves the readability of noisy English texts for ESL learners.
pdf
bib
abs
To What Extent Can English-as-a-Second Language Learners Read Economic News Texts?
Yo Ehara
Proceedings of the Third Workshop on Economics and Natural Language Processing
In decision making in the economic field, an especially important requirement is to rapidly understand news to absorb ever-changing economic situations. Given that most economic news is written in English, the ability to read such information without waiting for a translation is particularly valuable in economics in contrast to other fields. In consideration of this issue, this research investigated the extent to which non-native English speakers are able to read economic news to make decisions accordingly – an issue that has been rarely addressed in previous studies. Using an existing standard dataset as training data, we created a classifier that automatically evaluates the readability of text with high accuracy for English learners. Our assessment of the readability of an economic news corpus revealed that most news texts can be read by intermediate English learners. We also found that in some cases, readability varies considerably depending on the knowledge of certain words specific to the economic field.
2020
pdf
bib
abs
Interpreting Neural CWI Classifiers’ Weights as Vocabulary Size
Yo Ehara
Proceedings of the Fifteenth Workshop on Innovative Use of NLP for Building Educational Applications
Complex Word Identification (CWI) is a task for the identification of words that are challenging for second-language learners to read. Even though the use of neural classifiers is now common in CWI, the interpretation of their parameters remains difficult. This paper analyzes neural CWI classifiers and shows that some of their parameters can be interpreted as vocabulary size. We present a novel formalization of vocabulary size measurement methods that are practiced in the applied linguistics field as a kind of neural classifier. We also contribute to building a novel dataset for validating vocabulary testing and readability via crowdsourcing.
2019
pdf
bib
An Approach to Summarize Concordancers’ Lists Visually to Support Language Learners in UnderstandingWord Usages
Yo Ehara
Proceedings of the 1st Workshop on Interactive Natural Language Technology for Explainable Artificial Intelligence (NL4XAI 2019)
2018
pdf
bib
Building an English Vocabulary Knowledge Dataset of Japanese English-as-a-Second-Language Learners Using Crowdsourcing
Yo Ehara
Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)
2017
pdf
bib
abs
Language-Independent Prediction of Psycholinguistic Properties of Words
Yo Ehara
Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 2: Short Papers)
The psycholinguistic properties of words, namely, word familiarity, age of acquisition, concreteness, and imagery, have been reported to be effective for educational natural language-processing tasks. Previous studies on predicting the values of these properties rely on language-dependent features. This paper is the first to propose a practical language-independent method for predicting such values by using only a large raw corpus in a language. Through experiments, our method successfully predicted the values of these properties in two languages. The results for English were competitive with the reported accuracy achieved using features specific to English.
2016
pdf
bib
abs
Generating Video Description using Sequence-to-sequence Model with Temporal Attention
Natsuda Laokulrat
|
Sang Phan
|
Noriki Nishida
|
Raphael Shu
|
Yo Ehara
|
Naoaki Okazaki
|
Yusuke Miyao
|
Hideki Nakayama
Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers
Automatic video description generation has recently been getting attention after rapid advancement in image caption generation. Automatically generating description for a video is more challenging than for an image due to its temporal dynamics of frames. Most of the work relied on Recurrent Neural Network (RNN) and recently attentional mechanisms have also been applied to make the model learn to focus on some frames of the video while generating each word in a describing sentence. In this paper, we focus on a sequence-to-sequence approach with temporal attention mechanism. We analyze and compare the results from different attention model configuration. By applying the temporal attention mechanism to the system, we can achieve a METEOR score of 0.310 on Microsoft Video Description dataset, which outperformed the state-of-the-art system so far.
2015
pdf
bib
Risk-aware distribution of SMT outputs for translation of documents targeting many anonymous readers
Yo Ehara
|
Masao Utiyama
|
Eiichiro Sumita
Proceedings of the 12th International Workshop on Spoken Language Translation: Papers
2014
pdf
bib
Formalizing Word Sampling for Vocabulary Prediction as Graph-based Active Learning
Yo Ehara
|
Yusuke Miyao
|
Hidekazu Oiwa
|
Issei Sato
|
Hiroshi Nakagawa
Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)
2013
pdf
bib
Understanding seed selection in bootstrapping
Yo Ehara
|
Issei Sato
|
Hidekazu Oiwa
|
Hiroshi Nakagawa
Proceedings of TextGraphs-8 Graph-based Methods for Natural Language Processing
2012
pdf
bib
Mining Words in the Minds of Second Language Learners: Learner-Specific Word Difficulty
Yo Ehara
|
Issei Sato
|
Hidekazu Oiwa
|
Hiroshi Nakagawa
Proceedings of COLING 2012
2008
pdf
bib
Multilingual Text Entry using Automatic Language Detection
Yo Ehara
|
Kumiko Tanaka-Ishii
Proceedings of the Third International Joint Conference on Natural Language Processing: Volume-I