RedPenNet for Grammatical Error Correction: Outputs to Tokens, Attentions to Spans

Bohdan Didenko, Andrii Sameliuk


Abstract
The text editing tasks, including sentence fusion, sentence splitting and rephrasing, text simplification, and Grammatical Error Correction (GEC), share a common trait of dealing with highly similar input and output sequences. This area of research lies at the intersection of two well-established fields: (i) fully autoregressive sequence-to-sequence approaches commonly used in tasks like Neural Machine Translation (NMT) and (ii) sequence tagging techniques commonly used to address tasks such as Part-of-speech tagging, Named-entity recognition (NER), and similar. In the pursuit of a balanced architecture, researchers have come up with numerous imaginative and unconventional solutions, which we’re discussing in the Related Works section. Our approach to addressing text editing tasks is called RedPenNet and is aimed at reducing architectural and parametric redundancies presented in specific Sequence-To-Edits models, preserving their semi-autoregressive advantages. Our models achieve F0.5 scores of 77.60 on the BEA-2019 (test), which can be considered as state-of-the-art the only exception for system combination (Qorib et al., 2022) and 67.71 on the UAGEC+Fluency (test) benchmarks. This research is being conducted in the context of the UNLP 2023 workshop, where it will be presented as a paper for the Shared Task in Grammatical Error Correction (GEC) for Ukrainian. This study aims to apply the RedPenNet approach to address the GEC problem in the Ukrainian language.
Anthology ID:
2023.unlp-1.15
Volume:
Proceedings of the Second Ukrainian Natural Language Processing Workshop (UNLP)
Month:
May
Year:
2023
Address:
Dubrovnik, Croatia
Editor:
Mariana Romanyshyn
Venue:
UNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
121–131
Language:
URL:
https://aclanthology.org/2023.unlp-1.15
DOI:
10.18653/v1/2023.unlp-1.15
Bibkey:
Cite (ACL):
Bohdan Didenko and Andrii Sameliuk. 2023. RedPenNet for Grammatical Error Correction: Outputs to Tokens, Attentions to Spans. In Proceedings of the Second Ukrainian Natural Language Processing Workshop (UNLP), pages 121–131, Dubrovnik, Croatia. Association for Computational Linguistics.
Cite (Informal):
RedPenNet for Grammatical Error Correction: Outputs to Tokens, Attentions to Spans (Didenko & Sameliuk, UNLP 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.unlp-1.15.pdf
Video:
 https://aclanthology.org/2023.unlp-1.15.mp4