Progressive Transformer-Based Generation of Radiology Reports

Farhad Nooralahzadeh, Nicolas Perez Gonzalez, Thomas Frauenfelder, Koji Fujimoto, Michael Krauthammer


Abstract
Inspired by Curriculum Learning, we propose a consecutive (i.e., image-to-text-to-text) generation framework where we divide the problem of radiology report generation into two steps. Contrary to generating the full radiology report from the image at once, the model generates global concepts from the image in the first step and then reforms them into finer and coherent texts using transformer-based architecture. We follow the transformer-based sequence-to-sequence paradigm at each step. We improve upon the state-of-the-art on two benchmark datasets.
Anthology ID:
2021.findings-emnlp.241
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2021
Month:
November
Year:
2021
Address:
Punta Cana, Dominican Republic
Editors:
Marie-Francine Moens, Xuanjing Huang, Lucia Specia, Scott Wen-tau Yih
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
2824–2832
Language:
URL:
https://aclanthology.org/2021.findings-emnlp.241
DOI:
10.18653/v1/2021.findings-emnlp.241
Bibkey:
Cite (ACL):
Farhad Nooralahzadeh, Nicolas Perez Gonzalez, Thomas Frauenfelder, Koji Fujimoto, and Michael Krauthammer. 2021. Progressive Transformer-Based Generation of Radiology Reports. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 2824–2832, Punta Cana, Dominican Republic. Association for Computational Linguistics.
Cite (Informal):
Progressive Transformer-Based Generation of Radiology Reports (Nooralahzadeh et al., Findings 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.findings-emnlp.241.pdf
Video:
 https://aclanthology.org/2021.findings-emnlp.241.mp4
Code
 uzh-dqbm-cmi/argon
Data
CheXpert