Exploring Knowledge Transfer in Clinical Natural Language Processing Tasks

Clinical domain NLP tasks generally involve information extraction, text classification, and text summary from clinical notes or electronic health records. However, due to the limited data resource, many NLP tasks in clinical domain have not been extensively studied as of the general domain NLP tasks. As transfer learning has achieved great success in many NLP applications and it especially benefits in applications where there is shared knowledge between the sub-tasks while data is limited, this project aims to gain insights on knowledge transfer in multiple clinical NLP tasks and analyze the impact of joint learning on the performance of individual task. Our main contributions are twofold: (1) we train a multi-task model based on clinical-BERT on varieties of NLP tasks including named-entity recognition (NER), sentence entailment, and text classification, and analyze the performance of the model with different tasks settings; (2) we train a NER model with different entity label annotations and investigate whether there exists knowledge transfers between different entity labels within the same dataset. Our results show that multi-task model achieves improved result on tasks that have shared knowledge (e.g., same task type or similar data distribution) and adding different entity annotations can benefit model performance on named-entity extraction. These fundamental findings shed lights on how to utilize transfer learning to improve clinical domain NLP applications.