Research Repository

Unified Transformer Multi-Task Learning for Intent Classification With Entity Recognition

Benayas, Alberto and Hashempour, Reyhaneh and Rumble, Damian and Jameel, Shoaib and De Amorim, Renato Cordeiro (2021) 'Unified Transformer Multi-Task Learning for Intent Classification With Entity Recognition.' IEEE Access, 9. pp. 147306-147314. ISSN 2169-3536

Unified_Transformer_Multi-Task_Learning_for_Intent_Classification_With_Entity_Recognition.pdf - Published Version
Available under License Creative Commons Attribution.

Download (1MB) | Preview


Intent classification (IC) and Named Entity Recognition (NER) are arguably the two main components needed to build a Natural Language Understanding (NLU) engine, which is a main component of conversational agents. The IC and NER components are closely intertwined and the entities are often connected to the underlying intent. Current research has primarily focused to model IC and NER as two separate units, which results in error propagation, and thus, sub-optimal performance. In this paper, we propose a simple yet effective novel framework for NLU where the parameters of the IC and the NER models are jointly trained in a consolidated parameter space. Text semantic representations are obtained from popular pre-trained contextual language models, which are fine-tuned for our task, and these parameters are propagated to other deep neural layers in our framework leading to a faithful unified modelling of the IC and NER parameters. The overall framework results in a faithful parameter sharing when the training is underway, leading to a more coherent learning. Experiments on two public datasets, ATIS and SNIPS, show that our model outperforms other methods by a noticeable margin. On the SNIPS dataset, we obtain a 1.42% improvement in NER in terms of the F1 score, and 1% improvement in intent accuracy score. On ATIS, we achieve 1.54% improvement in intent accuracy score. We also present qualitative results to showcase the effectiveness of our model.

Item Type: Article
Uncontrolled Keywords: Intent classification; named entity recognition; multi-task learning; transfer learning
Divisions: Faculty of Science and Health
Faculty of Science and Health > Computer Science and Electronic Engineering, School of
SWORD Depositor: Elements
Depositing User: Elements
Date Deposited: 26 Jan 2022 15:24
Last Modified: 26 Jan 2022 15:24

Actions (login required)

View Item View Item