TTL: transformer-based two-phase transfer learning for cross-lingual news event detection

Hettiarachchi, Hansi and Adedoyin-Olowe, Mariam and Bhogal, Jagdev and Gaber, Mohamed Medhat (2023) TTL: transformer-based two-phase transfer learning for cross-lingual news event detection. International Journal of Machine Learning and Cybernetics. ISSN 1868-8071

Preview

Text s13042-023-01795-9.pdf - Published Version Available under License Creative Commons Attribution. Download (3MB)

Abstract

Today, we have access to a vast data amount, especially on the internet. Online news agencies play a vital role in this data generation, but most of their data is unstructured, requiring an enormous effort to extract important information. Thus, automated intelligent event detection mechanisms are invaluable to the community. In this research, we focus on identifying event details at the sentence and token levels from news articles, considering their fine granularity. Previous research has proposed various approaches ranging from traditional machine learning to deep learning, targeting event detection at these levels. Among these approaches, transformer-based approaches performed best, utilising transformers’ transferability and context awareness, and achieved state-of-the-art results. However, they considered sentence and token level tasks as separate tasks even though their interconnections can be utilised for mutual task improvements. To fill this gap, we propose a novel learning strategy named Two-phase Transfer Learning (TTL) based on transformers, which allows the model to utilise the knowledge from a task at a particular data granularity for another task at different data granularity, and evaluate its performance in sentence and token level event detection. Also, we empirically evaluate how the event detection performance can be improved for different languages (high- and low-resource), involving monolingual and multilingual pre-trained transformers and language-based learning strategies along with the proposed learning strategy. Our findings mainly indicate the effectiveness of multilingual models in low-resource language event detection. Also, TTL can further improve model performance, depending on the involved tasks’ learning order and their relatedness concerning final predictions.

Item Type:	Article
Identification Number:	10.1007/s13042-023-01795-9
Dates:	Date Event 31 January 2023 Accepted 8 March 2023 Published Online
Uncontrolled Keywords:	transformer, two-phase transfer learning, cross-lingual event detection, news media
Subjects:	CAH11 - computing > CAH11-01 - computing > CAH11-01-01 - computer science
Divisions:	Faculty of Computing, Engineering and the Built Environment > College of Computing
Depositing User:	Gemma Tonks
Date Deposited:	15 Mar 2023 09:27
Last Modified:	15 Mar 2023 09:27
URI:	https://www.open-access.bcu.ac.uk/id/eprint/14255

Actions (login required)

View Item