TTL: transformer-based two-phase transfer learning for cross-lingual news event detection
Hettiarachchi, Hansi and Adedoyin-Olowe, Mariam and Bhogal, Jagdev and Gaber, Mohamed Medhat (2023) TTL: transformer-based two-phase transfer learning for cross-lingual news event detection. International Journal of Machine Learning and Cybernetics. ISSN 1868-8071
|
Text
s13042-023-01795-9.pdf - Published Version Available under License Creative Commons Attribution. Download (3MB) |
Abstract
Today, we have access to a vast data amount, especially on the internet. Online news agencies play a vital role in this data generation, but most of their data is unstructured, requiring an enormous effort to extract important information. Thus, automated intelligent event detection mechanisms are invaluable to the community. In this research, we focus on identifying event details at the sentence and token levels from news articles, considering their fine granularity. Previous research has proposed various approaches ranging from traditional machine learning to deep learning, targeting event detection at these levels. Among these approaches, transformer-based approaches performed best, utilising transformers’ transferability and context awareness, and achieved state-of-the-art results. However, they considered sentence and token level tasks as separate tasks even though their interconnections can be utilised for mutual task improvements. To fill this gap, we propose a novel learning strategy named Two-phase Transfer Learning (TTL) based on transformers, which allows the model to utilise the knowledge from a task at a particular data granularity for another task at different data granularity, and evaluate its performance in sentence and token level event detection. Also, we empirically evaluate how the event detection performance can be improved for different languages (high- and low-resource), involving monolingual and multilingual pre-trained transformers and language-based learning strategies along with the proposed learning strategy. Our findings mainly indicate the effectiveness of multilingual models in low-resource language event detection. Also, TTL can further improve model performance, depending on the involved tasks’ learning order and their relatedness concerning final predictions.
| Item Type: | Article | ||||||
|---|---|---|---|---|---|---|---|
| Identification Number: | https://doi.org/10.1007/s13042-023-01795-9 | ||||||
| Dates: |
|
||||||
| Uncontrolled Keywords: | transformer, two-phase transfer learning, cross-lingual event detection, news media | ||||||
| Subjects: | CAH11 - computing > CAH11-01 - computing > CAH11-01-01 - computer science | ||||||
| Divisions: | Faculty of Computing, Engineering and the Built Environment > School of Computing and Digital Technology | ||||||
| Depositing User: | Gemma Tonks | ||||||
| Date Deposited: | 15 Mar 2023 09:27 | ||||||
| Last Modified: | 15 Mar 2023 09:27 | ||||||
| URI: | https://www.open-access.bcu.ac.uk/id/eprint/14255 |
Actions (login required)
 |
View Item |
Tools
Tools