Extended Multilingual Protest News Detection - Shared Task 1, CASE 2021 and 2022

Hürriyeto˘glu, Ali and Mutlu, Osman and Duruşan, Fırat and Uca, Onur and Gürel, Alaeddin and Radford, Benjamin J and Dai, Yaoyao and Hettiarachchi, Hansi and Stoehr, Niklas and Nomoto, Tadashi and Slavcheva, Milena and Vargas, Francielle and Javid, Aaqib and Beyhan, Fatih and Yörük, Erdem (2022) Extended Multilingual Protest News Detection - Shared Task 1, CASE 2021 and 2022. In: The 2022 Conference on Empirical Methods in Natural Language Processing, 7th - 8th December 2022, Abu Dhabi, United Arab Emirates (Hybrid).

Preview

Text 2022.case-1.31v2.pdf - Published Version Available under License Creative Commons Attribution. Download (107kB)

Abstract

We report results of the CASE 2022 Shared Task 1 on Multilingual Protest Event Detection. This task is a continuation of CASE 2021 that consists of four subtasks that are i) document classification, ii) sentence classification, iii) event sentence coreference identification, and iv) event extraction. The CASE 2022 extension consists of expanding the test data with more data in previously available languages, namely, English, Hindi, Portuguese, and Spanish, and adding new test data in Mandarin, Turkish, and Urdu for Sub-task 1, document classification. The training data from CASE 2021 in English, Portuguese and Spanish were utilized. Therefore, predicting document labels in Hindi, Mandarin, Turkish, and Urdu occurs in a zero-shot setting. The CASE 2022 workshop accepts reports on systems developed for predicting test data of CASE 2021 as well. We observe that the best systems submitted by CASE 2022 participants achieve between 79.71 and 84.06 F1-macro for new languages in a zero-shot setting. The winning approaches are mainly ensembling models and merging data in multiple languages. The best two submissions on CASE 2021 data outperform submissions from last year for Subtask 1 and Subtask 2 in all languages. Only the following scenarios were not outperformed by new submissions on CASE 2021: Subtask 3 Portuguese & Subtask 4 English.

Item Type:	Conference or Workshop Item (Paper)
Dates:	Date Event 9 October 2022 Accepted 31 December 2022 Published Online
Subjects:	CAH11 - computing > CAH11-01 - computing > CAH11-01-01 - computer science
Divisions:	Faculty of Computing, Engineering and the Built Environment > College of Computing
Depositing User:	Hansi Hettiarachchi
Date Deposited:	25 May 2023 12:45
Last Modified:	25 May 2023 12:45
URI:	https://www.open-access.bcu.ac.uk/id/eprint/14399

Actions (login required)

View Item