Ubiquitous Multimodality as a Tool in Violin Performance Classification

Wilson, William and Granieri, Niccolo and Ali-MacLachlan, Islah (2023) Ubiquitous Multimodality as a Tool in Violin Performance Classification. In: 4th International Symposium on the Internet of Sounds, 26th - 27th October 2023, Pisa, Italy.

Preview

Text Ubiquitous_Multimodality_as_a_Tool_in_Violin_Performance_Classification.pdf - Accepted Version Download (1MB)

Abstract

Through integrated sensors, wearable devices such as fitness trackers and smart-watches provide convenient interfaces by which multimodal time-series data may be recorded. Fostering multimodality in data collection allows for the observation of recorded actions, exercises or performances with consideration towards multiple transpiring aspects. This paper details an exploration of machine-learning based classification upon a dataset of audio-gestural violin recordings, collated through the use of a purpose-built smartwatch application. This interface allowed for the recording of synchronous gestural and audio data, which proved well-suited towards classification by deep neural networks (DNNs). Recordings were segmented into individual bow strokes, these were classified through completion of three tasks: Participant Recognition, Articulation Recognition, and Scale Recognition. Higher participant classification accuracies were observed through the use of lone gestural data, while multi-input deep neural networks (MI-DNNs) achieved varying increases in accuracy during completion of the latter two tasks, through concatenation of separate audio and gestural subnetworks. Across tasks and across network architectures, test-classification accuracies ranged between 63.83% and 99.67%. Articulation Recognition accuracies were consistently high, averaging 99.37%.

Item Type:	Conference or Workshop Item (Paper)
Identification Number:	10.1109/IEEECONF59510.2023.10335435
Dates:	Date Event 1 October 2023 Accepted 4 December 2023 Published Online
Uncontrolled Keywords:	Performance evaluation, Wearable Health Monitoring Systems, Music, Artificial neural networks, Computer architecture, Network architecture, Mobile handsets
Subjects:	CAH11 - computing > CAH11-01 - computing > CAH11-01-01 - computer science
Divisions:	Faculty of Computing, Engineering and the Built Environment > College of Computing
Depositing User:	Gemma Tonks
Date Deposited:	15 Feb 2024 13:29
Last Modified:	15 Feb 2024 13:30
URI:	https://www.open-access.bcu.ac.uk/id/eprint/15248

Actions (login required)

View Item