Drum Synthesis and Rhythmic Transformation with Adversarial Autoencoders

Tomczak, Maciej and Goto, Masataka and Hockman, Jason (2020) Drum Synthesis and Rhythmic Transformation with Adversarial Autoencoders. MM '20: Proceedings of the 28th ACM International Conference on Multimedia. pp. 2427-2435.

Tomczak_ACMMM20.pdf - Accepted Version

Download (1MB)


Creative rhythmic transformations of musical audio refer to automated methods for manipulation of temporally-relevant sounds in time. This paper presents a method for joint synthesis and rhythm transformation of drum sounds through the use of adversarial autoencoders (AAE). Users may navigate both the timbre and rhythm of drum patterns in audio recordings through expressive control over a low-dimensional latent space. The model is based on an AAE with Gaussian mixture latent distributions that introduce rhythmic pattern conditioning to represent a wide variety of drum performances. The AAE is trained on a dataset of bar-length segments of percussion recordings, along with their clustered rhythmic pattern labels. The decoder is conditioned during adversarial training for mixing of data-driven rhythmic and timbral properties. The system is trained with over 500000 bars from 5418 tracks in popular datasets covering various musical genres. In an evaluation using real percussion recordings, the reconstruction accuracy and latent space interpolation between drum performances are investigated for audio generation conditioned by target rhythmic patterns.

Item Type: Article
Identification Number: https://doi.org/10.1145/3394171.3413519
12 October 2020Published
26 July 2020Accepted
Subjects: CAH11 - computing > CAH11-01 - computing > CAH11-01-05 - artificial intelligence
CAH25 - design, and creative and performing arts > CAH25-02 - performing arts > CAH25-02-02 - music
Divisions: Faculty of Computing, Engineering and the Built Environment > School of Computing and Digital Technology
Depositing User: Maciej Tomczak
Date Deposited: 29 Oct 2021 11:37
Last Modified: 22 Mar 2023 12:00
URI: https://www.open-access.bcu.ac.uk/id/eprint/12287

Actions (login required)

View Item View Item


In this section...