A Computer Vision Inspired Automatic Acoustic Material Tagging System for Virtual Environments

Colombo, Mattia and Dolhasz, Alan and Harvey, Carlo (2020) A Computer Vision Inspired Automatic Acoustic Material Tagging System for Virtual Environments. In: IEEE Conference on Games, 24th-27th August 2020, Kindai, Osaka. Virtual.

CoG2020_short.pdf - Accepted Version

Download (1MB)


This paper presents the ongoing work on an approach to material information retrieval in virtual environments (VEs). Our approach uses convolutional neural networks to classify materials by performing semantic segmentation on images captured in the VE. Class maps obtained are then re-projected onto the environment. We use transfer learning and fine-tune a pretrained segmentation model on images captured in our VEs. The geometry and semantic information can then be used to create mappings between objects in the VE and acoustic absorption coefficients. This can then be input for physically-based audio renderers, allowing a significant reduction in manual material tagging.

Item Type: Conference or Workshop Item (Paper)
14 June 2020Accepted
27 August 2020Published Online
Subjects: CAH11 - computing > CAH11-01 - computing > CAH11-01-01 - computer science
CAH11 - computing > CAH11-01 - computing > CAH11-01-05 - artificial intelligence
Divisions: Faculty of Computing, Engineering and the Built Environment > School of Computing and Digital Technology
Depositing User: Carlo Harvey
Date Deposited: 20 Aug 2020 14:53
Last Modified: 22 Mar 2023 12:01
URI: https://www.open-access.bcu.ac.uk/id/eprint/9711

Actions (login required)

View Item View Item


In this section...