A Computer Vision Inspired Automatic Acoustic Material Tagging System for Virtual Environments

Colombo, Mattia and Dolhasz, Alan and Harvey, Carlo (2020) A Computer Vision Inspired Automatic Acoustic Material Tagging System for Virtual Environments. In: IEEE Conference on Games, 24th-27th August 2020, Kindai, Osaka. Virtual.

Preview

Text CoG2020_short.pdf - Accepted Version Download (1MB)

Abstract

This paper presents the ongoing work on an approach to material information retrieval in virtual environments (VEs). Our approach uses convolutional neural networks to classify materials by performing semantic segmentation on images captured in the VE. Class maps obtained are then re-projected onto the environment. We use transfer learning and fine-tune a pretrained segmentation model on images captured in our VEs. The geometry and semantic information can then be used to create mappings between objects in the VE and acoustic absorption coefficients. This can then be input for physically-based audio renderers, allowing a significant reduction in manual material tagging.

Item Type:	Conference or Workshop Item (Paper)
Dates:	Date Event 14 June 2020 Accepted 27 August 2020 Published Online
Subjects:	CAH11 - computing > CAH11-01 - computing > CAH11-01-01 - computer science CAH11 - computing > CAH11-01 - computing > CAH11-01-05 - artificial intelligence
Divisions:	Faculty of Computing, Engineering and the Built Environment > College of Computing
Depositing User:	Carlo Harvey
Date Deposited:	20 Aug 2020 14:53
Last Modified:	22 Mar 2023 12:01
URI:	https://www.open-access.bcu.ac.uk/id/eprint/9711

Actions (login required)

View Item