A Computer Vision Inspired Automatic Acoustic Material Tagging System for Virtual Environments

Colombo, Mattia and Dolhasz, Alan and Harvey, Carlo (2020) A Computer Vision Inspired Automatic Acoustic Material Tagging System for Virtual Environments. In: IEEE Conference on Games, 24th-27th August 2020, Kindai, Osaka. Virtual.

[img]
Preview
Text
CoG2020_short.pdf - Accepted Version

Download (1MB)

Abstract

This paper presents the ongoing work on an approach to material information retrieval in virtual environments (VEs). Our approach uses convolutional neural networks to classify materials by performing semantic segmentation on images captured in the VE. Class maps obtained are then re-projected onto the environment. We use transfer learning and fine-tune a pretrained segmentation model on images captured in our VEs. The geometry and semantic information can then be used to create mappings between objects in the VE and acoustic absorption coefficients. This can then be input for physically-based audio renderers, allowing a significant reduction in manual material tagging.

Item Type: Conference or Workshop Item (Paper)
Date: 27 August 2020
Subjects: G400 Computer Science
G700 Artificial Intelligence
Divisions: Faculty of Computing, Engineering and the Built Environment > School of Computing and Digital Technology > Digital Media Technology
Depositing User: Harvey
Date Deposited: 20 Aug 2020 14:53
Last Modified: 05 Sep 2020 10:50
URI: http://www.open-access.bcu.ac.uk/id/eprint/9711

Actions (login required)

View Item View Item

Research

In this section...