A Computer Vision Inspired Automatic Acoustic Material Tagging System for Virtual Environments
Colombo, Mattia and Dolhasz, Alan and Harvey, Carlo (2020) A Computer Vision Inspired Automatic Acoustic Material Tagging System for Virtual Environments. In: IEEE Conference on Games, 24th-27th August 2020, Kindai, Osaka. Virtual.
|
Text
CoG2020_short.pdf - Accepted Version Download (1MB) |
Abstract
This paper presents the ongoing work on an approach to material information retrieval in virtual environments (VEs). Our approach uses convolutional neural networks to classify materials by performing semantic segmentation on images captured in the VE. Class maps obtained are then re-projected onto the environment. We use transfer learning and fine-tune a pretrained segmentation model on images captured in our VEs. The geometry and semantic information can then be used to create mappings between objects in the VE and acoustic absorption coefficients. This can then be input for physically-based audio renderers, allowing a significant reduction in manual material tagging.
| Item Type: | Conference or Workshop Item (Paper) | ||||||
|---|---|---|---|---|---|---|---|
| Dates: |
|
||||||
| Subjects: | CAH11 - computing > CAH11-01 - computing > CAH11-01-01 - computer science CAH11 - computing > CAH11-01 - computing > CAH11-01-05 - artificial intelligence |
||||||
| Divisions: | Faculty of Computing, Engineering and the Built Environment > School of Computing and Digital Technology | ||||||
| Depositing User: | Carlo Harvey | ||||||
| Date Deposited: | 20 Aug 2020 14:53 | ||||||
| Last Modified: | 22 Mar 2023 12:01 | ||||||
| URI: | https://www.open-access.bcu.ac.uk/id/eprint/9711 |
Actions (login required)
 |
View Item |
Tools
Tools