Multi-Modal Perception for Selective Rendering

Harvey, Carlo and Debattista, Kurt and Bashford-Rogers, Thomas and Chalmers, Alan (2016) Multi-Modal Perception for Selective Rendering. Computer Graphics Forum, 36 (1). pp. 172-183. ISSN 0167-7055

Multi-Modal Perception for Selective Rendering.pdf - Accepted Version

Download (39MB)


A major challenge in generating high-fidelity virtual environments (VEs) is to be able to provide realism at interactive rates. The high-fidelity simulation of light and sound is still unachievable in real time as such physical accuracy is very computationally demanding. Only recently has visual perception been used in high-fidelity rendering to improve performance by a series of novel exploitations; to render parts of the scene that are not currently being attended to by the viewer at a much lower quality without the difference being perceived. This paper investigates the effect spatialized directional sound has on the visual attention of a user towards rendered images. These perceptual artefacts are utilized in selective rendering pipelines via the use of multi-modal maps. The multi-modal maps are tested through psychophysical experiments to examine their applicability to selective rendering algorithms, with a series of fixed cost rendering functions, and are found to perform significantly better than only using image saliency maps that are naively applied to multi-modal VEs.

Item Type: Article
Additional Information: This is the peer reviewed version of the following article: Harvey, C., Debattista, K., Bashford-Rogers, T. and Chalmers, A. (2017), Multi-Modal Perception for Selective Rendering. Computer Graphics Forum, 36: 172–183. doi:10.1111/cgf.12793 which has been published in final form at This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Self-Archiving.
Identification Number:
25 January 2016Published Online
Subjects: CAH11 - computing > CAH11-01 - computing > CAH11-01-01 - computer science
Divisions: Faculty of Computing, Engineering and the Built Environment
Faculty of Computing, Engineering and the Built Environment > School of Computing and Digital Technology
Depositing User: Ian Mcdonald
Date Deposited: 16 Oct 2017 12:41
Last Modified: 22 Mar 2023 12:01

Actions (login required)

View Item View Item


In this section...