Perceptually-Informed No-Reference Image Harmonisation

Dolhasz, Alan and Harvey, Carlo and Williams, Ian (2020) Perceptually-Informed No-Reference Image Harmonisation. In: Springer Book of VISAPP 2020. Springer. (In Press)

[img] Text
Restricted to Repository staff only

Download (14MB) | Request a copy


Many image synthesis tasks, such as image compositing, rely on the process of image harmonisation. The goal of harmonisation is to create a plausible combination of component elements. The subjective quality of this combination is directly related to the existence of human-detectable appearance differences between these component parts, suggesting that consideration for human perceptual tolerances is an important aspect of designing automatic harmonisation algorithms. In this paper, we first investigate the impact of a perceptually-calibrated composite artifact detector on the performance of a state-of-the-art deep harmonisation model. We first evaluate a two-stage model, whereby the performance of both pre-trained models and their naive combination is assessed against a large data-set of 68128 automatically generated image composites. We find that without any task-specific adaptations, the two-stage model achieves comparable results to the baseline harmoniser fed with ground truth composite masks. Based on these findings, we design and train an end-to-end model, and evaluate its performance against a set of baseline models. Overall, our results indicate that explicit modelling and incorporation of image features conditioned on a human perceptual task improves the performance of no-reference harmonisation algorithms. We conclude by discussing the generalisability of our approach in the context of related work.

Item Type: Book Section
Date: 15 July 2020
Uncontrolled Keywords: image compositing, harmonisation, artifact detection, end-to-end compositing, deep learning
Subjects: G400 Computer Science
G700 Artificial Intelligence
Divisions: Faculty of Computing, Engineering and the Built Environment > School of Computing and Digital Technology > Digital Media Technology
Depositing User: Carlo Harvey
Date Deposited: 10 Dec 2020 13:27
Last Modified: 10 Dec 2020 13:27

Actions (login required)

View Item View Item


In this section...