Perceptually-Informed No-Reference Image Harmonisation
Dolhasz, Alan and Harvey, Carlo and Williams, Ian (2022) Perceptually-Informed No-Reference Image Harmonisation. In: VISIGRAPP 2020: Computer Vision, Imaging and Computer Graphics Theory and Applications. Springer, pp. 394-413. ISBN 9783030948924
|
Text
SPRINGER_FINAL_DRAFT_FINAL.pdf - Accepted Version Download (14MB) |
Abstract
Many image synthesis tasks, such as image compositing, rely on the process of image harmonisation. The goal of harmonisation is to create a plausible combination of component elements. The subjective quality of this combination is directly related to the existence of human-detectable appearance differences between these component parts, suggesting that consideration for human perceptual tolerances is an important aspect of designing automatic harmonisation algorithms. In this paper, we first investigate the impact of a perceptually-calibrated composite artifact detector on the performance of a state-of-the-art deep harmonisation model. We first evaluate a two-stage model, whereby the performance of both pre-trained models and their naive combination is assessed against a large data-set of 68128 automatically generated image composites. We find that without any task-specific adaptations, the two-stage model achieves comparable results to the baseline harmoniser fed with ground truth composite masks. Based on these findings, we design and train an end-to-end model, and evaluate its performance against a set of baseline models. Overall, our results indicate that explicit modelling and incorporation of image features conditioned on a human perceptual task improves the performance of no-reference harmonisation algorithms. We conclude by discussing the generalisability of our approach in the context of related work.
| Item Type: | Book Section | ||||||
|---|---|---|---|---|---|---|---|
| Dates: |
|
||||||
| Uncontrolled Keywords: | image compositing, harmonisation, artifact detection, end-to-end compositing, deep learning | ||||||
| Subjects: | CAH11 - computing > CAH11-01 - computing > CAH11-01-01 - computer science CAH11 - computing > CAH11-01 - computing > CAH11-01-05 - artificial intelligence |
||||||
| Divisions: | Faculty of Computing, Engineering and the Built Environment > School of Computing and Digital Technology | ||||||
| Depositing User: | Carlo Harvey | ||||||
| Date Deposited: | 10 Dec 2020 13:27 | ||||||
| Last Modified: | 01 Jan 2024 03:00 | ||||||
| URI: | https://www.open-access.bcu.ac.uk/id/eprint/10522 |
Actions (login required)
 |
View Item |
Tools
Tools