Image Adaptation for Colour Vision Deficient Viewers Using Vision Transformers

Colour Vision Deficiency (CVD) occurs when anomalous retinal cone spectral responses impact the ability to distinguish between certain colours. To enhance image quality and viewing experience, recolouring algorithms seek to modify pixel values so that this does not lead to a loss of detail or image quality. Recent approaches to recolouring for CVD viewers employ neural models which exploit higher order features to direct colour adaptation. In this work, we build upon the idea that visual neural models exhibit emergent behaviour which mimics the human visual system. We make use of these learned behaviours to guide the colour adaptation process by considering regions of the image that are the most semantically meaningful for a non-CVD viewer and compensate for them appropriately if they are absent or distorted in a CVD-simulated version of the image. We find that a minimal algorithm built atop a pre-trained model produces results that substantially boost contrast and salience for viewers affected by CVD. We also investigate a few cases where modifications are absent, indicating that a neurally guided salience-based model may also provide a means of determining when recolouring is not necessary. Additionally, we introduce a novel metric that quantifies the contrast increase or decrease under changes in image colour.