A Neural Approach for Skin Spectral Reconstruction

Accurate reproduction of human skin color requires knowledge of skin spectral reflectance data, which is often unavailable. Traditionally, spectral reconstruction algorithms attempt to recover the spectra using commonly available RGB camera response. Among various methods employed, polynomial regression has proven beneficial for skin spectral reconstruction. Despite their simplicity and interpretability, nonlinear regression methods may deliver sub-optimal results as the size of the data increases. Furthermore, they are prone to overfitting and require carefully adjusted hyperparameters through regularization. Another challenging issue in skin spectral reconstruction is the lack of high-quality skin hyperspectral databases available for research. In this paper, we gather skin spectral data from publicly available databases and extract the effective dimensions of these spectra using principal component analysis (PCA). We show that plausible skin spectra can be accurately modeled through a linear combination of six spectral bases. We propose a new approach for estimating the weights of such a linear combination from RGB data using neural networks, leading to the reconstruction of spectra. Furthermore, we utilize a daylight model to estimate the underlying scene illumination metamer. We demonstrate that our proposed model can effectively reconstruct facial skin spectra and render facial appearance with high color fidelity.