Single Image Portrait Relighting with 3D Generative Priors

Abstract

Rendering novel, relit views of a human head, given a monocular portrait image as input, is inherently challenging. One possible solution would be to explicitly decompose the input into geometry, material and lighting via differentiable rendering, but such an approach would be constrained by the assumptions and approximations made by the respective models used for these three components. Instead, we propose 3DPR, a learned image-based relighting model, that is trained on One-Light-at-A-Time (OLAT) images. Capturing OLAT images requires a light stage, which is a device not commonly available even to researchers, so data is scarce and limited in diversity, constraining the generality of models. This is why we leverage a pre-trained generative but non-relightable head model, that has learned a rich prior of faces from large-scale in-the-wild image datasets. The input portrait is first embedded in the latent manifold of such a model through an encoder-based inversion process and then, with a novel triplane-based reflectance network, we synthesize high-fidelity OLAT images. Notably, since the reflectance network operates in the latent space of the generative head model, a relatively small number of lightstage images are sufficient to train the reflectance model. Combining the generated OLATs according to a given HDRI environment maps yields accurate relighting results. Through quantitative and qualitative evaluations, we demonstrate that 3DPR outperforms previous methods, particularly in preserving identity and in capturing lighting effects such as specularities, self-shadows, and subsurface scattering.

Citation Style:    Display Alpha ACM SIGGRAPH IEEEtran Publication

Single Image Portrait Relighting with 3D Generative Priors. Pramod Rao, Xilong Zhou, Abhimitra Meka, Gereon Fox, Mallikarjun B R, Fangneng Zhan, Tim Weyrich, Bernd Bickel, Hanspeter Pfister, Wojciech Matusik, Thabom Beeler, Mohamed Elgharib, Marc Habermann, Christian Theobalt. Conditionally accepted to SIGGRAPH Asia 2025 Conference Papers, 12 pages, December 2025.Pramod Rao, Xilong Zhou, Abhimitra Meka, Gereon Fox, Mallikarjun B R, Fangneng Zhan, Tim Weyrich, Bernd Bickel, Hanspeter Pfister, Wojciech Matusik, Thabom Beeler, Mohamed Elgharib, Marc Habermann, and Christian Theobalt. Single image portrait relighting with 3d generative priors. In SIGGRAPH Asia Conference Papers, New York, NY, USA, December 2025. Association for Computing Machinery.Rao, P., Zhou, X., Meka, A., Fox, G., B R, M., Zhan, F., Weyrich, T., Bickel, B., Pfister, H., Matusik, W., Beeler, T., Elgharib, M., Habermann, M., and Theobalt, C. 2025. Single image portrait relighting with 3d generative priors. In SIGGRAPH Asia Conference Papers, Association for Computing Machinery, New York, NY, USA.P. Rao, X. Zhou, A. Meka, G. Fox, M. B R, F. Zhan, T. Weyrich, B. Bickel, H. Pfister, W. Matusik, T. Beeler, M. Elgharib, M. Habermann, and C. Theobalt, “Single image portrait relighting with 3d generative priors,” inSIGGRAPH Asia Conference Papers. New York, NY, USA: Association for Computing Machinery, Dec. 2025. BibTeX