HP3: Tuning-Free Head-Preserving Portrait Personalization Via 3D-Controlled Diffusion Models

摘要

Portrait personalization (PP) has garnered considerable attention recently due to its potential applications. However, existing methods only preserve the face region, with limited capability to customize head attributes, which restricts their practicality. To address these challenges, we introduce HP3, the first head-preserving framework for zero-shot PP. It can generate realistic portraits that preserve the source head, while controlling head expression and pose through the driving image. To accomplish this, we first design the head encoder and 3D reconstruction module to obtain head features and 3D priors. Next, the head controller is devised to align them, producing 3D-aware head conditions. These conditions are injected into UNet via the adaptive connector to achieve superior head preservation and control. Qualitative and quantitative experiments demonstrate that HP3 significantly outperforms SOTA methods. Our project is available at https://github.com/YoucanBaby/HP3.