Soo Celinta Wajiga GFPGAN

Blind face restoration means recovering a clean, detailed face from a damaged photo when the exact damage is unknown. A real old snapshot might be blurred, shrunk, noisy and saved as a heavily compressed JPEG, all at once. GFPGAN, short for Generative Facial Prior GAN, attacks this by combining two sources of information: what can still be seen in the input, and what a separate network already knows about how faces look. That second source is the generative facial prior, taken from StyleGAN2, a generative adversarial network (GAN) trained to produce photorealistic faces.

The method was introduced in 'Towards Real-World Blind Face Restoration with Generative Facial Prior' by Xintao Wang, Yu Li, Honglun Zhang and Ying Shan of Tencent's Applied Research Center, posted to arXiv on 11 January 2021 and accepted at CVPR 2021. Earlier approaches leaned on geometric priors such as facial landmarks or parsing maps, which are hard to estimate from a badly degraded input, or on reference photos of the same person, which are rarely available. GAN inversion methods such as PULSE searched for a StyleGAN latent code that matched the input, which was slow and often produced a convincing face that was not the same person. GFPGAN set out to keep the realism of a GAN prior while staying faithful to the input, in one forward pass.

The pipeline has three stages. First, a degradation removal module, a U-Net with seven downsampling and seven upsampling steps, cleans the input and produces two outputs: multi-resolution spatial features and a compact latent feature. Second, the latent feature is mapped through small fully connected layers to intermediate latent codes for a StyleGAN2 generator pretrained at 512 by 512 pixels, which generates rich facial features. Third, Channel-Split Spatial Feature Transform layers, called CS-SFT, blend the two. At each resolution, the input's spatial features predict a scale and shift that modulate half of the StyleGAN2 feature channels, while the other half pass through untouched. The modulated half keeps the result tied to the real face; the untouched half preserves the GAN's texture and realism.

Training combines several losses. A reconstruction loss pairs pixel L1 distance with a perceptual loss from a VGG-19 network, and is applied at several pyramid levels inside the degradation removal module. An adversarial loss pushes the whole face to look real. Small local discriminators judge crops of the left eye, right eye and mouth, with a Gram-matrix style loss to sharpen fine texture. An identity preserving loss compares ArcFace face-recognition embeddings of the output and the ground truth, so the restored face still reads as the same person. The paper trained on the 70,000-image FFHQ dataset resized to 512 pixels, synthesizing damaged inputs by blurring, downsampling by a factor of 1 to 8, adding Gaussian noise and JPEG compressing at quality 60 to 100. Training ran for 800,000 iterations on four Nvidia Tesla P40 GPUs.

On the synthetic CelebA-Test benchmark of 3,000 images, the paper reports GFPGAN at an LPIPS of 0.3646, FID of 42.62 and NIQE of 4.077, the best perceptual scores among the methods compared, including PSFRGAN, DFDNet, HiFaceGAN and PULSE. LPIPS measures perceptual distance to the ground truth, FID compares image statistics to real faces, and NIQE scores naturalness without a reference; lower is better for all three. Its identity distance of 34.60 was also the lowest, while its PSNR of 25.08 dB was slightly below a fine-tuned DeblurGANv2 at 25.91, a reminder that pixel-accurate and realistic-looking are different goals. The authors also tested on real low-quality sets: LFW-Test, CelebChild-Test and 407 faces from 188 web photos.

The open-source release grew after the paper. The GitHub repository, under the Apache 2.0 license, lists V1 as the paper model with colorization, V1.2 as sharper but sometimes unnatural with a made-up look, V1.3 as more natural with slight identity change, and V1.4 as adding slightly more detail and better identity. A 'clean' architecture removed the need for custom CUDA extensions, which made installation much easier. Non-face regions are enlarged by Real-ESRGAN, a sibling project from the same team. The repository showed about 37,700 GitHub stars when checked for this article.

Compared with alternatives, GFPGAN sits between older prior-based restorers and newer approaches. CodeFormer, published at NeurIPS 2022 by Shangchen Zhou and colleagues, replaces the continuous GAN prior with a learned discrete codebook and a transformer that predicts codes, and offers a fidelity weight to trade realism against faithfulness. Its authors report better robustness under heavy damage. Diffusion-based face restorers are a further line of work. GFPGAN's advantages are speed and simplicity: one forward pass, no per-image optimization, and modest hardware needs.

The limitations matter. The paper notes that under very severe degradation, restored details twist into artifacts, and very large head poses yield unnatural faces, because synthetic training damage differs from real-world damage. Grayscale inputs can produce biased skin color since there is no color information to anchor it. More fundamentally, any generative restorer fills missing detail with plausible guesses. The output can look sharp yet differ from the real person, so GFPGAN results should not be treated as evidence of what someone looked like, for example in forensic or journalistic use. Running it locally needs a PyTorch install and benefits from a GPU; hosted runs on Replicate cost fractions of a cent.