Virtual Try-On with Diffusion Models

Virtual try-on has two jobs that pull against each other: keep the person unchanged (face, hair, skin tone, body shape, pose) and faithfully reproduce the garment (cut, color, print, logo, fabric texture). Early approaches such as VITON (2018) and CP-VTON worked in two stages. First they warped the flat garment image to match the person's pose, often using a thin-plate spline or learned flow field, then a second network blended the warped garment into the photo. These methods struggled with large pose changes, occlusions such as crossed arms, and complex prints, because a 2D warp cannot invent parts of the garment that were never visible. Diffusion models changed the second stage. Instead of blending, the model regenerates the clothing region from noise while being conditioned on the garment image, the person's pose keypoints, a body-parsing map and a masked version of the original photo. Google's TryOnDiffusion (2023) used two parallel networks that exchange information through cross-attention, letting the garment be implicitly warped inside the model rather than by a separate step; Google used this family of techniques for a try-on feature in its shopping results, shown on real models of different sizes. Open research models such as StableVITON, OOTDiffusion and IDM-VTON adapted Stable Diffusion for the task. The common misconception is that try-on tells you your size. It does not. The model has no measurements of the garment's stretch or your body; it produces a plausible image of how clothes of that style tend to drape. Failure modes include tops that look identically fitted on every body, lost or smeared text and logos, altered skin tone or body shape, invented hands and arms, and weaker results for body types, skin tones, clothing styles or poses that are rare in training data.