概述
DDPM 源自 2020 年,是原始的隨機採樣器,需要約 1,000 個步驟。DDIM 源自 2020 至 2021 年,重用相同的訓練網路但採取確定性捷徑,在 20 到 100 個步驟內達到相似品質,速度提升 10 到 50 倍。
重點摘要
- DDPM 和 DDIM 是採樣器,從噪音到圖像反向運行相同的訓練擴散網路。
- 2020 年的 DDPM 論文使用了 1,000 個噪音步驟,並報告在 CIFAR-10 上的 FID 為 3.17。
- DDIM 於 ICLR 2021 年發表,在 20 到 100 個步驟內達到可比擬的品質,速度提升 10 到 50 倍。
- Eta 參數設定 DDIM 的隨機性:eta 0 是確定性的,eta 1 的行為類似於 DDPM。
- 確定性 DDIM 可以反向運行,將真實圖像編碼為噪音,這構成了許多擴散圖像編輯方法的基礎。
- 從 DDPM 轉換到 DDIM 不需要重新訓練,因為兩者共享相同的訓練目標。
深入探討
A diffusion model is an image generator that learns by destroying pictures with noise and then learning to undo the damage. During training, a clean image is mixed with Gaussian noise (random values drawn from a bell curve) according to a fixed schedule, and a neural network is taught to predict the noise that was added at each level. Generation runs the process the other way: start from pure noise and remove a little at a time. A sampler is the recipe for that backward walk. DDPM and DDIM are the two foundational samplers, and both work with the same trained network.
DDPM stands for Denoising Diffusion Probabilistic Models, the paper by Jonathan Ho, Ajay Jain and Pieter Abbeel posted to arXiv on 19 June 2020 and published at NeurIPS 2020. It made a 2015 idea from Jascha Sohl-Dickstein and colleagues practical. The paper used T = 1,000 noise levels with a noise variance, called beta, rising linearly from 0.0001 to 0.02. Its network was a U-Net, an encoder-decoder convolutional network, with the timestep fed in as a sinusoidal embedding. The key simplification was to train the network to predict the added noise with a plain mean-squared-error loss, which the authors showed is closely related to denoising score matching. On CIFAR-10 the paper reported an Inception score of 9.46 and an FID of 3.17, where FID (Frechet Inception Distance) compares generated and real image statistics and lower is better.
DDPM sampling works like this. Draw x_T, an image-shaped array of pure noise. At each step t, the network predicts the noise in x_t. The sampler uses that prediction to compute the mean of a slightly cleaner image, then adds a fresh dose of random noise scaled to that step, giving x_(t-1). Repeat until t reaches zero. Because every step injects new randomness, the process is a Markov chain: each state depends only on the one before it. That randomness is mathematically required for the derivation, and it is also why DDPM needs nearly all 1,000 steps. The cost is real: the DDIM paper reports that sampling 50,000 32 by 32 images from a DDPM took around 20 hours on an Nvidia 2080 Ti, against under a minute for a GAN, and nearly 1,000 hours for 256 by 256 images.
DDIM, Denoising Diffusion Implicit Models, by Jiaming Song, Chenlin Meng and Stefano Ermon, was posted on 6 October 2020 and published at ICLR 2021. Its insight is that DDPM's training objective does not pin down a single backward process. A whole family of non-Markovian processes, where each step may depend on the starting point x_0 and not only the previous state, shares the same training loss. So a network trained as a DDPM can be sampled with any member of that family without retraining. DDIM picks members that allow skipping: instead of visiting all 1,000 levels, it can jump along a subsequence of, say, 50 levels.
A DDIM step has three parts. First, use the predicted noise to estimate the clean image x_0. Second, point back toward the current noise direction. Third, combine the two at the noise level of the next chosen timestep, optionally adding fresh randomness. A parameter called eta controls that randomness: eta = 1 recovers DDPM-like stochastic sampling, and eta = 0 gives fully deterministic DDIM, where the same starting noise always yields the same image. In the paper's Table 1, deterministic DDIM on CIFAR-10 reached an FID of 13.36 with 10 steps, 6.84 with 20, 4.67 with 50 and 4.16 with 100, while DDPM-style sampling (eta = 1) scored 41.07 at 10 steps, and the larger-variance variant Ho and colleagues used scored 367.43, essentially unusable. The authors report that DDIM matches 1,000-step quality within 20 to 100 steps, a 10 to 50 times wall-clock speedup, and that on CelebA a 100-step DDPM scores about the same as a 20-step DDIM.
Determinism has side benefits. Because noise maps to image along a fixed path, the path can be run in reverse to encode a real image into its starting noise, called DDIM inversion. The DDIM paper reports CIFAR-10 reconstruction error falling from 0.014 at 10 steps to 0.0001 at 1,000 steps. Interpolating between two starting noises gives smooth, meaningful blends of the two outputs, which DDPM cannot do because the same starting noise leads to different images. Null-text Inversion, posted in November 2022, builds real-photo editing on top of DDIM inversion, and notes that plain inversion breaks down when classifier-free guidance amplifies its small per-step errors.
Neither sampler is the end of the story. DDIM is a first-order method, meaning each step uses only the current slope. Later solvers treat sampling as solving an ordinary differential equation with higher-order steps: DPM-Solver, published at NeurIPS 2022, reports a CIFAR-10 FID of 4.70 in 10 steps and 2.87 in 20. Libraries such as diffusers now offer many schedulers, and DDIM remains a reference baseline. Known failure modes include blurry or washed-out images at very low step counts, accumulated error in inversion under strong guidance, and a training-inference mismatch: the diffusers documentation points to a 2023 paper showing that common schedules never reach pure noise, which limits very bright or dark outputs unless the betas are rescaled and timesteps are spaced to start from the last one.
In practice, both samplers cost one network evaluation per step, so total compute scales linearly with step count, as the DDIM paper's timing plots confirm. The DDPM paper's CIFAR-10 model trained for 800,000 steps in 10.6 hours on a TPU v3-8, roughly equal to eight V100 GPUs, and sampled 256 images in 17 seconds. Switching samplers needs no retraining, so choosing between them is purely an inference-time decision.
04工作範例
在同一模型上比較 DDPM 風格和 DDIM 採樣
在 GPU 上載入 Stable Diffusion 管道於 Hugging Face diffusers 中,並使用 torch.Generator 固定亂數種子。
將排程器替換為 DDIMScheduler.from_config(pipe.scheduler.config)。
使用固定的提示詞、50 個推論步驟和設為 0.0 的 eta 生成一張影像。
使用相同的種子和設定再次生成,並確認兩張影像的像素完全相同,這顯示了確定性取樣。
將 eta 重複設為 1.0 兩次,並為步驟雜訊使用不同的種子,觀察到輸出現在會有所不同。
以 10、20 和 100 個步驟執行 eta 0.0 設定,並比較清晰度和細節。
記錄每次執行的實際執行時間 (wall-clock time),以確認成本隨步數大約呈線性成長。
它顯示了什麼
學習者最後會看到一個小網格,顯示 eta 0 是可重複的,eta 1 增加了多樣性,並且品質從 10 步急劇上升到約 50 步,然後趨於平緩。時間記錄應顯示與步數呈近乎線性的成長,這與 DDIM 論文的測量結果一致。
現實世界的實施
原始的 CompVis Stable Diffusion 用法公開了 --ddim_steps 選項和 --ddim_eta 設定,其中 0.0 的 eta 提供來自固定種子的確定性 DDIM 取樣。
Hugging Face diffusers 提供了 DDIMScheduler,它可以一行程式碼交換到 Stable Diffusion 管道中,以減少每張影像的去雜訊步驟數。
Null-text Inversion(Mokady 及其同事,2022 年)逆向運行 DDIM 取樣以尋找重現真實照片的雜訊,然後透過更改其文字提示詞來編輯照片。
研究人員在兩個 DDIM 起始雜訊之間進行內插,以從一個生成的臉部平滑變形到另一個,這是 DDIM 論文在 CelebA 和 LSUN 影像上展示的特性。
團隊透過 20 到 50 個 DDIM 步驟快速預覽許多提示詞,因為 DDIM 論文顯示在該範圍內的品質可與 1,000 步取樣相媲美。
DDPM 和 DDIM 採樣器的未來
Research since DDIM has focused on fewer steps with no quality loss. Higher-order ODE solvers such as DPM-Solver (2022) cut sampling to around 10 to 20 network evaluations, and the diffusers documentation for DDIM now carries guidance from a 2023 paper on zero terminal signal-to-noise schedules and trailing timestep spacing, showing that the details of the sampling grid still matter. Open problems include making inversion exact under classifier-free guidance, which Null-text Inversion (2022) addressed by optimizing the unconditional text embedding per image, and understanding why stochastic sampling sometimes beats deterministic sampling at high step counts, as the DDIM paper's own 1,000-step comparison shows. Distillation methods that compress a many-step sampler into one or a few steps are an active line of work, and whether such approaches fully replace classic samplers across image, video and audio models is not yet settled in published results. DDPM and DDIM remain the baseline these newer methods are measured against.
付諸實踐
為什麼這很重要
- 速度與規模. 視覺人工智慧可以大規模自動化檢查、檢測和標記任務。
- 配裝選擇. 創意團隊可以透過更少的手動修改來更快地建立概念原型。
- 團隊與工作流程. 操作可以使用以前難以處理的影像和視訊訊號。
留意
- 如果出處不明,肖像權和同意可能會成為法律風險。
- 模型表現可能因光照、人口統計和環境的不同而有所不同。
- 除非監控置信閾值,否則誤報可能會被忽略。
後續步驟
- 定義精確度、召回率和錯誤成本的接受標準。
- 使用符合實際生產條件的數據進行測試。
- 為低置信度或高影響力的預測添加人工審核。
- 追蹤模型漂移並在相機或資料集變更後重新驗證。
資料來源與延伸閱讀
- arXiv (NeurIPS 2020)去噪擴散機率模型
- arXiv (ICLR 2021)去噪擴散隱式模型
- Hugging Face diffusers documentationDDIMScheduler
- arXiv (NeurIPS 2022)DPM-Solver:一種用於擴散機率模型採樣的高速 ODE 求解器,約需 10 步
- arXiv使用引導式擴散模型進行真實圖像編輯的無文本反演
- GitHub (CompVis)CompVis/stable-diffusion
不斷探索
常見問題
什麼是 DDPM 和 DDIM 採樣器?
DDPM 和 DDIM 是兩種反向運行擴散模型的程序,透過逐步去噪將純雜訊轉化為圖像。DDPM 於 2020 年提出,是原始的隨機採樣器,需要約 1,000 步。DDIM 於 2020 年至 2021 年間提出,重用相同的訓練網路,但採用確定性捷徑,在 20 到 100 步內達到相似品質,速度提升 10 到 50 倍。
DDIM 相對於 DDPM 的主要實際優勢是什麼?
DDIM 遵循確定性的非馬可夫軌跡,使其可以跳過許多時間步,以大約 10-50 個步驟(而不是數百個步驟)產生高品質影像。
擴散模型的神經網路在訓練過程中實際上學會預測什麼?
此網路經過訓練可以預測每個時間步驟所添加的高斯雜訊 (epsilon),然後 DDPM 和 DDIM 使用該雜訊來反轉該過程。
為什麼 DDIM 每次都能從相同的起始雜訊產生完全相同的影像?
DDIM 在其更新中刪除了隨機雜訊項,使得從雜訊到影像的路徑完全確定,因此可重現。
在 DDIM 中,參數 eta 的什麼值可以恢復原始的隨機 DDPM 行為?
eta 參數在兩個採樣器之間進行內插:eta=1 給出完全的 DDPM 隨機性,而 eta=0 給出完全確定的 DDIM。
原始 DDPM 通常需要多少步驟才能獲得高品質樣本?
DDPM 會回溯每個噪音級別,增加隨機性,因此通常需要數百到一千個反向步驟。