Amabwiriza ya AI

Icyitegererezo cya DDPM na DDIM

DDPM na DDIM ni inzira ebyiri zo gukoresha icyitegererezo cyo gukwirakwiza (diffusion model) inyuma, zihindura urusaku rw'umwimerere (pure noise) zikagira ishusho intambwe ku yindi.

Umva iki gitaboAI-byavuzwe · ~8 min

Kuriyi page9 min soma
  1. Incamake
  2. Ibyingenzi byingenzi
  3. Kwibira cyane
  4. Comparing DDPM-style and DDIM sampling on the same model
  5. Gushyira mu bikorwa Isi
  6. Kazoza ka DDPM na DDIM
  7. Kubishyira mubikorwa
  8. Inkomoko no gusoma
  9. Komeza Ubushakashatsi
  10. Ibibazo bikunze kubazwa

Incamake

DDPM, from 2020, is the original stochastic sampler and needs about 1,000 steps. DDIM, from 2020 to 2021, reuses the same trained network but takes deterministic shortcuts, reaching similar quality in 20 to 100 steps, a 10 to 50 times speedup.

Ibyingenzi byingenzi

  1. DDPM and DDIM are samplers that run the same trained diffusion network backward from noise to an image.
  2. The 2020 DDPM paper used 1,000 noise steps and reported an FID of 3.17 on CIFAR-10.
  3. DDIM, published at ICLR 2021, reaches comparable quality in 20 to 100 steps, a 10 to 50 times speedup.
  4. The eta parameter sets DDIM's randomness: eta 0 is deterministic and eta 1 behaves like DDPM.
  5. Deterministic DDIM can be run in reverse to encode a real image into noise, which underpins many diffusion image-editing methods.
  6. Changing from DDPM to DDIM requires no retraining because both share the same training objective.

Kwibira cyane

A diffusion model is an image generator that learns by destroying pictures with noise and then learning to undo the damage. During training, a clean image is mixed with Gaussian noise (random values drawn from a bell curve) according to a fixed schedule, and a neural network is taught to predict the noise that was added at each level. Generation runs the process the other way: start from pure noise and remove a little at a time. A sampler is the recipe for that backward walk. DDPM and DDIM are the two foundational samplers, and both work with the same trained network.

DDPM stands for Denoising Diffusion Probabilistic Models, the paper by Jonathan Ho, Ajay Jain and Pieter Abbeel posted to arXiv on 19 June 2020 and published at NeurIPS 2020. It made a 2015 idea from Jascha Sohl-Dickstein and colleagues practical. The paper used T = 1,000 noise levels with a noise variance, called beta, rising linearly from 0.0001 to 0.02. Its network was a U-Net, an encoder-decoder convolutional network, with the timestep fed in as a sinusoidal embedding. The key simplification was to train the network to predict the added noise with a plain mean-squared-error loss, which the authors showed is closely related to denoising score matching. On CIFAR-10 the paper reported an Inception score of 9.46 and an FID of 3.17, where FID (Frechet Inception Distance) compares generated and real image statistics and lower is better.

DDPM sampling works like this. Draw x_T, an image-shaped array of pure noise. At each step t, the network predicts the noise in x_t. The sampler uses that prediction to compute the mean of a slightly cleaner image, then adds a fresh dose of random noise scaled to that step, giving x_(t-1). Repeat until t reaches zero. Because every step injects new randomness, the process is a Markov chain: each state depends only on the one before it. That randomness is mathematically required for the derivation, and it is also why DDPM needs nearly all 1,000 steps. The cost is real: the DDIM paper reports that sampling 50,000 32 by 32 images from a DDPM took around 20 hours on an Nvidia 2080 Ti, against under a minute for a GAN, and nearly 1,000 hours for 256 by 256 images.

DDIM, Denoising Diffusion Implicit Models, by Jiaming Song, Chenlin Meng and Stefano Ermon, was posted on 6 October 2020 and published at ICLR 2021. Its insight is that DDPM's training objective does not pin down a single backward process. A whole family of non-Markovian processes, where each step may depend on the starting point x_0 and not only the previous state, shares the same training loss. So a network trained as a DDPM can be sampled with any member of that family without retraining. DDIM picks members that allow skipping: instead of visiting all 1,000 levels, it can jump along a subsequence of, say, 50 levels.

A DDIM step has three parts. First, use the predicted noise to estimate the clean image x_0. Second, point back toward the current noise direction. Third, combine the two at the noise level of the next chosen timestep, optionally adding fresh randomness. A parameter called eta controls that randomness: eta = 1 recovers DDPM-like stochastic sampling, and eta = 0 gives fully deterministic DDIM, where the same starting noise always yields the same image. In the paper's Table 1, deterministic DDIM on CIFAR-10 reached an FID of 13.36 with 10 steps, 6.84 with 20, 4.67 with 50 and 4.16 with 100, while DDPM-style sampling (eta = 1) scored 41.07 at 10 steps, and the larger-variance variant Ho and colleagues used scored 367.43, essentially unusable. The authors report that DDIM matches 1,000-step quality within 20 to 100 steps, a 10 to 50 times wall-clock speedup, and that on CelebA a 100-step DDPM scores about the same as a 20-step DDIM.

Determinism has side benefits. Because noise maps to image along a fixed path, the path can be run in reverse to encode a real image into its starting noise, called DDIM inversion. The DDIM paper reports CIFAR-10 reconstruction error falling from 0.014 at 10 steps to 0.0001 at 1,000 steps. Interpolating between two starting noises gives smooth, meaningful blends of the two outputs, which DDPM cannot do because the same starting noise leads to different images. Null-text Inversion, posted in November 2022, builds real-photo editing on top of DDIM inversion, and notes that plain inversion breaks down when classifier-free guidance amplifies its small per-step errors.

Neither sampler is the end of the story. DDIM is a first-order method, meaning each step uses only the current slope. Later solvers treat sampling as solving an ordinary differential equation with higher-order steps: DPM-Solver, published at NeurIPS 2022, reports a CIFAR-10 FID of 4.70 in 10 steps and 2.87 in 20. Libraries such as diffusers now offer many schedulers, and DDIM remains a reference baseline. Known failure modes include blurry or washed-out images at very low step counts, accumulated error in inversion under strong guidance, and a training-inference mismatch: the diffusers documentation points to a 2023 paper showing that common schedules never reach pure noise, which limits very bright or dark outputs unless the betas are rescaled and timesteps are spaced to start from the last one.

In practice, both samplers cost one network evaluation per step, so total compute scales linearly with step count, as the DDIM paper's timing plots confirm. The DDPM paper's CIFAR-10 model trained for 800,000 steps in 10.6 hours on a TPU v3-8, roughly equal to eight V100 GPUs, and sampled 256 images in 17 seconds. Switching samplers needs no retraining, so choosing between them is purely an inference-time decision.

04Urugero rwakazi

Comparing DDPM-style and DDIM sampling on the same model

  1. Load a Stable Diffusion pipeline in Hugging Face diffusers on a GPU and fix a random seed with torch.Generator.

  2. Replace the scheduler with DDIMScheduler.from_config(pipe.scheduler.config).

  3. Generate one image with a fixed prompt, 50 inference steps and eta set to 0.0.

  4. Generate again with the same seed and settings and confirm the two images are pixel-identical, which shows deterministic sampling.

  5. Repeat with eta set to 1.0 twice with different seeds for the step noise and observe that outputs now vary.

  6. Run the eta 0.0 setting at 10, 20 and 100 steps and compare sharpness and detail.

  7. Record wall-clock time for each run to confirm that cost grows roughly linearly with the number of steps.

Ibyo byerekana

The learner ends with a small grid showing that eta 0 is repeatable, eta 1 adds variety, and quality rises steeply from 10 to about 50 steps before flattening. The timing log should show near-linear growth with step count, matching the DDIM paper's measurements.

Gushyira mu bikorwa Isi

The original CompVis Stable Diffusion scripts expose a --ddim_steps option and a --ddim_eta setting, where an eta of 0.0 gives deterministic DDIM sampling from a fixed seed.

Hugging Face diffusers ships DDIMScheduler, which can be swapped into a Stable Diffusion pipeline in one line to cut the number of denoising steps per image.

Null-text Inversion (Mokady and colleagues, 2022) runs DDIM sampling in reverse to find the noise that reproduces a real photo, then edits the photo by changing its text prompt.

Researchers interpolate between two DDIM starting noises to morph smoothly from one generated face to another, a property the DDIM paper demonstrates on CelebA and LSUN images.

Teams preview many prompts quickly with 20 to 50 DDIM steps, since the DDIM paper shows quality comparable to 1,000-step sampling within that range.

Kazoza ka DDPM na DDIM

Research since DDIM has focused on fewer steps with no quality loss. Higher-order ODE solvers such as DPM-Solver (2022) cut sampling to around 10 to 20 network evaluations, and the diffusers documentation for DDIM now carries guidance from a 2023 paper on zero terminal signal-to-noise schedules and trailing timestep spacing, showing that the details of the sampling grid still matter. Open problems include making inversion exact under classifier-free guidance, which Null-text Inversion (2022) addressed by optimizing the unconditional text embedding per image, and understanding why stochastic sampling sometimes beats deterministic sampling at high step counts, as the DDIM paper's own 1,000-step comparison shows. Distillation methods that compress a many-step sampler into one or a few steps are an active line of work, and whether such approaches fully replace classic samplers across image, video and audio models is not yet settled in published results. DDPM and DDIM remain the baseline these newer methods are measured against.

Kubishyira mubikorwa

Impamvu ari ngombwa

  • Umuvuduko n'igipimo. AI igaragara irashobora gukora igenzura, gutahura, no gutondekanya imirimo kurwego.
  • Kubaka amahitamo. Amakipe arema arashobora prototype ibitekerezo byihuse hamwe nintoki nkeya.
  • Itsinda hamwe nakazi. Ibikorwa birashobora gukoresha amashusho nibimenyetso bya videwo byari bigoye gutunganya.

Witondere

  • Uburenganzira bwishusho hamwe no kwemererwa birashobora guhinduka ibyago byemewe n'amategeko niba ibimenyetso bidasobanutse.
  • Imikorere yicyitegererezo irashobora gutandukana kumurika, demografiya, nibidukikije.
  • Ibyiza byibinyoma birashobora kutamenyekana keretse niba ibyiringiro byateganijwe bikurikiranwa.

Intambwe ikurikira

  1. Sobanura ibipimo byo kwemererwa kugiciro, kwibutsa, nibiciro byamakosa.
  2. Gerageza hamwe namakuru ajyanye nuburyo nyabwo bwo gukora.
  3. Ongeraho isubiramo ryabantu kubwizere buke cyangwa guhanura cyane.
  4. Kurikirana icyitegererezo cya drift hanyuma uhindurwe nyuma ya kamera cyangwa dataset ihinduka.

Inkomoko no gusoma

  1. arXiv (NeurIPS 2020)Kwamagana Diffusion Ikibazo Cyicyitegererezo
  2. arXiv (ICLR 2021)Denoising Diffusion Implicit Models
  3. Hugging Face diffusers documentationDDIMScheduler
  4. arXiv (NeurIPS 2022)DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps
  5. arXivNull-text Inversion for Editing Real Images using Guided Diffusion Models
  6. GitHub (CompVis)CompVis/stable-diffusion

Komeza Ubushakashatsi

Ibaruwa y'amakuru y'ubuntu

Ibuka amakuru ya buri munsi ya AI

Inkuru eshatu zizewe za AI buri gitondo cy'iminsi y'akazi, zanditse mu rurimi rworoshye. Ubuntu iteka ryose, nta kwamamaza.

Imeri imwe buri munsi y'akazi. Kwiyimura mu buryo bumwe. Ntitwigera tugurisha cyangwa ngo dusangire aderesi yawe.

Isuzume

Take the DDPM and DDIM Samplers quiz

Ibisubizo ako kanya kuri buri kibazo, n'icyemezo gishobora gusangirwa gifite ID yagenzurwa umaze gutsinda isomo.

Tangira ikibazo

Fasha uburezi bwa AI bw'ubuntu. AI Understanding ni umuryango utegamiye kuri leta wa 501(c)(3) — nta kwamamaza, nta kwishyura, iteka ryose. Tanga impano

Ibibazo bikunze kubazwa

What is DDPM and DDIM Samplers?

DDPM and DDIM are two procedures for running a diffusion model backward, turning pure noise into an image one denoising step at a time. DDPM, from 2020, is the original stochastic sampler and needs about 1,000 steps. DDIM, from 2020 to 2021, reuses the same trained network but takes deterministic shortcuts, reaching similar quality in 20 to 100 steps, a 10 to 50 times speedup.

Ni izihe nyungu nyamukuru za DDIM kurenza DDPM?

DDIM ikurikiza inzira igenamigambi, itari Markoviya ituma isimbuka igihe kinini, ikabyara amashusho meza mu ntambwe zigera ku 10-50 aho kuba amagana.

Ni ubuhe buryo bwo gukwirakwiza imiyoboro y’imitsi yiga guhanura mugihe cy'amahugurwa?

Umuyoboro watojwe guhanura urusaku rwa Gaussiya (epsilon) rwongewe kuri buri gihe cyagenwe, DDPM na DDIM zombi zikoresha noneho kugirango zihindure inzira.

Kuki DDIM ishobora kubyara ishusho imwe kuva urusaku rutangira buri gihe?

DDIM ita ijambo ryijwi ridasanzwe mukuvugurura kwayo, bigatuma inzira iva murusaku igana ishusho igenwa neza bityo ikabyara.

Muri DDIM, niyihe gaciro ka parameter eta igarura imyitwarire yumwimerere ya DDPM?

Ibipimo bya eta bihuza hagati yintangarugero zombi: eta = 1 itanga DDPM yuzuye idakuka, mugihe eta = 0 itanga DDIM yuzuye.

Hafi yintambwe zingahe DDPM yumwimerere yari ikeneye kuburugero rwiza?

DDPM isubira inyuma murwego rwose rwurusaku rwongeramo ibintu bidasanzwe, mubisanzwe rero bisabwa kurutonde rwamajana kugeza ku gihumbi.