本頁閱讀時間 9 分鐘
概述
Because its weights were downloadable and ran on a single consumer graphics card, it became the base for a large ecosystem of open image tools, fine-tunes and research.
重點摘要
- Stability AI released Stable Diffusion publicly on August 22, 2022 under the CreativeML OpenRAIL-M license, with downloadable weights.
- Stable Diffusion runs diffusion in a compressed latent space; version 1 downsamples images by a factor of 8, so 512 pixels become 64 latent cells.
- Version 1 pairs an 860 million parameter U-Net with a frozen 123 million parameter CLIP ViT-L/14 text encoder.
- Classifier-free guidance blends conditional and unconditional predictions; the reference script uses a guidance scale of 7.5 and 50 sampling steps.
- SDXL enlarged the U-Net about threefold, and Stable Diffusion 3 switched to a rectified-flow transformer with separate text and image weights.
- In November 2025 the UK High Court held that Stable Diffusion's weights do not store copies of Getty's images.
深入探討
Stable Diffusion is a text-to-image model: you type a description, and it produces an image that matches it. It belongs to a class called diffusion models, which learn to turn random noise into a picture one small step at a time. What made it different from earlier systems such as DALL-E 2 was not only quality but access. Stability AI released the first version publicly on August 22, 2022 under the CreativeML OpenRAIL-M license, which allows commercial and non-commercial use with use-based restrictions, and the weights could be downloaded and run locally. Stability's announcement put the memory need at about 6.9 GB of graphics card memory (VRAM).
The research behind it is the latent diffusion model, described by Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser and Bjorn Ommer in 'High-Resolution Image Synthesis with Latent Diffusion Models', first posted to arXiv on December 20, 2021 and published at CVPR 2022. The paper's key idea is to run the expensive diffusion process inside the latent space of a pretrained autoencoder rather than on full-resolution pixels. An autoencoder is a pair of networks: an encoder that compresses an image into a smaller grid of numbers, called a latent, and a decoder that reconstructs the image from it. The authors argued this keeps visual detail while cutting compute, noting that pixel-space diffusion models often need hundreds of GPU days to train.
Generation works in a few distinct stages. First, a text encoder turns the prompt into a sequence of vectors. In version 1, this is a frozen CLIP ViT-L/14 text encoder with 123 million parameters, according to the CompVis model card. Second, the system starts from a latent filled with random noise. Third, a U-Net, a convolutional network with 860 million parameters in version 1, predicts the noise present in that latent, and a sampler removes a portion of it. The U-Net reads the text vectors through cross-attention layers, which let each region of the image look up the words most relevant to it. This loop repeats for a set number of steps, 50 in the reference script. Finally, the autoencoder's decoder converts the clean latent back into pixels. Because the autoencoder downsamples by a factor of 8, a 512 by 512 image is generated as a 64 by 64 latent.
Prompt adherence is controlled by classifier-free guidance, introduced by Jonathan Ho and Tim Salimans in July 2022. During training, the text condition is sometimes dropped, so the same network learns both a conditional and an unconditional prediction. At sampling time the two predictions are blended, pushing the result further in the direction the text suggests. The CompVis model card notes that checkpoints v1-3 and v1-4 were trained with 10% text-conditioning dropout for this reason, and the reference script uses a guidance scale of 7.5. Higher scales follow the prompt more literally at the cost of variety.
The training data came from LAION, a nonprofit that publishes large image-text datasets scraped from the web. Per the model card, v1-1 trained for 237,000 steps at 256 by 256 on laion2B-en and then 194,000 steps at 512 by 512 on a high-resolution subset, while later checkpoints continued on 'laion-aesthetics v2 5+', a subset filtered for images a classifier rated as more aesthetic. Stability AI provided compute, Runway collaborated, and LAION supported the work.
Later versions changed the architecture substantially, so claims about 'Stable Diffusion' should name a version. SDXL, described in a paper posted July 4, 2023, uses a U-Net about three times larger than earlier versions, a second text encoder, training on multiple aspect ratios, and an optional refinement model. Stable Diffusion 3, described in 'Scaling Rectified Flow Transformers for High-Resolution Image Synthesis' (March 5, 2024), replaced the U-Net with a transformer that keeps separate weights for image and text tokens and lets information flow in both directions. It also moved from standard diffusion to rectified flow, a formulation that connects data and noise along a straight line. The authors reported improved typography, text comprehension and human preference ratings.
Compared with closed services such as DALL-E or Midjourney, the main difference is control. With local weights, users can fine-tune on their own images, add control signals such as edge maps, run without per-image fees, and inspect the model. The trade-offs are that the user must supply hardware, manage safety settings, and accept that quality depends heavily on version, sampler and settings. Stability's 2022 release shipped a safety classifier enabled by default, but in a local install it can be adjusted or removed, which raised concern about misuse.
The limitations matter in practice. Version 1 was pretrained at 256 by 256 and finetuned at 512 by 512, so that is the size it was built for, and SDXL's move to multiple aspect ratios and SD3's focus on typography and text comprehension show which weaknesses the developers themselves set out to fix. Because the training images were scraped from the web, the model inherits whatever is over- or under-represented in that data. Copyright remains contested: in Getty Images v Stability AI, decided by the UK High Court in November 2025, Getty dropped its training claim for lack of a UK connection, the judge held the model was not an infringing copy because its weights do not store copies of Getty's images, and a small number of generated Getty watermarks were found to infringe trademarks. Other jurisdictions and cases may reach different conclusions.
04工作範例
Tracing one image through Stable Diffusion v1
Write a prompt such as 'a red bicycle leaning on a brick wall' and pick an output size of 512 by 512 pixels.
Compute the latent size: divide each side by the downsampling factor of 8 to get a 64 by 64 latent grid.
Pass the prompt through the CLIP ViT-L/14 text encoder to get a sequence of token vectors, and also encode an empty prompt.
Fill the 64 by 64 latent with random Gaussian noise using a fixed seed so the run is repeatable.
For each of 50 sampler steps, have the U-Net predict noise twice (with and without the prompt), combine them with guidance scale 7.5, and remove part of the noise.
Decode the final latent with the autoencoder decoder to get a 512 by 512 image.
Repeat with the same seed but a guidance scale of 2, then 12, and compare how literally each image follows the prompt.
它顯示了什麼
Three images from the same seed that differ only in guidance strength. The check is that the seed reproduces the identical image when rerun with identical settings, and that higher guidance produces a closer but less varied match to the prompt.
現實世界的實施
Running the original CompVis txt2img script, which by default samples a 512 by 512 image in 50 PLMS steps with a guidance scale of 7.5, on a single GPU with 10 GB or more of memory.
Inpainting: masking part of a photo and regenerating only that region from a text prompt, a task where the 2022 latent diffusion paper reported state-of-the-art results.
Using SDXL's separate refinement model as a post-hoc image-to-image pass to sharpen detail in images produced by the SDXL base model, as described in the July 2023 SDXL paper.
Comparing the v1-1 through v1-4 checkpoints, which the CompVis model card documents as trained for different numbers of steps on different LAION subsets, to see how data curation changes output style.
Legal analysis: the UK High Court's November 2025 Getty Images v Stability AI judgment examined whether Stable Diffusion's weights store copies of training images and found they do not.
The Future of Stable Diffusion: How the Open Text-to-Image Model Works
Research since 2022 has moved the family away from its original design. The March 2024 SD3 paper reported predictable scaling trends, with lower validation loss tracking better image quality and human preference, which points research toward larger transformer backbones and flow-based training rather than bigger U-Nets. Open questions include how to measure prompt fidelity reliably, how to make text rendering and spatial relationships dependable, and how to cut the number of sampling steps without losing quality. Legal and governance questions remain open. The November 2025 UK judgment in Getty Images v Stability AI was narrow, and Getty had dropped its training claim for lack of a UK connection, so it did not settle whether training on scraped images is lawful elsewhere. Licensing also varies across versions, so users must read the license attached to each release rather than assuming the original OpenRAIL-M terms apply. Safety tooling, such as default output classifiers and provenance labeling, is another active area, because locally run open weights can be modified by anyone who downloads them.
付諸實踐
為什麼這很重要
- 速度與規模. 視覺人工智慧可以大規模自動化檢查、檢測和標記任務。
- 配裝選擇. 創意團隊可以透過更少的手動修改來更快地建立概念原型。
- 團隊與工作流程. 操作可以使用以前難以處理的影像和視訊訊號。
留意
- 如果出處不明,肖像權和同意可能會成為法律風險。
- 模型表現可能因光照、人口統計和環境的不同而有所不同。
- 除非監控置信閾值,否則誤報可能會被忽略。
後續步驟
- 定義精確度、召回率和錯誤成本的接受標準。
- 使用符合實際生產條件的數據進行測試。
- 為低置信度或高影響力的預測添加人工審核。
- 追蹤模型漂移並在相機或資料集變更後重新驗證。
資料來源與延伸閱讀
- arXiv (CVPR 2022)使用潛在擴散模型的高解析度影像合成
- CompVis, LMU Munich (GitHub)CompVis/stable-diffusion: A latent text-to-image diffusion model
- Stability AIStable Diffusion 公開發佈
- arXivClassifier-Free Diffusion Guidance
- arXivSDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis
- arXivScaling Rectified Flow Transformers for High-Resolution Image Synthesis
- Society for Computers and LawHigh Court issues eagerly-awaited judgment in Getty v Stability AI case
不斷探索
常見問題
What is Stable Diffusion: How the Open Text-to-Image Model Works?
Stable Diffusion is a family of text-to-image models whose first version, released publicly on August 22, 2022, generates pictures by removing noise in a compressed latent space rather than on raw pixels. Because its weights were downloadable and ran on a single consumer graphics card, it became the base for a large ecosystem of open image tools, fine-tunes and research.
從較高的層面來看,擴散模型如何產生影像?
擴散模型學習逆轉雜訊過程:從雜訊開始,迭代地去噪,直到出現連貫的影像。
Where does the original Stable Diffusion v1 pipeline perform its learned denoising work?
Latent computation is an architectural choice; hardware suitability still depends on the setup.
How does text condition the documented original Stable Diffusion v1 denoiser?
The v1 description identifies its text encoder and cross-attention conditioning; later versions can differ.
What does classifier-free guidance combine in the documented sampling formula?
The scale controls a combination of predictions and needs evaluation for the chosen model and settings.
Which statement accurately qualifies a change to sampler step count?
The intended sampling regime matters, so evaluate the actual checkpoint and solver.