概述
目前大多數工具使用擴散模型,從隨機噪音開始,逐步清理直到出現符合提示的圖像。這很重要,因為這些工具現在既便宜又廣泛可用,使它們在設計、插圖以及日益增長的可信假圖源方面變得有用。
重點摘要
- 大多數 AI 圖像生成器使用擴散模型,從隨機噪音開始,並在文字提示的指導下逐步去除噪音。
- Latent diffusion,即 Stable Diffusion 背後的方法,將去噪過程放在壓縮空間中,這使得在消費級顯示卡上實現生成成為可能。
- OpenAI 的 CLIP 透過 4 億張圖像與標題對學習將圖片與文字連結,成為文字到圖像系統的通用橋樑。
- Stable Diffusion 在 2022 年 8 月 22 日的公開發布僅需約 6.9 GB 的顯示記憶體,讓家庭用戶也能進行圖像生成。
- 美國版權局於 2025 年 1 月結論,僅有提示並不足以讓 AI 生成的圖像具備版權保護。
- 生成的圖像並不能證明事件發生過,因此像 C2PA 內容憑證等來源標籤以及明確披露都很重要。
深入探討
An AI image generator is a program that has studied a very large collection of images, usually paired with captions, and learned the visual patterns that connect words to pictures. When given a prompt such as a red fox in snow, watercolor style, it does not search for an existing picture or paste together pieces of stored photos. It produces new pixels that are statistically consistent with what it learned about foxes, snow and watercolor. The output can be a finished image, a variation of an uploaded image, or an edit of one region of a photo.
The most widely used mechanism today is diffusion. During training, the system takes real images and adds random noise to them in many small steps until they become pure static. A neural network learns to reverse one step at a time by predicting the noise that was added. To generate, the process runs backward: start with random static, ask the network to remove a little noise, repeat dozens of times, and an image gradually emerges. A text prompt steers each step so that the emerging image matches the description. Because each run starts from different random noise, called the seed, the same prompt gives different images every time unless the seed is fixed.
Two supporting pieces make text-to-image work. The first is a text encoder that turns the prompt into numbers the image model can use. OpenAI's CLIP, posted in February 2021, learned to match images with captions from 400 million image and text pairs collected from the internet, and models like it became the bridge between language and pictures. Google's Imagen paper in May 2022 found that using a larger text-only language model as the encoder improved image quality and prompt matching more than enlarging the image model itself. The second piece is efficiency. Running diffusion on full-size pixels is slow, so the 2021 latent diffusion paper by Rombach and colleagues, which became the basis of Stable Diffusion, first compresses images with an autoencoder, a network that squeezes an image into a much smaller grid of numbers and can expand it back. Diffusion then runs in that compressed latent space, and the result is decoded into pixels at the end. The paper also added cross-attention layers so the model can be conditioned on text, layouts or other inputs.
Most tools offer several modes built on the same model. Text-to-image creates a picture from a prompt alone. Image-to-image starts from an uploaded picture partly covered in noise, so the result keeps its rough composition while changing style or detail. Inpainting regenerates only a masked region, such as replacing a sky or removing an object, and outpainting extends an image beyond its original borders; the latent diffusion paper reported state-of-the-art inpainting results. Some systems also accept layouts, sketches or bounding boxes that fix where objects should appear, which the same paper's cross-attention design was built to support. Three settings matter in almost every tool. The seed fixes the starting noise so a result can be reproduced. The number of steps trades speed for refinement. The guidance strength controls how literally the model follows the prompt, with high values giving closer adherence but less variety and sometimes harsh, oversaturated images. Upscaling then enlarges the final picture, but it invents plausible detail rather than recovering real detail that was never there.
The field moved quickly. Generative adversarial networks (GANs), introduced by Ian Goodfellow and colleagues in June 2014, pitted a generator network against a discriminator network that tries to spot the fakes. In February 2021 OpenAI described DALL-E, which treated text and image tokens as one long sequence for a transformer to predict, and CLIP appeared the same month. Latent diffusion followed in December 2021. In 2022 came DALL-E 2 in April, Imagen in May and the public release of Stable Diffusion on August 22, which put a capable model on ordinary graphics cards; Stability AI's announcement put its memory use at 6.9 GB. Since then, commercial tools and open models have improved resolution, prompt following and the ability to render text inside images.
It helps to compare the main approaches. GANs generate an image in a single pass, so they are fast, but they are hard to train and tend to cover a narrower range of images. Autoregressive models, such as the first DALL-E, build an image piece by piece like writing a sentence, which is flexible but slow for large images. Diffusion models are slower at generation because they need many denoising steps, but they train stably and produce varied, detailed results, which is why they dominate current tools. Compared with stock photography or human illustration, a generator is fast and cheap per image, but it cannot guarantee that a picture shows anything that really happened, and its legal status is less settled.
The limitations are concrete. Models often struggle with exact counts, precise spatial relationships, readable text and consistent details such as hands or the same character across several images. They reproduce patterns in their training data, which can include stereotypes about who appears in which roles. Because training sets were largely scraped from the web, there are open legal disputes about whether training on copyrighted images without permission is lawful; the US Copyright Office released a pre-publication report on generative AI training in May 2025. Separately, the Office's January 2025 report on copyrightability concluded that prompts alone do not provide enough human control for the output to be protected, although human-authored elements, creative selection and arrangement, and modifications can be. Realistic fakes of real people and events are the most serious misuse, which is why provenance standards such as C2PA Content Credentials record how an image was made.
In practice, there are two ways to use image generation. Hosted services run the model in the cloud and usually charge a subscription or per-image credits, with the provider's content rules and terms of use deciding what is allowed and who may use the outputs commercially. Open models such as Stable Diffusion can be run locally; the original 2022 release needed a graphics card with about 6.9 GB of memory, and its license allowed commercial and non-commercial use subject to use restrictions. Training a new model from scratch is far more expensive: the latent diffusion paper notes that pixel-space diffusion models often needed hundreds of GPU days to train, which is the cost its compressed approach was designed to reduce. For most people the real work is iteration: writing clear prompts, generating several candidates, editing the best one, checking it at final size, and labeling it honestly as AI-generated.
04工作範例
執行並評估受控的文字到圖像實驗
寫一個具體的提示,說明主題、場景、風格和構圖,例如:傍晚時分的燈塔,平面向量插畫,廣角鏡頭,左側空白天空作為標題。
生成四張不同隨機種子的圖像,並注意即使提示相同,構圖和細節也會有所變化。
固定最佳結果的種子,並改變提示中的一個元素,例如將傍晚改為正午,觀察哪些元素保持穩定,哪些會改變。
如果工具提供指導或提示強度設定,請嘗試低值與高值,並比較每張圖像與提示的貼合程度之間的多樣性。
以完整尺寸檢視所選圖像,留意常見錯誤:文字扭曲、物件多餘或缺失、不可能的幾何形狀以及光線不一致。
記錄工具、模型版本、提示、種子與設定,並在公開使用圖像前檢查工具的條款與是否有任何內容憑證。
它顯示了什麼
學習者最終會得到可重現的圖像與製作過程的文字紀錄。檢查方式很簡單:以相同的種子與設定重新執行相同模型,應產生相同或幾乎相同的圖像,且檢查筆記會顯示其是否適合預期的用途與尺寸。
現實世界的實施
在家用電腦上開放模型:Stability AI 於 2022 年 8 月 22 日公開發布 Stable Diffusion,授權允許商業使用,且模型權重約佔 6.9 GB 顯示卡記憶體。
改為編輯而非從頭繪製:2021 年的潛在擴散論文(支援 Stable Diffusion)報告了在圖像修補(inpainting)方面的最先進成果,該技術可填補照片的遮罩區域,例如移除物件或延伸背景。
設計與概念藝術:團隊從文字提示產生許多快速變體的場景、產品拍攝或版面配置,然後手動挑選並精緻化其中一個,保持人類選擇與編輯的迴路。
來源標籤:C2PA 標準,其指導委員會包括 Adobe、Google、Microsoft、OpenAI 以及 Sony,附加內容憑證以記錄影像來源及其編輯方式。
版權決策:2025 年 1 月,美國版權局結論認為僅有提示不足以提供足夠控制力,使 AI 生成的影像具備版權保護,雖然人類的選擇、編排與編輯可受到保護。
AI 圖像生成的未來:文字到圖像模型如何運作
Research is moving along several lines, none of them settled. Control is one: users want exact layouts, consistent characters across a series and correct text inside images, and the 2021 latent diffusion paper's cross-attention design already allowed conditioning on layouts and other inputs beyond text. Speed is another, with work on samplers and distilled models that need far fewer denoising steps than early diffusion systems. Provenance and trust are a third. The C2PA coalition, whose steering committee includes Adobe, Google, Meta, Microsoft and OpenAI, is promoting Content Credentials so that the history of an image can travel with it, although labels can be stripped when files are copied or screenshotted. Law and policy are still forming. The US Copyright Office's January 2025 report concluded that prompts alone do not make an output copyrightable, and its May 2025 pre-publication report on generative AI training addresses whether training on copyrighted works is lawful, a question also being tested in courts. Evaluation remains difficult because automated scores such as FID do not capture whether an image followed every part of a prompt, which is why benchmarks like Imagen's DrawBench, introduced in 2022, rely on side-by-side human ratings. Video and 3D generation extend the same diffusion ideas, and their progress will raise the same questions about consent, labeling and authorship.
付諸實踐
為什麼這很重要
- 速度與規模. 視覺人工智慧可以大規模自動化檢查、檢測和標記任務。
- 配裝選擇. 創意團隊可以透過更少的手動修改來更快地建立概念原型。
- 團隊與工作流程. 操作可以使用以前難以處理的影像和視訊訊號。
留意
- 如果出處不明,肖像權和同意可能會成為法律風險。
- 模型表現可能因光照、人口統計和環境的不同而有所不同。
- 除非監控置信閾值,否則誤報可能會被忽略。
後續步驟
- 定義精確度、召回率和錯誤成本的接受標準。
- 使用符合實際生產條件的數據進行測試。
- 為低置信度或高影響力的預測添加人工審核。
- 追蹤模型漂移並在相機或資料集變更後重新驗證。
資料來源與延伸閱讀
- arXiv (Goodfellow et al., 2014)生成對抗網絡
- arXiv (Ramesh et al., OpenAI, 2021)零樣本文字到影像生成
- arXiv (Radford et al., OpenAI, 2021)從自然語言監督學習可轉移視覺模型
- arXiv / CVPR 2022 (Rombach et al.)使用潛在擴散模型的高解析度影像合成
- arXiv (Saharia et al., Google, 2022)具備深度語言理解的寫實文字到影像擴散模型
- Stability AIStable Diffusion 公開發佈
- U.S. Copyright Office版權與人工智慧
- C2PAC2PA:內容來源與真實性聯盟
不斷探索
常見問題
什麼是 AI 圖像生成:文字轉圖像模型如何運作?
AI 圖像生成是利用訓練過的神經網路,根據文字描述、草圖、其他圖像或標示要變更區域的遮罩,創造全新圖片的技術。大多數現有工具使用擴散模型,從隨機噪聲開始,逐步去噪,直到產生符合提示的圖像。這很重要,因為這些工具現在成本低廉且廣泛可得,對設計與插畫非常有用,同時也成為製造逼真偽造品的日益增長的來源。
哪種學習過程描述了擴散圖像模型的特性?
指南指出擴散模型學習去噪過程;此描述並不意味輸出是紀錄性的文件。
一項產品結合了圖像生成、編輯、升級與後製。團隊應該評估什麼?
指南說明產品可能結合多個階段,因此完整的工作流程與最終產出資產都很重要。
哪些細節可以在提示中協助傳達視覺目標?
指南將這些列為有用的視覺限制,同時警告某些細節仍需直接檢查。
圖表需要精確的小標籤與重複的幾何圖形。創作者應該怎麼做?
指南指出精確文字與重複幾何需要直接檢查;其範例建議在需要時使用可編輯文字或向量元素。
為什麼要在預期的顯示尺寸下檢查圖像?
指南建議進行最終尺寸檢查,因為縮圖可能會隱藏在橫幅或印刷版面中可見的問題。