Transformatëru diffusion
Transformatëri diffusion (DiTs) dañuy wecci U-Net bi ci xol biy defar nataal ak wideo ngir am yaxu ndigg Transformatër.
Résumé
This architecture powers leading systems like Stable Diffusion 3 and OpenAI's Sora, and it scales remarkably well as you add compute.
Plongeur bu xóot
Modèle diffusion yi dañuy defar ay nataal ci tàmbalee ci bruit bu sell ba noppi dindi bruit bi ci anam wu leer. Ay at ci ginaaw reso bi daan dindi bruit bi U-Net la woon, di architecture buy boole. Transformatëru diffusion bi Peebles ak Xie dugal ci 2022, mooy wecci U-Net ak Transformatër. Nataal bi dañu koy njëkka kompresse ci barab bu nëbbu, ñu xaaj ko ay pàcc yu ndaw, pàcc bu nekk nekk token, lu melni kàddu yi ci modelu làkk. Transformer bi dafay jëfandikoo token yooyu ak xel mu dal ci jéego bu nekk ci denoising. Li gëna am solo ci li ñu gis mooy ni DiT dafay gëna yokk bu baax soo yokkee dayo model bi ak wàññi dayo patch bi, topp yooni scaling yu sell yi. Scalability bi moo waral sistemu bind-ci-wideo ak sistemu bind-ci-nataal yu mujjee toxu ci Transformer backbones.
Gis-gis xarala
Benn ci coppite yu am solo yi mooy anam wi DiTs di defee ay kondisioneer yu melni timestep ak bind. Duñu def benn lëkkaloo bu yomb, waaye dañuy jëfandikoo normalisasioŋ buy méngoo ak couche yi (adaLN), foofu la reso bi di seetlu eskaal bi ak parametre yiy soppi couche normalisasioŋ yi ci siñaalu kondisioneer bi. Variante adaLN-zero dafay tàmbali yii suko defee bloc bu nekk tàmbali ni fonction dàntite, di dakkal tàggat bi. Patch yi dañu leen platre ci ay token, ñu toppatoo leen ci bloc Transformer yi ñuy bàyyi xel ci seen bopp, ba noppi ñu boole leen ba noppi dekodewaat leen ci pixel yi.
njeextalu pexe
Gaawaay ak yaatuwaay
Visual IA mën na otomatise saytu, gis ak etiketu liggéey ci eskaal.
Tabax tànneef
Ekipu kreatif yi mën nañu defar konsept yu gëna gaaw te duñu def lu bari ci loxo.
Ekip ak def liggéey
Liggéeyukaay yi mën nañu jëfandikoo siñaal nataal wala wideo yu jafewoon lool ci liggéey.
Ëlëgu Transformatëri Diffusion
Transformatëri diffusion yi ñoo nekk yax giñ jagleel mejaa yiy génne ay mbir. Seenug jëmmal bu lalu ci token daf leen di def ñu mëna boole nataal yi, wideo yi, ba ci defar ay anam yu bari ci benn architecture buñ mëna escaleer. Gëstu dafay push ci wideo yu gëna gudd, dayo bu gëna rëy, ak gëna bàyyi xel ci njëgu quadratic bu token yu bari. Xaarandil lëkkaloo gi am ci digganté làkk yi ak xeetu gis-gis yi, fu ay rëset ak jumtukaayi jëfandikoo Transformer yu noonu mel, di gaawlu yokkute ci xeetu àdduna ak wideo yuy weccoo xalaat.
Doxal ci àdduna dëgg
OpenAI's Sora uses a Transformer backbone over spacetime patches to generate minute-long, high-fidelity videos from text prompts.
Diffusion 3 bu dëgër dafay jëfandikoo transformatëru diffusion multimodal (MMDiT) ngir gëna méngale nataal yiñ defar ak ay leeral ci mbind mi.
Gëstukat yi dañuy eskale DiT ba ci ay miliyaar ciy parametre, ba noppi xool ni kalite nataal bi di gëna yokk, di tegtal dogal yi ci wàllu budget ordinatër.
Benn istijoo dafay jëfandikoo xeetu DiT ngir yokk clip yu gàtt, di jëfandikoo kadre wideo yu bari ni yeneen token patch ngir dindi bruit.
Risk yi ak balustrade yi
Yelleefi nataal ak nangu mën na nekk risku yoon sudee fi ñu bawoo leerul.
Performance model bi mën na wuute ci leeraay bi, demographie bi ak environmaa bi.
Njuumteg positive yi mën nañu dem te kenn duko seetlu fileek xool wuñu buntu wóolu sa bopp.
Roadmap ngir samp gi
Mandargal kritërium nangug njub, woowaat ak njëgu njuumte.
Saytu ak done yu méngoo ak anam yi ñuy liggéeyee dëgg.
Yokk jàngat nit ngir xam fu wóorul dara wala am njeexital yu rëy.
Toppal model drift bi nga baaxal ko ginaaw bi kamera bi wala done yi soppeekoo.
Weyal di banneexu
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Diffusion Transformers quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gis bi ci topp
Reseau transformateur spatial
Laaj yi ñuy faral di laaj
What is Diffusion Transformers?
Transformatëri diffusion (DiTs) dañuy wecci U-Net bi ci xol biy defar nataal ak wideo ngir am yaxu ndigg Transformatër. Architecture bii dafay dooleel sistem yu mag yu melni Stable Diffusion 3 ak AIU_PROTECTED_12_'s, te dafay yokk bu baax sooy yokk ordinatër.
Lan mooy wecci transformatëru diffusion buñu ko méngale ak xeetu diffusion yiñ gëna xam?
DiTs yi ñooy wecci U-Net, di reso convolutionnel biy def denoising, ak yaxu ndigg Transformer.
naka lay def ba DiT soppi nataal mu nekk lu Transformer mëna def?
Nataal bu nëbbu bi dañu ko xaaj ay pàcc yu ndaw, pàcc bu nekk dafay nekk màndarga, buy nuru kàddu yi ci xeetu làkk.
Lan la këyitu DiT bi njëkk gis ci wàllu eskalaasioŋ?
Kalite DiT dafay gëna baax ci anam wu sell, buñu mëna seentu sooy yokk modelu ordinatër bi ak jëfandikoo ay patch yu gëna ndaw, topp yooni scaling.
naka lay def ba DiTs di dugal ay kondisioneer yu melni timestep ak bind?
DiTs yi dañuy jëfandikoo normalisasioŋ couche adaptif ngir seetlu eskaal bi ak toxal parametre yi ngir couche normalisasioŋ yi ci siñaalu kondisioneer bi.
Lan la 'adaLN-zero' di def?
adaLN-zero mooy tàmbali modulasioŋ bi suko defee blok bu nekk nekk dàntite ci ndoorte li, loolu mooy dakkal ak gëna suqali tàggat bi.