Ama-Diffusion Transformers
Ama-Diffusion Transformers (DiTs) ashintshanisa i-U-Net ye-convolutional enhliziyweni yezithombe namajeneretha wevidiyo ukuze athole umgogodla we-Transformer.
Uhlolojikelele
This architecture powers leading systems like Stable Diffusion 3 and OpenAI's Sora, and it scales remarkably well as you add compute.
I-Deep Dive
Amamodeli okusabalalisa akhiqiza izithombe ngokuqala emsindweni omsulwa futhi ngokuphindaphindayo awusho ube yisithombe esibumbene. Iminyaka eminingi inethiwekhi eyenza lokho kukhipha umsindo kwakuyi-U-Net, i-convolutional architecture. I-Diffusion Transformer, eyethulwe ngabakwaPeebles no-Xie ngo-2022, ithatha indawo ye-U-Net ngeTransformer. Isithombe siqale sicindezelwe sibe yindawo ecashile, sihlukaniseke sibe iziqephu ezincane, futhi isiqeshana ngasinye siba uphawu, njengamagama emodeli yolimi. I-Transformer ibe isicubungula lawa mathokheni ngokuzinaka esinyathelweni ngasinye sokwenza umsindo. Okutholakele okubalulekile kwaba ukuthi ukusebenza kwe-DiT kuba ngcono ngokubikezelwa njengoba ukhulisa usayizi wemodeli futhi unciphisa usayizi wesichibi, ulandela imithetho ehlanzekile yokukala. Lokhu kukhula kungakho amasistimu wombhalo uye kuvidiyo kanye neziphetho eziphezulu zombhalo uye esithombeni ziye kakhulu zithuthele ku-Transformer backbones.
I-Technical Insight
Ukuqamba okusha okubalulekile yindlela ama-DiTs afaka ngayo isimo njengesinyathelo sesikhathi nokwaziswa kombhalo. Kunokuhlanganisa okulula, zisebenzisa i-adaptive layer normalization (i-adaLN), lapho inethiwekhi ibikezela isikali kanye namapharamitha okushintsha ezendlalelo zokujwayelekile kusukela kusignali yokumisa. Okuhlukile kwe-adaLN-zero kuqalisa lokhu ukuze ibhulokhi ngayinye iqale njengomsebenzi wobunikazi, ukuzinzisa ukuqeqeshwa. Amapheshana anwetshwa abe amathokheni, acutshungulwe ngamabhulokhi e-Transformer ajwayelekile ngokuzinaka, abese ehlanganiswa futhi ekhoda abuyiselwe abe ngamaphikseli.
I-Strategic Impact
Isivinini nesikali
I-Visual AI ingakwazi ukuhlola, ukutholwa, nokumaka imisebenzi esikalini.
Yakha ukukhetha
Amathimba aqanjiwe angakwazi ukulinganisa imiqondo ngokushesha ngezibuyekezo ezimbalwa ezenziwa mathupha.
Ithimba kanye nokusebenza komsebenzi
Imisebenzi ingasebenzisa amasiginali wesithombe nawevidiyo obekunzima ukuwenza ngaphambilini.
Ikusasa Lama-Diffusion Transformers
Ama-Diffusion Transformers aba umgogodla ozenzakalelayo wemidiya ekhiqizayo. Idizayini yabo esekwe kumathokheni ibenza babe ngokwemvelo ekuhlanganiseni izithombe, ividiyo, ngisho nesizukulwane se-multimodal ngaphansi kwesakhiwo esisodwa esingaka. Ucwaningo luphokophela kuvidiyo ende, ukulungiswa okuphezulu, nokunaka okuphumelelayo ukuze kuncishiswe izindleko ze-quadratic zamathokheni amaningi. Lindela ukuhlangana phakathi kolimi namamodeli ombono, lapho amaresiphi afanayo e-Transformer esikalini nengqalasizinda kusebenza kokubili, kusheshisa inqubekelaphambili kumamodeli omhlaba kanye nevidiyo esebenzisanayo.
Ukuqaliswa Komhlaba Wangempela
OpenAI's Sora isebenzisa i-Transformer backbone phezu kwamapeshi esikhathi sasemkhathini ukuze kukhiqizwe amavidiyo anobude beminithi, athembeke kakhulu ngeziyalezo zombhalo.
I-Stable Diffusion 3 isebenzisa i-multimodal Diffusion Transformer (MMDiT) ukuze iqondanise kangcono izithombe ezikhiqiziwe nezincazelo zombhalo ezinemininingwane.
Abacwaningi bakala i-DiT ibe izigidigidi zamapharamitha futhi babone ikhwalithi yesithombe ithuthuka ngokubikezela, beqondisa izinqumo zebhajethi yekhompyutha.
Isitudiyo sisebenzisa imodeli esekwe ku-DiT ukuze sinwebe iziqeshana ezimfushane, siphathe amafreyimu evidiyo engeziwe njengamathokheni e-patch ukuze enze umsindo.
Izingozi & Guardrails
Amalungelo ezithombe kanye nemvume kungaba ubungozi bezomthetho uma ukuvela kungacacile.
Ukusebenza kwemodeli kungahluka kukho konke ukukhanya, izibalo zabantu, kanye nezindawo.
Okuhle okungelona iqiniso kungase kungabonakali ngaphandle uma izinga lokuzethemba liqashelwa.
Ukuqalisa Umhlahlandlela
Chaza indlela yokwamukela yokunemba, ukukhumbula, nezindleko zamaphutha.
Hlola ngedatha efana nezimo zangempela zokukhiqiza.
Engeza isibuyekezo somuntu ukuze uthole ukuzethemba okuphansi noma izibikezelo zomthelela omkhulu.
Landelela ukukhukhuleka kwemodeli bese uqinisekisa kabusha ngemva kwezinguquko zekhamera noma zesethi yedatha.
Qhubeka Uhlole
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Diffusion Transformers quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Umhlahlandlela olandelayo
I-Spatial Transformer Networks
Imibuzo evame ukubuzwa
What is Diffusion Transformers?
Ama-Diffusion Transformers (DiTs) ashintshanisa i-U-Net ye-convolutional enhliziyweni yezithombe namajeneretha wevidiyo ukuze athole umgogodla we-Transformer. Lesi sakhiwo sinikeza amandla amasistimu aholayo afana ne-Stable Diffusion 3 kanye ne-OpenAI's Sora, futhi ikhula kahle kakhulu njengoba wengeza ukubala.
Iyiphi ingxenye eshintshwayo i-Diffusion Transformer uma iqhathaniswa namamodeli ajwayelekile okusabalalisa?
Ama-DiTs athatha indawo ye-U-Net, inethiwekhi yokuguqulwa eyenza i-denoising, ngomgogodla we-Transformer.
I-DiT isenza kanjani isithombe sibe into engenziwa i-Transformer?
Isithombe esifihlekile sihlukaniswa sibe iziqephu ezincane, futhi ipheshi ngalinye liba uphawu, olufana namagama emodeli yolimi.
Litholeni iphepha lokuqala le-DiT mayelana nokukala?
Ikhwalithi ye-DiT ithuthuka ngendlela ehlanzekile, ebikezelwayo njengoba ukhulisa ikhompuyutha yemodeli futhi usebenzisa iziqephu ezincane, ulandela imithetho yokukala.
I-DiTs ivamise ukuyifaka kanjani isimo njengesinyathelo sesikhathi kanye nomyalo wombhalo?
Ama-DiT asebenzisa ukujwayela kwesendlalelo esiguquguqukayo ukuze abikezele isikali kanye nemingcele yokushintsha yezendlalelo zokujwayelekile kusukela kusignali yokumisa.
Kufezani ukuqaliswa kwe-'adaLN-zero'?
I-adaLN-zero iqalisa ukuguquguquka ukuze ibhulokhi ngayinye isebenze njengobunikazi, obunzinza futhi buthuthukise ukuqeqeshwa.