Mgbasa Transformers
Mgbasa mgbasa ozi (DiTs) na-agbanwe U-Net convolutional na obi nke onyonyo na ndị na-emepụta vidiyo maka ọkpụkpụ azụ Transformer.
Nchịkọta
This architecture powers leading systems like Stable Diffusion 3 and OpenAI's Sora, and it scales remarkably well as you add compute.
Ime miri emi
Ụdị mgbasa ozi na-emepụta ihe oyiyi site na ịmalite site na mkpọtụ dị ọcha ma na-eweghachi ya ugboro ugboro ka ọ bụrụ foto dị n'otu. Ruo ọtụtụ afọ, netwọk ahụ na-eme mkpesa ahụ bụ U-Net, ụkpụrụ ụlọ na-agbanwe agbanwe. Ihe ntụgharị mgbasa ozi, nke Peebles na Xie webatara na 2022, jiri ihe ntụgharị dochie U-Net. A na-ebu ụzọ pịnye onyonyo a ka ọ bụrụ oghere zoro ezo, kewaa n'ime obere patches, ma nke ọ bụla na-aghọ akara ngosi, dị ka okwu n'ụdị asụsụ. Onye ngbanwe ahụ na-ahazi akara ngosi ndị a na nlebara anya onwe ya na nzọụkwụ ọ bụla na-emebi iwu. Otu nchọta dị mkpa bụ na arụmọrụ DiT na-akawanye nke ọma ka ị na-abawanye nha ụdị ma na-ebelata nha patch, na-agbaso iwu nchacha dị ọcha. Nke a scalability bụ ihe mere ederede-na-video na elu-njedebe ederede-na-ihe oyiyi usoro na-akwaga n'ụzọ dị ukwuu na Transformer azụ ọkpụkpụ.
Nghọta nka nka
Isi ihe ọhụrụ bụ ka DiT si etinye ntụ oyi dị ka nkeji oge na ngwa ngwa ederede. Kama nchikota dị mfe, ha na-eji normalization Layer adaptive (adaLN), ebe netwọk na-ebu amụma ọnụ ọgụgụ na ngbanwe parampat maka normalization n'ígwé si mgbama oyi. Ụdị adaLN-efu na-ebute ihe ndị a ka ngọngọ ọ bụla na-amalite dị ka ọrụ njirimara, na-eme ka ọzụzụ kwụsie ike. A na-akụtu patches ka ọ bụrụ akara, ejiri nlebara anya onwe onye na-ahazi ya site na ọkọlọtọ Transformer blocks, wee chịkọta ya wee dekọọ ya na pikselụ.
Mmetụta atụmatụ
Ọsọ na ọnụ ọgụgụ
Visual AI nwere ike megharịa nyocha, nchọpụta na mkpado ọrụ n'ọtụtụ.
Mee nhọrọ
Otu ndị na-emepụta ihe nwere ike imepụta echiche ngwa ngwa site na ngbanwe akwụkwọ ntuziaka ole na ole.
Team na usoro ọrụ
Ọrụ nwere ike iji onyonyo na akara vidiyo siri ike ịhazi.
Ọdịnihu nke Mgbasa Transformers
Ndị ntụgharị mgbasa ozi na-aghọ ọkpụkpụ azụ azụ maka mgbasa ozi na-emepụta ihe. Nhazi ha dabere na akara ngosi na-eme ka ha bụrụ ihe okike maka ijikọ onyonyo, vidiyo na ọbụna ọgbọ multimodal n'okpuru otu ụkpụrụ ụlọ. Nchọpụta na-akpali n'ebe vidiyo dị ogologo, mkpebi dị elu, na nlebara anya nke ọma karị iji zụọ ọnụ ahịa quadratic nke ọtụtụ akara ngosi. Na-atụ anya nkwekọ n'etiti asụsụ na ụdị ọhụụ, ebe usoro ntụzịaka mgbanwe mgbanwe mgbanwe na akụrụngwa na-eje ozi ma, na-eme ka ọganihu dị ngwa n'ụdị ụwa yana vidiyo mmekọrịta.
Mmejuputa n'ezie n'ụwa
OpenAI's Sora na-eji ọkpụkpụ azụ Transformer gafere patches spacetime iji wepụta vidiyo ogologo nkeji, nke nwere ntụkwasị obi dị elu site na mkpali ederede.
Stable Diffusion 3 na-anakwere mgbanwe mgbanwe mgbasa ozi multimodal (MMDiT) iji hazie onyonyo ewepụtara nke ọma yana nkọwa ederede zuru oke.
Ndị na-eme nchọpụta na-atụba DiT n'ọtụtụ ijeri paramita ma na-ahụ na ịdịmma onyonyo na-abawanye nke ọma, na-eduzi mkpebi mmefu ego.
Ụlọ ihe nkiri na-eji ụdị dabere na DiT ịgbatị obere mkpachị, na-emeso okpokolo agba vidiyo ndị ọzọ dị ka ihe mgbakwunye patch ọzọ iji mebie aha.
Ihe ize ndụ & okporo ụzọ nche
Ikike onyonyo na nkwenye nwere ike bụrụ ihe egwu dị n'iwu ma ọ bụrụ na edoghị anya.
Ọrụ nlereanya nwere ike ịdịgasị iche n'ofe ọkụ, igwe mmadụ, na gburugburu.
Enwere ike ghara ịhụ ihe dị mma ma ọ bụrụ na enyochaghị oke ntụkwasị obi.
Map mmejuputa
Kọwaa ụkpụrụ nnabata maka nkenke, icheta, na ụgwọ njehie.
Nwalee na data dabara na ọnọdụ mmepụta n'ezie.
Tinye nyocha mmadụ maka obere obi ike ma ọ bụ amụma mmetụta dị elu.
Sochie ihe nlere anya wee megharịa ka emechara mgbanwe igwefoto ma ọ bụ dataset.
Nọgide na-eme nchọpụta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Diffusion Transformers quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ntuziaka na-esote
Netwọk Transformer oghere
Ajụjụ a na-ajụkarị
What is Diffusion Transformers?
Mgbasa mgbasa ozi (DiTs) na-agbanwe U-Net convolutional na obi nke onyonyo na ndị na-emepụta vidiyo maka ọkpụkpụ azụ Transformer. Ihe owuwu a na-enye ike na-eduga sistemu dị ka Stable Diffusion 3 na OpenAI's Sora, ọ na-agbakọkwa nke ọma ka ị na-agbakwunye mgbakọ.
Kedu akụkụ nke Transformer na-edochi anya ma e jiri ya tụnyere ụdị mgbasa ozi ọdịnala?
DiT dochie U-Net, netwọk convolutional nke rụrụ arụrụ arụ, jiri ọkpụkpụ Transformer.
Kedu ka DiT si atụgharị onyonyo ka ọ bụrụ ihe Transformer nwere ike hazie?
A na-ekewa onyonyo a ezoro ezo n'ime obere patches, ma nke ọ bụla na-aghọ akara ngosi, yiri okwu n'ụdị asụsụ.
Kedu ihe akwụkwọ mbụ DiT chọtara gbasara scaling?
Àgwà DiT na-akawanye mma n'ụzọ dị ọcha, nke a pụrụ ịkọ amụma ka ị na-ebuli ihe nlereanya ma na-eji obere patches, na-agbaso iwu nhazi.
Kedu ka DiT na-esi ejikarị ịgbanye ntụ oyi dị ka nkeji oge na ngwa ngwa ederede?
Ndị DiT na-eji normalization oyi akwa na-agbanwe agbanwe iji buru amụma ọnụ ọgụgụ na mgbanwe mgbanwe maka ọkwa nhazi nke ọma site na mgbama oyi.
Kedu ihe mmalite 'adaLN-zero' na-arụzu?
adaLN-zero na-ebute modulation ka ngọngọ ọ bụla na-amalite na mbụ dị ka njirimara, nke na-eme ka ọ kwụsie ike ma na-akwalite ọzụzụ.