Yadawa Transformers
Diffusion Transformers (DiTs) suna musanya U-Net mai jujjuyawar a zuciyar hoto da janareta na bidiyo don kashin bayan Transformer.
Dubawa
This architecture powers leading systems like Stable Diffusion 3 and OpenAI's Sora, and it scales remarkably well as you add compute.
Zurfafa nutsewa
Samfuran watsawa suna haifar da hotuna ta farawa daga tsattsauran amo da kuma musa shi cikin hoto mai daidaituwa. Tsawon shekaru hanyar sadarwar da ke yin wannan ƙin yarda ita ce U-Net, gine-ginen juyin juya hali. Mai Canjawa Mai Yadawa, wanda Peebles da Xie suka gabatar a cikin 2022, ya maye gurbin U-Net tare da Transformer. Hoton an fara matsa shi zuwa wuri mai ɓoye, an raba shi zuwa ƙananan faci, kuma kowane faci ya zama alama, kamar kalmomi a cikin ƙirar harshe. Sannan Transformer yana aiwatar da waɗannan alamun tare da kulawar kai a kowane mataki na ƙaryatãwa. Wani mahimmin ganowa shine cewa aikin DiT yana haɓaka da tsinkaya yayin da kuke ƙara girman samfuri da rage girman facin, bin ƙa'idodi masu tsafta. Wannan scalability shine dalilin da ya sa tsarin rubutu-zuwa-bidiyo da babban tsarin rubutu-zuwa-hoto sun yi ƙaura zuwa ƙashin baya na Transformer.
Fahimtar Fasaha
Babban bidi'a shine yadda DiTs ke yin alluran sanyaya kamar saurin lokaci da saurin rubutu. Maimakon daidaitawa mai sauƙi, suna amfani da daidaitawa na daidaitawa na daidaitawa (adaLN), inda cibiyar sadarwar ke tsinkayar ma'auni da juzu'i don daidaita matakan daidaitawa daga siginar sanyaya. Bambancin adaLN-sifili yana fara waɗannan don haka kowane toshe yana farawa azaman aikin ainihi, yana ƙarfafa horo. Ana karkatar da faci zuwa alamomi, ana sarrafa su ta daidaitattun tubalan Transformer tare da kulawa da kai, sannan a sake haɗa su kuma a canza su zuwa pixels.
Dabarun Tasiri
Gudu da sikelin
Kayayyakin AI na iya sarrafa aiki da bincike, ganowa, da ayyuka masu alama a sikelin.
Gina zaɓuɓɓuka
Ƙungiyoyin ƙirƙira za su iya samar da ra'ayoyi cikin sauri tare da ƙarancin bita da hannu.
Ƙungiya da aikin aiki
Ayyuka na iya amfani da siginar hoto da bidiyo waɗanda a baya suke da wahalar aiwatarwa.
Makomar Diffusion Transformers
Yadawa Transformers suna zama tsohuwar kashin baya don kafofin watsa labarai na samarwa. Ƙirar tushen alamar su ya sa su zama na halitta don haɗa hotuna, bidiyo, har ma da ƙarni na multimodal a ƙarƙashin gine-gine mai ƙima ɗaya. Bincike yana matsawa zuwa ga tsayin bidiyo, ƙuduri mafi girma, da ingantaccen kulawa don daidaita ƙimar ƙima na alamomi da yawa. Yi tsammanin haɗuwa tsakanin nau'ikan harshe da hangen nesa, inda ire-iren girke-girke na sikeli na Transformer da kayayyakin more rayuwa ke aiki duka, haɓaka ci gaba a cikin ƙirar duniya da bidiyo mai mu'amala.
Aiwatar da Gaskiyar Duniya
OpenAI's Sora yana amfani da kashin baya na Transformer akan faci na lokacin sarari don samar da bidiyoyi masu tsayi na mintuna, masu inganci daga saƙon rubutu.
Stable Diffusion 3 yana ɗaukar na'urar watsawa ta multimodal (MMDiT) don daidaita hotuna da aka ƙirƙira tare da cikakkun bayanan rubutu.
Masu bincike suna auna DiT zuwa biliyoyin ma'auni kuma suna lura da ingancin hoto yana inganta yadda ya kamata, suna jagorantar yanke shawara na kasafin kuɗi.
Studio yana amfani da samfurin tushen DiT don tsawaita gajerun shirye-shiryen bidiyo, yana ɗaukar ƙarin firam ɗin bidiyo azaman ƙarin alamun faci don ƙi.
Hatsari & Tsare-tsare
Haƙƙoƙin hoto da yarda na iya zama haxarin doka idan ba a fayyace ba.
Ayyukan samfuri na iya bambanta a ko'ina cikin haske, ƙididdiga, da mahalli.
Ƙarya tabbataccen ƙila ba za a iya lura da shi ba sai dai idan an kula da ƙofofin amincewa.
Taswirar Hanya
Ƙayyade ma'auni na karɓa don daidaito, tunowa, da farashi na kuskure.
Gwada tare da bayanan da suka dace da ainihin yanayin samarwa.
Ƙara bita na ɗan adam don ƙarancin amincewa ko tsinkaya mai tasiri.
Bi diddigin ƙirar ƙira kuma sake ingantawa bayan canje-canjen kamara ko saitin bayanai.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Diffusion Transformers quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Cibiyoyin Taswira na sarari
Tambayoyin da ake yawan yi
What is Diffusion Transformers?
Diffusion Transformers (DiTs) suna musanya U-Net mai jujjuyawar a zuciyar hoto da janareta na bidiyo don kashin bayan Transformer. Wannan gine-ginen yana ba da ikon jagorancin tsarin kamar Stable Diffusion 3 da OpenAI's Sora, kuma yana da ma'auni sosai yayin da kuka ƙara lissafi.
Wane bangare ne Mai Canjawa Mai Yadawa ya maye gurbin idan aka kwatanta da samfuran yadawa na gargajiya?
DiTs sun maye gurbin U-Net, cibiyar sadarwa mai jujjuyawar da ta yi ƙin yarda, tare da kashin baya na Transformer.
Ta yaya DiT ke juya hoto zuwa wani abu da Transformer zai iya aiwatarwa?
Hoton da ke ɓoye ya kasu zuwa ƙananan faci, kuma kowane faci ya zama alama, kwatankwacin kalmomi a cikin ƙirar harshe.
Menene ainihin takardar DiT ta samo game da sikeli?
Ingancin DiT yana haɓaka ta hanya mai tsabta, mai iya faɗi yayin da kuke ƙara ƙididdige ƙira da amfani da ƙananan faci, bin dokokin ƙira.
Ta yaya DiTs yawanci ke yin alluran sanyaya kamar saurin lokaci da saƙon rubutu?
DiTs suna amfani da daidaitawar Layer daidaitacce don tsinkayar sikeli da juzu'i don daidaita yadudduka daga siginar sanyaya.
Menene farkon 'adaLN-zero' ya cim ma?
adaLN-sifili yana fara daidaitawa don haka kowane toshe da farko yana aiki azaman ainihi, wanda ke daidaitawa da haɓaka horo.