DreamFusion uye Score Distillation Sampling
DreamFusion inogadzira zvinhu zve3D kubva muzvinyorwa nekushandisa 2D mufananidzo diffusion modhi semutsoropodzi, kusambodzidzira pane chero data re3D.
Pfupiso
Its core invention, Score Distillation Sampling, became the foundational recipe for the entire text-to-3D field.
Kudzika Kwakadzika
DreamFusion, kubva kuGoogle muna 2022, yakabvunza kuti: ko 2D text-to-image modhi ingadzidzise chiitiko che 3D kutarisa chaiko kubva kumakona ese? Iyo inokwidziridza NeRF (Neural Radiance Field) kuitira kuti kushandura kubva kune isina kujairika kamera maonero, kana yanzwika uye ichiratidzwa kune yakaomeswa diffusion modhi (Mufananidzo), zvibodzwa semifananidzo inonzwisisika yechinyorwa chekukurumidza. Chinonyanya kushandisa hapana 3D yekudzidzisa data. Mhedzisiro iyi ndeye Score Distillation Sampling (SDS): pachinzvimbo chekudzokera kumashure kuburikidza neiyo diffusion modhi inodhura U-Net, SDS inoshandisa iyo modhi yakafanotaurwa ruzha sechiratidzo chegradient zvakananga pamapikisi akapihwa. Kudzokorodza izvi muzviuru zvemaonero kunoveza chinhu chakabatana che3D, chakakwana nejometry uye chitarisiko chinotsamira pakuona, kubva mumutsara mumwe.
Technical Insight
SDS inobata iyo yekuparadzira modhi seyakaomeswa nechando basa. Inopa iyo NeRF, inowedzera ruzha, inobvunza kupararira U-Net kufanotaura ruzha irworwo, uye inoverengera gradient se (yakafanotaurwa ruzha minus yakawedzera ruzha) inosundirwa kumashure pamufananidzo wakapihwa uye nekudaro huremu hweNeRF. Kusvetuka iyo U-Net Jacobian inoita kuti ibatike. Yepamusoro classifier-yemahara nhungamiro (yakatenderedza zana) inodiwa pamhedzisiro inopinza, izvo zvinokonzeresa kuti hunhu huwedzere kuzara, dzimwe nguva kudzima 'DreamFusion look.'
Strategic Impact
Kumhanya uye chiyero
Visual AI inogona kuita otomatiki yekuongorora, yekuona, uye yekumaka mabasa pachiyero.
Vaka sarudzo
Zvikwata zvekugadzira zvinogona prototype pfungwa nekukurumidza nekudzokororwa kwemaoko mashoma.
Team uye workflow
Mashandisirwo anogona kushandisa masaini emifananidzo nemavhidhiyo ayo aimbove akaoma kugadzirisa.
Ramangwana re DreamFusion uye Score Distillation Sampling
SDS yakaburitsa mutsara wakapfuma webasa kugadzirisa kusasimba kwayo: Magic3D yekugadzirisa nekumhanya, ProlificDreamer's Variational Score Distillation yekupinza, yakawanda siyana kubuda, uye nzira dzinorwisa iyo 'Janus' yakawanda-chiso chekugadzira. Munda uri kuwedzera kubatanidza SDS neakawanda-maonero ekupararira ekutanga uye nekukurumidza 3D inomiririra seGaussian Splatting. Tarisira kuti mavara-kusvika-ku-3D akure nekukurumidza uye nekutendeka kwe geometrically, zvichidzikamisa gaka nemidziyo yakagadzirwa nemaoko.
Real-World Implementation
Kugadzira 3D modhi ye 'DSLR pikicha yetsindi yakapfeka ngowani diki' kubva pane zvinyorwa chete
Kugadzira dhizaini mutambo uye AR midziyo isina bhuku re 3D kuveza
Kugadzira meshes inotengeswa kunze uko maartist anonatsa pane kuvaka kubva pakutanga
Tsvagiridzo yekutanga yekuongorora mitsva yemavara-ku-ku-3D nzira kurwisa SDS
Njodzi & Guardrails
Kodzero dzemifananidzo uye kubvumirwa kunogona kuve njodzi dzepamutemo kana provenance isina kujeka.
Kuita kwemuenzaniso kunogona kusiyanisa kupenya, huwandu hwevanhu, uye nharaunda.
Manyepo enhema anogona kusacherechedzwa kunze kwekunge zvikumbaridzo zvekuvimba zvikatariswa.
Implementation Roadmap
Tsanangura maitiro ekugamuchirwa echokwadi, kurangarira, uye mutengo wekukanganisa.
Edzai nedata rinoenderana nemamiriro chaiwo ekugadzira.
Wedzera ongororo yemunhu kune yakaderera-kusavimbika kana yakakwirira-inokanganisa kufanotaura.
Tevera modhi kudonha uye simbisa mushure mekuchinja kwekamera kana dataset.
Ramba Uchiongorora
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the DreamFusion and Score Distillation Sampling quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gaidhi rinotevera
Zvibodzwa-Based Generative Models
Mibvunzo inowanzo bvunzwa
What is DreamFusion and Score Distillation Sampling?
DreamFusion inogadzira zvinhu zve3D kubva muzvinyorwa nekushandisa 2D mufananidzo diffusion modhi semutsoropodzi, kusambodzidzira pane chero data re3D. Yayo yakakosha kugadzirwa, Score Distillation Sampling, yakave iyo yekutanga resipi yeese mameseji-ku-3D ndima.
Ndeipi 3D inomiririra iyo DreamFusion inokwenenzvera?
DreamFusion inokwidziridza NeRF kuitira kuti maonero ayo akashandurwa agutse 2D diffusion modhi mutongo wekukurumidza kwemavara.
Yakawanda sei 3D yekudzidziswa data inodiwa DreamFusion?
DreamFusion inoshandisa yakaomeswa 2D mavara-kune-image diffusion modhi sekutarisa kwayo chete, isingade 3D data yekudzidziswa.
Ndeipi yakakosha computational shortcut muScore Distillation Sampling?
SDS inoshandisa iyo yakafanotaurwa-minus-akawedzera ruzha seyeredhiyo yakananga pamapikisheni akashandurwa, kudzivirira inodhura U-Net Jacobian.
Nei DreamFusion ichishandisa yakakwira classifier-yemahara nhungamiro (~ 100)?
Iyo SDS gradient ine ruzha uye haina kusimba, saka zvisingaite kutungamira kwepamusoro kunodiwa kune crisp, on-prompt geometry, kunyangwe ichikonzeresa-saturation.
Ndeipi 2D modhi yakashandisa yekutanga DreamFusion seyakaomeswa nechando?
DreamFusion yakavakirwa pa Google's Imagen mavara-kune-mufananidzo diffusion modhi seyakaomeswa zvibodzwa.