DreamFusion kanye Score Distillation Sampling Guide

Uhlolojikelele

Its core invention, Score Distillation Sampling, became the foundational recipe for the entire text-to-3D field.

I-Deep Dive

I-DreamFusion, evela ku-Google ngo-2022, yabuza: ingabe imodeli ye-2D yokuguqula umbhalo ube isithombe ingafundisa isigcawu se-3D ukuthi sibheke ngqo kuwo wonke ama-engeli? Ithuthukisa i-NeRF (Neural Radiance Field) ukuze ukunikezwa okuvela ezindaweni zokubuka zekhamera ezingahleliwe, lapho kunomsindo futhi kuboniswa imodeli yokusabalalisa efriziwe (Isithombe), kufinyelele njengezithombe ezinengqondo zokwaziswa kombhalo. Okubaluleke kakhulu ayisebenzisi idatha yokuqeqeshwa kwe-3D. Impumelelo i-Score Distillation Sampling (SDS): esikhundleni sokuhlehla nge-U-Net ebizayo yemodeli yokusabalalisa, i-SDS isebenzisa umsindo obikezelwe wemodeli njengesignali yegradient ngokuqondile kumaphikseli anikeziwe. Ukuphinda lokhu ezinkulungwaneni zemibono kudweba impahla ye-3D ehambisanayo, egcwele i-geometry nokubukeka okuncike ekubukeni, emshweni owodwa.

I-Technical Insight

I-SDS iphatha imodeli yokusabalalisa njengomsebenzi wokushaya amagoli. Inikeza i-NeRF, yengeze umsindo, icela ukusabalalisa kwe-U-Net ukuthi ibikezele lowo msindo, futhi ibale ukuthambeka njengokuthi (umsindo obikezelwe kususwe umsindo owengeziwe) ududulwe emuva esithombeni esinikeziwe kanjalo nezisindo ze-NeRF. Ukweqa i-U-Net Jacobian kuyenza ithandeke. Kudingeka isiqondiso esingena-classifier eliphezulu (esilinganiselwa ku-100) ukuze uthole imiphumela ebukhali, ebangela isici esigcwele ngokweqile, kwesinye isikhathi sifiphele 'I-DreamFusion look.'

I-Strategic Impact

Isivinini nesikali

I-Visual AI ingakwazi ukuhlola, ukutholwa, nokumaka imisebenzi esikalini.

Yakha ukukhetha

Amathimba aqanjiwe angakwazi ukulinganisa imiqondo ngokushesha ngezibuyekezo ezimbalwa ezenziwa mathupha.

Ithimba kanye nokusebenza komsebenzi

Imisebenzi ingasebenzisa amasiginali wesithombe nawevidiyo obekunzima ukuwenza ngaphambilini.

Ikusasa le-DreamFusion kanye ne-Score Distillation Sampling

I-SDS iveze umugqa ocebile womsebenzi wokulungisa ubuthakathaka bayo: I-Magic3D yokuxazulula nesivinini, i-ProlificDreamer's Variational Score Distillation ukuze uthole ukuphuma okubukhali, okuhlukahlukene kakhulu, nezindlela ezihlasela i-artifact yobuso abaningi ye-'Janus'. Inkambu iyanda ngokumatanisa i-SDS nezinto eziphambili zokusabalalisa ukubuka okuningi kanye nezethulo ezisheshayo ze-3D ezifana ne-Gaussian Splatting. Lindela umbhalo oya ku-3D ukuze ukhule ngokushesha futhi uthembeke ngokwejometri kakhudlwana, unciphise igebe ngamafa emodeli yesandla.

Ukuqaliswa Komhlaba Wangempela

Ukukhiqiza imodeli ye-3D 'yesithombe se-DSLR sengwejeje egqoke isigqoko esincane' ngombhalo wodwa

Ukudala umdlalo osalungiswa kanye nezimpahla ze-AR ngaphandle kokubaza kwe-3D okwenziwa ngesandla

Ukukhiqiza ama-meshes athekelisa abaculi abawacwengayo esikhundleni sokwakha kusukela ekuqaleni

Ucwaningo oluyisisekelo lokuhlola izindlela ezintsha zombhalo uye ku-3D ngokumelene ne-SDS

Izingozi & Guardrails

Amalungelo ezithombe kanye nemvume kungaba ubungozi bezomthetho uma ukuvela kungacacile.

Ukusebenza kwemodeli kungahluka kukho konke ukukhanya, izibalo zabantu, kanye nezindawo.

Okuhle okungelona iqiniso kungase kungabonakali ngaphandle uma izinga lokuzethemba liqashelwa.

Ukuqalisa Umhlahlandlela

Chaza indlela yokwamukela yokunemba, ukukhumbula, nezindleko zamaphutha.

Hlola ngedatha efana nezimo zangempela zokukhiqiza.

Engeza isibuyekezo somuntu ukuze uthole ukuzethemba okuphansi noma izibikezelo zomthelela omkhulu.

Landelela ukukhukhuleka kwemodeli bese uqinisekisa kabusha ngemva kwezinguquko zekhamera noma zesethi yedatha.

Qhubeka Uhlole

I-Computer Vision

Qonda amasistimu ayisisekelo anika amandla i-AI ebonakalayo.

Funda Umhlahlandlela

I-AI Image Generation

Hlola ukugeleza komsebenzi wokudala kanye nokuhwebelana kwamamodeli.

Funda Umhlahlandlela

Free newsletter

Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.

One email each weekday. Unsubscribe in one click. We never sell or share your address.

Test yourself

Take the DreamFusion and Score Distillation Sampling quiz

Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.

Qala imibuzo

Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation

Umhlahlandlela olandelayo

Amamodeli Okukhiqiza Asuselwa kumaphuzu

Funda umhlahlandlela olandelayo Phequlula yonke imibuzo

Imibuzo evame ukubuzwa

What is DreamFusion and Score Distillation Sampling?

I-DreamFusion ikhiqiza izinto ze-3D embhalweni ngokusebenzisa imodeli yokusabalalisa isithombe engu-2D njengomgxeki, ingalokothi iqeqeshe kunoma iyiphi idatha ye-3D. Ukusungulwa kwayo okuyinhloko, i-Score Distillation Sampling, ibe iresiphi eyisisekelo kuyo yonke inkambu yombhalo kuya ku-3D.

Isiphi isethulo se-3D i-DreamFusion esithuthukisayo?

I-DreamFusion ithuthukisa i-NeRF ukuze ukubukwa kwayo okunikeziwe kwenelise ukwahlulela kwemodeli yokusabalalisa ye-2D yokwaziswa kombhalo.

Ingakanani idatha yokuqeqeshwa kwe-3D i-DreamFusion idinga?

I-DreamFusion isebenzisa imodeli eqandisiwe ye-2D yokusabalalisa umbhalo-kuya-isithombe njengokugadwa kwayo kuphela, okungadingi idatha yokuqeqeshwa kwe-3D.

Isiphi isinqamuleli esibalulekile sekhompyutha ku-Score Distillation Sampling?

I-SDS isebenzisa umsindo obikezelwe wokususa-ongeziwe njengegradient eqondile kumaphikseli anikeziwe, igwema i-U-Net Jacobian ebizayo.

Kungani i-DreamFusion isebenzisa isiqondiso esingena-classifier esiphezulu kakhulu (~100)?

I-gradient ye-SDS inomsindo futhi ayinawo amandla, ngakho-ke ukuqondisa okuphezulu ngokungavamile kuyadingeka kujometri ekhanyayo, esheshayo, nakuba idala ukugcwala kwesikhala ngokweqile.

Iyiphi imodeli ye-2D eyasetshenziswa i-DreamFusion yasekuqaleni njengeqhwa ngaphambili?

I-DreamFusion yakhelwe phezu kwe-Google's Imagen imodeli yokusabalalisa umbhalo-kuya-isithombe njengomsebenzi wokushaya wamagoli.