Uhlolojikelele
U-Marigold uphinda asebenzise imodeli yokusabalalisa yesithombe esiqeqeshwe kusengaphambili (I-Stable Diffusion) ukuze abikezele amamephu ajulile anemininingwane eminingi. Kubonisa ukuthi ungakwazi ukushintsha ulwazi olucebile olubonakalayo lwejeneretha lube ithuluzi lokubona elinembile elinedatha yokuqeqeshwa encane ngokumangalisayo.
I-Marigold Diffusion Depth Estimation ingeyokugeleza komsebenzi wokubona ngekhompyutha okuhumusha noma okukhiqiza imidiya ebonakalayo ukuze ihlaziywe, isebenze, futhi isungule.
I-Deep Dive
U-Marigold (ETH Zurich, CVPR 2024 Best Paper Honourable Mention) uhlela kabusha ukulinganisa kokujula njengenkinga yesizukulwane esinemibandela. Esikhundleni sokuqeqesha inethiwekhi ejulile kusukela ekuqaleni, ishuna kahle i-Stable Diffusion ukuze 'ikhiqize' imephu ejulile ebekwe esithombeni sokufaka. Ukuqonda ukuthi imodeli eqeqeshelwe ukuhlanganisa izithombe ze-photorealistic isivele ifundile ijometri yesigcawu, ukukhanya, kanye nesakhiwo ekujuleni kwendawo yayo ecashile, okuyizinto ezihamba phambili eziwusizo ekujuleni. Ngokumangazayo, i-Marigold yayicutshungulwe kahle kumadathasethi okwenziwa kuphela (njenge-Hypersim kanye ne-Virtual KITTI) kodwa ihlanganisa kahle izithombe zangempela ezingasho lutho. Ikhiqiza ukujula kwesihlobo okungaguquki kwe-affine ngemininingwane emihle kakhulu, nakuba i-denoising ephindaphindayo iyenza ihambe kancane kunamamodeli okuphakela phambili njenge-DepthAnything.
I-Technical Insight
I-Marigold isebenza endaweni ecashile ye-Stable Diffusion. Kokubili isithombe nokujula kwemephu kubhalwe ngekhodi i-VAE efanayo; i-U-Net icushwe kahle ukuze isho umsindo wokujula okucashile okubekwe esithombeni esihlanzekile esicashile. Lapho kucatshangwa khona isebenzisa iluphu evamile ephindaphindayo ephindaphindayo, bese ikhipha amakhodi ukujula okucashile. Ngenxa yokuthi isampula, ukugijima okuningi kungahlanganiswa ukuze kube nokuzinza, ukuhweba kubalwe ukuze kube nokunemba. Kamuva 'i-LCM' nezinguqulo ze-distilled zesinyathelo esisodwa zisika inqwaba yezinyathelo zehle ziye endaweni eyodwa.
I-Mastering Marigold Diffusion Depth Estimation
U-Marigold uphinda asebenzise imodeli yokusabalalisa yesithombe esiqeqeshwe kusengaphambili (I-Stable Diffusion) ukuze abikezele amamephu ajulile anemininingwane eminingi. Kubonisa ukuthi ungakwazi ukushintsha ulwazi olucebile olubonakalayo lwejeneretha lube ithuluzi lokubona elinembile elinedatha yokuqeqeshwa encane ngokumangalisayo. I-Marigold Diffusion Depth Estimation ingeyokugeleza komsebenzi wokubona ngekhompyutha okuhumusha noma okukhiqiza imidiya ebonakalayo ukuze ihlaziywe, isebenze, futhi isungule. Ukuze wakhe ukuqonda okujulile, phatha i-Marigold Diffusion Depth Estimation njengemodeli yokusebenza, hhayi isici esisodwa: chaza imiphumela efiselekayo, ucacise ukucabanga, futhi uhlukanise lokho uhlelo olungakwenza ngokwethembeka kulokho okusadinga ukwahlulela kochwepheshe.
Empeleni, amaqembu aqinile asebenzisa ukunemba kwebhalansi ye-Marigold Diffusion Depth Estimation namaqiniso okusebenza njengekhwalithi yedatha, ukuhluka kokukhanya, nokuvumelana kwamalebula. Babhala imibandela yempumelelo ecacile, ukuhlola okuqhathaniswa nedatha engokoqobo nokugeleza komsebenzi, futhi baphindaphinde ngokusekelwe kumaphethini okuhluleka aqashiwe esikhundleni sokuwina kwebhentshimakhi yesikhathi esisodwa. Yilapho ukuqonda kwethiyori kuguquka kube amandla ahlala njalo kuwo wonke umkhiqizo, inqubomgomo, kanye nokusebenza.
I-Visual AI ingakwazi ukuhlola, ukutholwa, nokumaka imisebenzi esikalini. Ngesikhathi esifanayo, amalungelo ezithombe kanye nemvume kungaba ubungozi bomthetho uma ukutholakala kungacacile. Indlela eqine kakhulu iwukuhlanganisa isivinini sokuhlola nesiyalo sokuphatha: qhuba abashayeli bezindiza, bamba ubufakazi, ushicilele amalogi ezinqumo, futhi ubuyekeze izivikelo ngokuqhubekayo njengoba imodeli yokuziphatha, okulindelwe ngabasebenzisi, kanye nezimfuneko zokulawula zishintsha.
I-Strategic Impact
I-Visual AI ingakwazi ukuhlola, ukutholwa, nokumaka imisebenzi esikalini.
I-Visual AI ingakwazi ukuhlola, ukutholwa, nokumaka imisebenzi esikalini. Ekusetshenzisweni kwekhwalithi ephezulu, lokhu kuhunyushwa emithethweni yokusebenza elinganisekayo, imingcele yobunikazi, nemikhuba yokubuyekeza ephindelelayo ukuze amaqembu akwazi ukukala ukuzethemba esikhundleni sokukala ukungaqondakali.
Amathimba aqanjiwe angakwazi ukulinganisa imiqondo ngokushesha ngezibuyekezo ezimbalwa ezenziwa mathupha.
Amathimba aqanjiwe angakwazi ukulinganisa imiqondo ngokushesha ngezibuyekezo ezimbalwa ezenziwa mathupha. Ekusetshenzisweni kwekhwalithi ephezulu, lokhu kuhunyushwa emithethweni yokusebenza elinganisekayo, imingcele yobunikazi, nemikhuba yokubuyekeza ephindelelayo ukuze amaqembu akwazi ukukala ukuzethemba esikhundleni sokukala ukungaqondakali.
Imisebenzi ingasebenzisa amasiginali wesithombe nawevidiyo obekunzima ukuwenza ngaphambilini.
Imisebenzi ingasebenzisa amasiginali wesithombe nawevidiyo obekunzima ukuwenza ngaphambilini. Ekusetshenzisweni kwekhwalithi ephezulu, lokhu kuhunyushwa emithethweni yokusebenza elinganisekayo, imingcele yobunikazi, nemikhuba yokubuyekeza ephindelelayo ukuze amaqembu akwazi ukukala ukuzethemba esikhundleni sokukala ukungaqondakali.
Ukuqaliswa Komhlaba Wangempela
Ikhipha ukujula okucolisekile ezithombeni zezakhiwo nezomkhiqizo ukuze kukhanye kabusha nemifanekiso ye-3D.
Ikhiqiza amamephu anemininingwane ephezulu asetshenziswa njengesimo sokwenziwa kwesithombe esilawulekayo nevidiyo.
Ifilimu yokusiza kanye namaqembu e-VFX emsebenzini we-matte ne-parallax lapho ukunemba komphetho kubaluleke khona.
Isebenza njengesisekelo socwaningo esibonisa ukuthi ungajwayelanisa kanjani okuphambili kokuzalayo nemisebenzi yokuqagela eminyene.
Amaphethini Okusebenzisa
I-Marigold Diffusion Depth Estimation in practice
Ikhipha ukujula okucolisekile ezithombeni zezakhiwo nezomkhiqizo ukuze kukhanye kabusha nemifanekiso ye-3D.
Ukukhipha ukujula okucolisekile ezithombeni zezakhiwo nemikhiqizo zokukhanyisa kabusha kanye ne-3D mockups Amaqembu ngokuvamile athola imiphumela engcono uma echaza ikhwalithi ephezulu ngaphambili, egcina indlela yokukhuphuka yomuntu yamakesi asemaphethelweni, futhi alandelele kokubili izinzuzo zokukhiqiza nezindleko zamaphutha ngokuhamba kwesikhathi.
I-Marigold Diffusion Depth Estimation in practice
Ikhiqiza amamephu anemininingwane ephezulu asetshenziswa njengesimo sokwenziwa kwesithombe esilawulekayo nevidiyo.
Ukukhiqiza amamephu anemininingwane ephezulu asetshenziswa njengesimo sesithombe esilawulekayo namaQembu akhiqiza amavidiyo ngokuvamile athola imiphumela engcono uma echaza izinga eliphezulu ngaphambili, egcina indlela yokukhuphuka yabantu yamakesi asemaphethelweni, futhi alandelele kokubili izinzuzo zokukhiqiza nezindleko zamaphutha ngokuhamba kwesikhathi.
I-Marigold Diffusion Depth Estimation in practice
Ifilimu yokusiza kanye namaqembu e-VFX emsebenzini we-matte ne-parallax lapho ukunemba komphetho kubaluleke khona.
Ifilimu yokusiza kanye namaqembu e-VFX emsebenzini we-matte kanye ne-parallax lapho okunembayo okubalulekile Amaqembu ngokuvamile athola imiphumela engcono lapho echaza ikhwalithi ephezulu ngaphambili, egcina indlela yokukhuphuka kwabantu yamacala asemaphethelweni, futhi alandelele kokubili izinzuzo zokukhiqiza nezindleko zamaphutha ngokuhamba kwesikhathi.
I-Marigold Diffusion Depth Estimation in practice
Isebenza njengesisekelo socwaningo esibonisa ukuthi ungajwayelanisa kanjani okuphambili kokuzalayo nemisebenzi yokuqagela eminyene.
Isebenza njengesisekelo socwaningo esibonisa indlela yokujwayelanisa okubalulekile okubalulekile emisebenzini yokubikezela okuminyene Amathimba ngokuvamile athola imiphumela engcono uma echaza izilinganiso zekhwalithi ngaphambili, egcina indlela yokukhuphuka yabantu yamakesi asemaphethelweni, futhi elandelela kokubili izinzuzo zokukhiqiza nezindleko zamaphutha ngokuhamba kwesikhathi.
Izingozi & Guardrails
Amalungelo ezithombe kanye nemvume kungaba ubungozi bezomthetho uma ukuvela kungacacile.
Ukusebenza kwemodeli kungahluka kukho konke ukukhanya, izibalo zabantu, kanye nezindawo.
Okuhle okungelona iqiniso kungase kungabonakali ngaphandle uma izinga lokuzethemba liqashelwa.
Ukuqalisa Umhlahlandlela
Chaza indlela yokwamukela yokunemba, ukukhumbula, nezindleko zamaphutha.
Chaza indlela yokwamukela yokunemba, ukukhumbula, nezindleko zamaphutha. Phatha isinyathelo ngasinye njengesango lobufakazi: uma imibandela ingafinyelelwa, misa ukukhishwa, vala igebe, bese unweba ukusetshenziswa.
Hlola ngedatha efana nezimo zangempela zokukhiqiza.
Hlola ngedatha efana nezimo zangempela zokukhiqiza. Phatha isinyathelo ngasinye njengesango lobufakazi: uma imibandela ingafinyelelwa, misa ukukhishwa, vala igebe, bese unweba ukusetshenziswa.
Engeza isibuyekezo somuntu ukuze uthole ukuzethemba okuphansi noma izibikezelo zomthelela omkhulu.
Engeza isibuyekezo somuntu ukuze uthole ukuzethemba okuphansi noma izibikezelo zomthelela omkhulu. Phatha isinyathelo ngasinye njengesango lobufakazi: uma imibandela ingafinyelelwa, misa ukukhishwa, vala igebe, bese unweba ukusetshenziswa.
Landelela ukukhukhuleka kwemodeli bese uqinisekisa kabusha ngemva kwezinguquko zekhamera noma zesethi yedatha.
Landelela ukukhukhuleka kwemodeli bese uqinisekisa kabusha ngemva kwezinguquko zekhamera noma zesethi yedatha. Phatha isinyathelo ngasinye njengesango lobufakazi: uma imibandela ingafinyelelwa, misa ukukhishwa, vala igebe, bese unweba ukusetshenziswa.