Overview
Plenoxels yakaratidza kuti unogona kuvaka patsva chiitiko che3D neNeRF-mhando yemhedzisiro pasina neural network zvachose - ingori gidhi remavoxels anochengeta ruvara uye density. Mhedzisiro yacho inodzidzisa ingangoita 100x nekukurumidza kupfuura iyo yekutanga NeRF uku ichifananidza hunhu hwayo hwekuona.
Plenoxels uye Voxel Radiance Minda ndeyemakomputa-kuona mafambiro anodudzira kana kuburitsa midhiya inooneka yekuongorora, mashandiro, uye kugadzira.
Deep Dive
NeRF inowana photorealism asi inononoka nekuti sampuli yega yega inoda kumberi kupfuura nepakati neural network, uye kudzidziswa kunogona kutora maawa kana mazuva. Plenoxels (Sara Fridovich-Keil, Alex Yu et al., 2022) akabvunza mubvunzo unotsamwisa: network yacho inotodiwa here? Mhinduro yavo yakanga iri kwete. Ivo vanomiririra chiitiko senge sparse 3D voxel grid. Imwe neimwe inogarwa voxel inochengeta imwechete opacity kukosha pamwe ne spherical harmonic coefficients iyo encode-inotsamira ruvara. Kupa pixel, sisitimu yacho inopindirana nekatatu hunhu uhwu pamwe nemwaranzi uye inoabatanidza neyakajairwa vhoriyamu render. Nekuti hapana network, chinhu chose chinogadziridzwa zvakananga ne gradient kudzika pane voxel kukosha, yakajairwa kutsvedzerera. Mhedzisiro yemusoro: inofananidzwa mhando neNeRF, yakadzidziswa mumaminitsi paGPU imwechete.
Technical Insight
Maonero-anotsamira ruvara ndiyo yakangwara chikamu. Panzvimbo penetiweki inoburitsa RGB pakona yekutarisa, voxel yega yega inochengeta diki seti yespherical harmonic (SH) coefficients pachiteshi chemavara. Kuongorora iyo SH hwaro munzira yeray inovakazve kuti ruvara rweiyo poindi inochinja sei nemaonero - kutora specular maratidziro uye maratidziro. Opacity inzira-yakazvimirira. Differentiable trilinear interpolation plus volume rendering inoita kuti yega yega voxel idzidziswe zvakananga, saka optimization yakatwasuka, isina network-isina-squares-style fit.
Mastering Plenoxels uye Voxel Radiance Minda
Plenoxels yakaratidza kuti unogona kuvaka patsva chiitiko che3D neNeRF-mhando yemhedzisiro pasina neural network zvachose - ingori gidhi remavoxels anochengeta ruvara uye density. Mhedzisiro yacho inodzidzisa ingangoita 100x nekukurumidza kupfuura iyo yekutanga NeRF uku ichifananidza hunhu hwayo hwekuona. Plenoxels uye Voxel Radiance Minda ndeyemakomputa-kuona mafambiro anodudzira kana kuburitsa midhiya inooneka yekuongorora, mashandiro, uye kugadzira. Kuvaka kunzwisisa kwakadzama, tora Plenoxels uye Voxel Radiance Fields semuenzaniso wekushandisa, kwete chinhu chimwe chete: tsanangura zvinodiwa, kujekesa fungidziro, uye patsanura izvo system inogona kuita nekuvimbika kubva kune ichiri kuda kutonga kwenyanzvi.
Mukuita, zvikwata zvakasimba zvinoshandisa Plenoxels uye Voxel Radiance Fields kuenzanirana nezviri kuitika senge data data, kusiyana kwemwenje, uye kuenderana kwemazita. Ivo vanonyora zvakajeka maitiro ebudiriro, bvunzo vachipokana ne data rechokwadi uye mafambiro ebasa, uye iterate zvichibva pane zvakacherechedzwa maitiro ekutadza kwete kuhwina-nguva imwe chete yebhenji. Apa ndipo apo kunzwisisa kwe theoretical kunoshanduka kuve kugona kwakasimba pane chigadzirwa, mutemo, uye mashandiro.
Visual AI inogona kuita otomatiki yekuongorora, yekuona, uye yekumaka mabasa pachiyero. Panguva imwecheteyo, kodzero dzeMufananidzo uye kubvumirwa kunogona kuve njodzi dzepamutemo kana hunhu husina kujeka. Nzira yakatsiga ndeyekubatanidza kukurumidza kuyedza nekutonga: mhanyisa vatyairi vendege, tora humbowo, buritsa matanda esarudzo, uye urambe uchivandudza chengetedzo semaitiro emuenzaniso, zvinotarisirwa nemushandisi, uye zvinodikanwa zvekutonga.
Strategic Impact
Visual AI inogona kuita otomatiki yekuongorora, yekuona, uye yekumaka mabasa pachiyero.
Visual AI inogona kuita otomatiki yekuongorora, yekuona, uye yekumaka mabasa pachiyero. Mukutumirwa kwemhando yepamusoro, izvi zvinoshandurirwa kuita mitemo inoyerwa yekushanda, miganhu yevaridzi, uye tsika dzekudzokorora dzinodzokororwa kuitira kuti zvikwata zvikwire kuvimba pane kukwidza kusajeka.
Zvikwata zvekugadzira zvinogona prototype pfungwa nekukurumidza nekudzokororwa kwemaoko mashoma.
Zvikwata zvekugadzira zvinogona prototype pfungwa nekukurumidza nekudzokororwa kwemaoko mashoma. Mukutumirwa kwemhando yepamusoro, izvi zvinoshandurirwa kuita mitemo inoyerwa yekushanda, miganhu yevaridzi, uye tsika dzekudzokorora dzinodzokororwa kuitira kuti zvikwata zvikwire kuvimba pane kukwidza kusajeka.
Mashandisirwo anogona kushandisa masaini emifananidzo nemavhidhiyo ayo aimbove akaoma kugadzirisa.
Mashandisirwo anogona kushandisa masaini emifananidzo nemavhidhiyo ayo aimbove akaoma kugadzirisa. Mukutumirwa kwemhando yepamusoro, izvi zvinoshandurirwa kuita mitemo inoyerwa yekushanda, miganhu yevaridzi, uye tsika dzekudzokorora dzinodzokororwa kuitira kuti zvikwata zvikwire kuvimba pane kukwidza kusajeka.
Real-World Implementation
Kurumidza kuvakazve chinhu chakabatwa kuita 3D asset mumaminitsi e-commerce kana museum digitization, panzvimbo yemaawa ekumirira.
Kukurumidza prototyping yenovel-view synthesis pane imwechete mutengi GPU yekutsvaga uye dzidzo.
Kugadzira zvinogadzirika, zvakajeka voxel zvimiro izvo maartist anogona kunyatso ongorora uye kuchekerera, kusiyana neopaque network uremu.
Kushanda semuenzaniso wekudzidzisa kuti chiitiko chinomiririra, kwete kudzidza kwakadzama, ndiko kunoburitsa mhedzisiro yephotorealistic.
Maitiro Ekuita
Plenoxels uye Voxel Radiance Minda mukuita
Kurumidza kuvakazve chinhu chakabatwa kuita 3D asset mumaminitsi e-commerce kana museum digitization, panzvimbo yemaawa ekumirira.
Kurumidza kuvakazve chinhu chakabatwa kuita 3D asset mumaminitsi e-e-commerce kana museum digitization, panzvimbo yemaawa ekumirira Matimu anowanzo kuwana mhedzisiro iri nani kana achinge atsanangura emhando yepamusoro kumberi, chengetedza nzira yekukwira kwevanhu yemakesi emupendero, uye kuteedzera zvese zvakawanikwa zvechigadzirwa uye kukanganisa mutengo nekufamba kwenguva.
Plenoxels uye Voxel Radiance Minda mukuita
Kukurumidza prototyping yenovel-view synthesis pane imwechete mutengi GPU yekutsvaga uye dzidzo.
Nekukurumidza prototyping yenovel-view synthesis pamutengi mumwechete GPU yekutsvagisa uye dzidzo Matimu anowanzo kuwana mhedzisiro iri nani kana achinge atsanangura emhando yepamusoro kumberi, chengetedza nzira yekukwira kwevanhu yemakesi ekumucheto, uye kuteedzera zvese zvakawanikwa zvechigadzirwa nemitengo yekukanganisa nekufamba kwenguva.
Plenoxels uye Voxel Radiance Minda mukuita
Kugadzira zvinogadzirika, zvakajeka voxel zvimiro izvo maartist anogona kunyatso ongorora uye kuchekerera, kusiyana neopaque network uremu.
Kugadzira zvinogadzirika, zvakajeka voxel zvimiro izvo maartist anogona kunyatso ongorora nekuchekeresa, kusiyana neopaque network huremu Matimu anowanzo kuwana mhedzisiro iri nani kana achinge atsanangura emhando yepamusoro kumberi, chengetedza nzira yekukwira kwevanhu yemakesi ekumucheto, uye kuteedzera zvese zvakawanikwa zvechigadzirwa uye mutengo wekukanganisa nekufamba kwenguva.
Plenoxels uye Voxel Radiance Minda mukuita
Kushanda semuenzaniso wekudzidzisa kuti chiitiko chinomiririra, kwete kudzidza kwakadzama, ndiko kunoburitsa mhedzisiro yephotorealistic.
Kushanda semuenzaniso wekudzidzisa kuti chinomiririra chiitiko, kwete kudzidza kwakadzama, ndiko kunoburitsa mafotorealistic mhedzisiro Matimu anowanzo kuwana mhedzisiro iri nani kana achinge atsanangura emhando yepamusoro kumberi, chengetedza nzira yekukwira kwevanhu yemakesi ekumucheto, uye kuteedzera zvese zvakawanikwa zvechigadzirwa uye mutengo wekukanganisa nekufamba kwenguva.
Njodzi & Guardrails
Kodzero dzemifananidzo uye kubvumirwa kunogona kuve njodzi dzepamutemo kana provenance isina kujeka.
Kuita kwemuenzaniso kunogona kusiyanisa kupenya, huwandu hwevanhu, uye nharaunda.
Manyepo enhema anogona kusacherechedzwa kunze kwekunge zvikumbaridzo zvekuvimba zvikatariswa.
Implementation Roadmap
Tsanangura maitiro ekugamuchirwa echokwadi, kurangarira, uye mutengo wekukanganisa.
Tsanangura maitiro ekugamuchirwa echokwadi, kurangarira, uye mutengo wekukanganisa. Bata nhanho yega yega segedhi rehumbowo: kana maitiro asina kusangana, imbomira kuburitsa, vhara gaka, uye wobva wawedzera kushandiswa.
Edzai nedata rinoenderana nemamiriro chaiwo ekugadzira.
Edzai nedata rinoenderana nemamiriro chaiwo ekugadzira. Bata nhanho yega yega segedhi rehumbowo: kana maitiro asina kusangana, imbomira kuburitsa, vhara gaka, uye wobva wawedzera kushandiswa.
Wedzera ongororo yemunhu kune yakaderera-kusavimbika kana yakakwirira-inokanganisa kufanotaura.
Wedzera ongororo yemunhu kune yakaderera-kusavimbika kana yakakwirira-inokanganisa kufanotaura. Bata nhanho yega yega segedhi rehumbowo: kana maitiro asina kusangana, imbomira kuburitsa, vhara gaka, uye wobva wawedzera kushandiswa.
Tevera modhi kudonha uye simbisa mushure mekuchinja kwekamera kana dataset.
Tevera modhi kudonha uye simbisa mushure mekuchinja kwekamera kana dataset. Bata nhanho yega yega segedhi rehumbowo: kana maitiro asina kusangana, imbomira kuburitsa, vhara gaka, uye wobva wawedzera kushandiswa.