Overview
DUSt3R inovakazve dense 3D geometry kubva kune mashoma emapikicha akajairwa asingade inozivikanwa kamera chinzvimbo kana calibration. Iyo inodonhedza iyo yechinyakare yakawanda-nhanho photogrammetry pombi kuita imwechete neural network inongoburitsa 3D mapoinzi.
DUSt3R Dense 3D Kuvakazve ndeyekombuta-chiratidzo chekufambisa chinodudzira kana kuburitsa midhiya yekuona yekuongorora, mashandiro, uye kugadzira.
Deep Dive
Classic 3D kuvakazve (chimiro-kubva-mukufamba pamwe neakawanda-maonero stereo) iketani isina kusimba: tarisa maficha, fananidza iwo, fungidzira kamera inomira, katatu, wobva wawedzera. Imwe neimwe nhanho inogona kutadza, uye iwe kazhinji unoda akawanda anopindirana mifananidzo uye inozivikanwa kamera intrinsics. DUSt3R (Wang et al., 2024) inogadzirisa dambudziko rese. Kungopihwa mifananidzo miviri chete, inoshandura-yakavakirwa network inodzoreredza zvakananga 'pointmap' kune yega yega - yakaomesesa per-pixel 3D kurongeka, ese ari maviri anoratidzwa mune imwecheteyo kurongeka furemu. Kubva pane iwo akarongedzerwa mapoinzi iwe unogona kuverenga kubva kudzika, kamera inomira, uye machisi anenge emahara. Kune inopfuura mifananidzo miviri, DUSt3R inoita kurongeka kwepasirese iyo inosona ese epaviri mapoinzi mapoinzi mune imwechete inowirirana point gore. Inoshanda kunyangwe nemakamera asina kuverengerwa uye mashoma, maonero akapatsanurwa.
Technical Insight
Mhedzisiro inobuda ndiyo mepu: yakakora 2D-ku-3D mepu inoisa pixel yega yega yemufananidzo panzvimbo yakajeka ye3D, iine mifananidzo miviri yevaviri yakadzoserwa mune yekutanga kamera yekuronga furemu. Nekuti manyorero ari pachena mune zvakagovaniswa 3D makongisheni, fungidziro yekumisa uye kuenzanirana inove yezasi kuverenga kwete zvinodikanwa. A Vision Transformer ine muchinjiko-kutarisisa pakati pematavi maviri emifananidzo inoita kuti network ikonzerese zvakabatana pamusoro pemaonero ese, kudzidza geometry zvakananga kubva kuhombe dataset yemifananidzo yakaiswa.
Mastering DUSt3R Dense 3D Kuvakazve
DUSt3R inovakazve dense 3D geometry kubva kune mashoma emapikicha akajairwa asingade inozivikanwa kamera chinzvimbo kana calibration. Iyo inodonhedza iyo yechinyakare yakawanda-nhanho photogrammetry pombi kuita imwechete neural network inongoburitsa 3D mapoinzi. DUSt3R Dense 3D Kuvakazve ndeyekombuta-chiratidzo chekufambisa chinodudzira kana kuburitsa midhiya yekuona yekuongorora, mashandiro, uye kugadzira. Kuti uvake kunzwisisa kwakadzama, bata DUSt3R Dense 3D Reconstruction semuenzaniso wekushandisa, kwete chinhu chimwe chete: tsanangura zvinodiwa, kujekesa fungidziro, uye patsanura izvo system inogona kuita nekuvimbika kubva kune ichiri kuda kutonga kwenyanzvi.
Mukuita, zvikwata zvakasimba zvinoshandisa DUSt3R Dense 3D Kuvakazve chiyero chechokwadi nehuchokwadi hwekushanda semhando yedata, kusiyana kwemwenje, uye kuenderana kwemazita. Ivo vanonyora zvakajeka maitiro ebudiriro, bvunzo vachipokana ne data rechokwadi uye mafambiro ebasa, uye iterate zvichibva pane zvakacherechedzwa maitiro ekutadza kwete kuhwina-nguva imwe chete yebhenji. Apa ndipo apo kunzwisisa kwe theoretical kunoshanduka kuve kugona kwakasimba pane chigadzirwa, mutemo, uye mashandiro.
Visual AI inogona kuita otomatiki yekuongorora, yekuona, uye yekumaka mabasa pachiyero. Panguva imwecheteyo, kodzero dzeMufananidzo uye kubvumirwa kunogona kuve njodzi dzepamutemo kana hunhu husina kujeka. Nzira yakatsiga ndeyekubatanidza kukurumidza kuyedza nekutonga: mhanyisa vatyairi vendege, tora humbowo, buritsa matanda esarudzo, uye urambe uchivandudza chengetedzo semaitiro emuenzaniso, zvinotarisirwa nemushandisi, uye zvinodikanwa zvekutonga.
Strategic Impact
Visual AI inogona kuita otomatiki yekuongorora, yekuona, uye yekumaka mabasa pachiyero.
Visual AI inogona kuita otomatiki yekuongorora, yekuona, uye yekumaka mabasa pachiyero. Mukutumirwa kwemhando yepamusoro, izvi zvinoshandurirwa kuita mitemo inoyerwa yekushanda, miganhu yevaridzi, uye tsika dzekudzokorora dzinodzokororwa kuitira kuti zvikwata zvikwire kuvimba pane kukwidza kusajeka.
Zvikwata zvekugadzira zvinogona prototype pfungwa nekukurumidza nekudzokororwa kwemaoko mashoma.
Zvikwata zvekugadzira zvinogona prototype pfungwa nekukurumidza nekudzokororwa kwemaoko mashoma. Mukutumirwa kwemhando yepamusoro, izvi zvinoshandurirwa kuita mitemo inoyerwa yekushanda, miganhu yevaridzi, uye tsika dzekudzokorora dzinodzokororwa kuitira kuti zvikwata zvikwire kuvimba pane kukwidza kusajeka.
Mashandisirwo anogona kushandisa masaini emifananidzo nemavhidhiyo ayo aimbove akaoma kugadzirisa.
Mashandisirwo anogona kushandisa masaini emifananidzo nemavhidhiyo ayo aimbove akaoma kugadzirisa. Mukutumirwa kwemhando yepamusoro, izvi zvinoshandurirwa kuita mitemo inoyerwa yekushanda, miganhu yevaridzi, uye tsika dzekudzokorora dzinodzokororwa kuitira kuti zvikwata zvikwire kuvimba pane kukwidza kusajeka.
Real-World Implementation
Kushandura mashoma mafoto efoni emukamuri kana chinhu kuti ive inoshandisika 3D point gore pasina kuongorora nzvimbo dzekamera.
Kudzoreredza kamera inomira uye kudzika kune bootstrap yakadzika 3D kuvakazve kana Gaussian splatting kubva mashoma, asina kuverengerwa mifananidzo.
Kugadzira patsva mapikicha kubva mudura kana mapikicha epainternet uko data recalibration isingawanikwe.
Kupa nekukurumidza geometry fungidziro yemarobhoti uye AR yekufamba kubva pamaonero maviri kana matatu chete.
Maitiro Ekuita
DUST3R Dense 3D Kuvakazve mukuita
Kushandura mashoma mafoto efoni emukamuri kana chinhu kuti ive inoshandisika 3D point gore pasina kuongorora nzvimbo dzekamera.
Kushandura mashoma akajairwa mafoto efoni emukamuri kana chinhu kuita inoshandisika 3D point gore pasina kuongorora nzvimbo dzekamera Matimu anowanzo kuwana mhedzisiro iri nani kana achinge atsanangura emhando yepamusoro kumberi, chengetedza nzira yekukwira kwevanhu yemakesi emupendero, uye kuteedzera zvese zvakawanikwa zvechigadzirwa uye kukanganisa mutengo nekufamba kwenguva.
DUST3R Dense 3D Kuvakazve mukuita
Kudzoreredza kamera inomira uye kudzika kune bootstrap yakadzika 3D kuvakazve kana Gaussian splatting kubva mashoma, asina kuverengerwa mifananidzo.
Kudzoreredza kamera inomira uye kudzika kune bootstrap yakadzika 3D kuvakazve kana Gaussian splatting kubva shoma, isina kurongeka mifananidzo Matimu anowanzo kuwana mhedzisiro iri nani paanotsanangura emhando zvikumbaridzo kumberi, chengetedza nzira yekukwira kwevanhu yemakesi ekumucheto, uye kuteedzera zvese zvakawanikwa zvechigadzirwa uye kukanganisa mutengo nekufamba kwenguva.
DUST3R Dense 3D Kuvakazve mukuita
Kugadzira patsva mapikicha kubva mudura kana mapikicha epainternet uko data recalibration isingawanikwe.
Kugadzira patsva zviratidziro kubva mudura kana mapikicha einternet uko kamera yecalibration data isingawanikwe Matimu anowanzo kuwana mhedzisiro iri nani kana achinge atsanangura emhando yepamusoro kumberi, chengetedza nzira yekukwira kwevanhu yemakesi ekumucheto, uye kuteedzera zvese zvakawanikwa zvechigadzirwa nemitengo yekukanganisa nekufamba kwenguva.
DUST3R Dense 3D Kuvakazve mukuita
Kupa nekukurumidza geometry fungidziro yemarobhoti uye AR yekufamba kubva pamaonero maviri kana matatu chete.
Kupa kukurumidza geometry fungidziro yemarobhoti uye AR yekufamba kubva pamaonero maviri kana matatu Matimu anowanzo kuwana mhedzisiro iri nani paanotsanangura emhando yepamusoro kumberi, chengetedza nzira yekukwira kwevanhu yemakesi emupendero, uye kuteedzera zvese zvakawanikwa zvechigadzirwa uye mutengo wekukanganisa nekufamba kwenguva.
Njodzi & Guardrails
Kodzero dzemifananidzo uye kubvumirwa kunogona kuve njodzi dzepamutemo kana provenance isina kujeka.
Kuita kwemuenzaniso kunogona kusiyanisa kupenya, huwandu hwevanhu, uye nharaunda.
Manyepo enhema anogona kusacherechedzwa kunze kwekunge zvikumbaridzo zvekuvimba zvikatariswa.
Implementation Roadmap
Tsanangura maitiro ekugamuchirwa echokwadi, kurangarira, uye mutengo wekukanganisa.
Tsanangura maitiro ekugamuchirwa echokwadi, kurangarira, uye mutengo wekukanganisa. Bata nhanho yega yega segedhi rehumbowo: kana maitiro asina kusangana, imbomira kuburitsa, vhara gaka, uye wobva wawedzera kushandiswa.
Edzai nedata rinoenderana nemamiriro chaiwo ekugadzira.
Edzai nedata rinoenderana nemamiriro chaiwo ekugadzira. Bata nhanho yega yega segedhi rehumbowo: kana maitiro asina kusangana, imbomira kuburitsa, vhara gaka, uye wobva wawedzera kushandiswa.
Wedzera ongororo yemunhu kune yakaderera-kusavimbika kana yakakwirira-inokanganisa kufanotaura.
Wedzera ongororo yemunhu kune yakaderera-kusavimbika kana yakakwirira-inokanganisa kufanotaura. Bata nhanho yega yega segedhi rehumbowo: kana maitiro asina kusangana, imbomira kuburitsa, vhara gaka, uye wobva wawedzera kushandiswa.
Tevera modhi kudonha uye simbisa mushure mekuchinja kwekamera kana dataset.
Tevera modhi kudonha uye simbisa mushure mekuchinja kwekamera kana dataset. Bata nhanho yega yega segedhi rehumbowo: kana maitiro asina kusangana, imbomira kuburitsa, vhara gaka, uye wobva wawedzera kushandiswa.