Uhlolojikelele
I-Diffusion Policy isebenzisa umqondo ofanayo wokwenza umsindo ngemuva kwezijeneretha zesithombe ezifana ne-Stable Diffusion ekulawuleni irobhothi: esikhundleni sokubikezela isenzo esisodwa esilandelayo, ikhiqiza ukulandelana okufushane kwezenzo zesikhathi esizayo ngokulungisa umsindo ngokuphindaphindiwe. Ibalulekile ngoba iphatha imvelo engcolile, enezimo eziningi zokukhohlisa kwangempela kangcono kakhulu kunezindlela ezindala.
Inqubomgomo Yokusabalalisa Yokulawula Irobhothi ingeyokugeleza komsebenzi okubona ngekhompyutha okuhumusha noma okukhiqiza imidiya ebonakalayo ukuze ihlaziywe, isebenze, nokudala.
I-Deep Dive
Yethulwa ngo-2023 ngabacwaningi base-Columbia, MIT, kanye ne-Toyota Research Institute, Inqubomgomo Yokusabalalisa ihlela kabusha ukufunda kwe-visuomotor njengokuqeda umsindo okunemibandela. Uma kubhekwa izithombe zekhamera zakamuva kanye nesimo serobhothi, iqala emsindweni ongahleliwe futhi iqhube izinyathelo ezimbalwa zokuhlukanisa umsindo ukuze kukhiqizwe 'isiqephu sesenzo' - isho izinyathelo zesikhathi ezilandelayo eziyi-8 kuya kwezingu-16 zokuma komphumela. Ukuwina okukhulu i-multimodality: uma umsebenzi unezixazululo ezimbalwa ezivumelekile (ungabamba inkomishi ukusuka kwesokunxele noma kwesokudla), ukwehla okujwayelekile kubenza babe isenzo esibi esimaphakathi, kuyilapho imodeli yokusabalalisa ingazibophezela ngokuhlanzekile kumodi eyodwa. Iphinde ifunde ngokuzinza emibonisweni yabantu (ukuhlanganisa ukuziphatha) futhi ibhekane kahle nezikhala zesenzo ezinobukhulu obuphezulu, ikwenze kube ukukhetha okuzenzakalelayo ezinhlelweni eziningi zesimanje zokukhohlisa.
I-Technical Insight
Ukuqeqeshwa kwengeza umsindo we-Gaussian ekuboniseni ukulandelana kwesenzo futhi kufundisa inethiwekhi (ngokuvamile i-U-Net noma i-transformer) ukubikezela lowo msindo, obekwe esimweni sokubhekwa okubukwayo nokubalulekile. Ngesikhathi sokusebenza izwakala isuka kumasampuli angahleliwe phezu kwezinyathelo ezimbalwa (DDPM/DDIM) ukuze iveze umzila wesenzo. Ukubikezela izingcezu kanye nokuhlela kabusha 'kwe-receding-horizon' kunikeza ukuvumelana kwesikhashana ngenkathi kuhlala kuxhumene nokuphawuliwe okusha.
I-Mastering Diffusion Policy for Robot Control
I-Diffusion Policy isebenzisa umqondo ofanayo wokwenza umsindo ngemuva kwezijeneretha zesithombe ezifana ne-Stable Diffusion ekulawuleni irobhothi: esikhundleni sokubikezela isenzo esisodwa esilandelayo, ikhiqiza ukulandelana okufushane kwezenzo zesikhathi esizayo ngokulungisa umsindo ngokuphindaphindiwe. Ibalulekile ngoba iphatha imvelo engcolile, enezimo eziningi zokukhohlisa kwangempela kangcono kakhulu kunezindlela ezindala. Inqubomgomo Yokusabalalisa Yokulawula Irobhothi ingeyokugeleza komsebenzi okubona ngekhompyutha okuhumusha noma okukhiqiza imidiya ebonakalayo ukuze ihlaziywe, isebenze, nokudala. Ukuze wakhe ukuqonda okujulile, phatha Inqubomgomo Yokusabalalisa Yokulawula Irobhothi njengemodeli yokusebenza, hhayi isici esisodwa: chaza imiphumela efiselekayo, ucacise ukucabanga, futhi uhlukanise lokho isistimu engakwenza ngokwethembeka kulokho okusadinga ukwahlulela kochwepheshe.
Empeleni, amaqembu aqinile asebenzisa Inqubomgomo Yokusabalalisa yokulawula ibhalansi yerobhothi namaqiniso okusebenza njengekhwalithi yedatha, ukuhluka kokukhanya, nokuvumelana kwamalebula. Babhala imibandela yempumelelo ecacile, ukuhlola okuqhathaniswa nedatha engokoqobo nokugeleza komsebenzi, futhi baphindaphinde ngokusekelwe kumaphethini okuhluleka aqashiwe esikhundleni sokuwina kwebhentshimakhi yesikhathi esisodwa. Yilapho ukuqonda kwethiyori kuguquka kube amandla ahlala njalo kuwo wonke umkhiqizo, inqubomgomo, kanye nokusebenza.
I-Visual AI ingakwazi ukuhlola, ukutholwa, nokumaka imisebenzi esikalini. Ngesikhathi esifanayo, amalungelo ezithombe kanye nemvume kungaba ubungozi bomthetho uma ukutholakala kungacacile. Indlela eqine kakhulu iwukuhlanganisa isivinini sokuhlola nesiyalo sokuphatha: qhuba abashayeli bezindiza, bamba ubufakazi, ushicilele amalogi ezinqumo, futhi ubuyekeze izivikelo ngokuqhubekayo njengoba imodeli yokuziphatha, okulindelwe ngabasebenzisi, kanye nezimfuneko zokulawula zishintsha.
I-Strategic Impact
I-Visual AI ingakwazi ukuhlola, ukutholwa, nokumaka imisebenzi esikalini.
I-Visual AI ingakwazi ukuhlola, ukutholwa, nokumaka imisebenzi esikalini. Ekusetshenzisweni kwekhwalithi ephezulu, lokhu kuhunyushwa emithethweni yokusebenza elinganisekayo, imingcele yobunikazi, nemikhuba yokubuyekeza ephindelelayo ukuze amaqembu akwazi ukukala ukuzethemba esikhundleni sokukala ukungaqondakali.
Amathimba aqanjiwe angakwazi ukulinganisa imiqondo ngokushesha ngezibuyekezo ezimbalwa ezenziwa mathupha.
Amathimba aqanjiwe angakwazi ukulinganisa imiqondo ngokushesha ngezibuyekezo ezimbalwa ezenziwa mathupha. Ekusetshenzisweni kwekhwalithi ephezulu, lokhu kuhunyushwa emithethweni yokusebenza elinganisekayo, imingcele yobunikazi, nemikhuba yokubuyekeza ephindelelayo ukuze amaqembu akwazi ukukala ukuzethemba esikhundleni sokukala ukungaqondakali.
Imisebenzi ingasebenzisa amasiginali wesithombe nawevidiyo obekunzima ukuwenza ngaphambilini.
Imisebenzi ingasebenzisa amasiginali wesithombe nawevidiyo obekunzima ukuwenza ngaphambilini. Ekusetshenzisweni kwekhwalithi ephezulu, lokhu kuhunyushwa emithethweni yokusebenza elinganisekayo, imingcele yobunikazi, nemikhuba yokubuyekeza ephindelelayo ukuze amaqembu akwazi ukukala ukuzethemba esikhundleni sokukala ukungaqondakali.
Ukuqaliswa Komhlaba Wangempela
Ingalo yerobhothi ephusha ibhulokhi emise okuka-T endaweni ephokophelwe, ibhentshimakhi lapho Inqubomgomo Yokusabalalisa isebenze ngokuphawulekayo ngaphezu kwezindlela zangaphambili zokuhlanganisa ukuziphatha
Amarobhothi e-Bimanual afunda imisebenzi yasekhishini ethambile njengokuphenyisisa ukudla noma ukuhlanganisa izingxenye ezivela kumademo okusebenza kwabantu
Ukukhetha imigqomo eyimfuhlumfuhlu lapho ukubamba okuningi okuvumelekile kukhona futhi inqubomgomo izibophezela kokukodwa esikhundleni sokulinganisa
Imojula ye-Action-head ngaphakathi kwezinhlelo zesenzo solimi lombono ekhiqiza ukunyakaza okushelelayo kwe-high-frequency yezandla ezihlakaniphile
Amaphethini Okusebenzisa
Inqubomgomo Yokusabalalisa Yokulawulwa Kwerobhothi isebenza
Ingalo yerobhothi ephusha ibhulokhi emise okuka-T endaweni ephokophelwe, ibhentshimakhi lapho Inqubomgomo Yokusabalalisa isebenze ngokuphawulekayo izindlela zangaphambilini zokuhlanganisa ukuziphatha.
Ingalo yerobhothi iphusha ibhulokhi emise okuka-T endaweni okuphokophelwe kuyo, ibhentshimakhi lapho Inqubomgomo Yokusabalalisa isebenze ngokuphawulekayo ngaphezu kwezindlela zangaphambili zokuhlanganisa ukuziphatha Amathimba ngokuvamile athola imiphumela engcono uma echaza imingcele yekhwalithi ngaphambili, agcina indlela yokukhuphuka yomuntu yamakesi asemaphethelweni, futhi alandelele kokubili izinzuzo zokukhiqiza nezindleko zamaphutha ngokuhamba kwesikhathi.
Inqubomgomo Yokusabalalisa Yokulawulwa Kwerobhothi isebenza
Amarobhothi asebenzisa i-Bimanual afunda imisebenzi yasekhishini ethambile njengokuphenyisisa ukudla noma ukuhlanganisa izingxenye ezivela kumademo okusebenza ngocingo lwabantu.
Amarobhothi asebenzisa i-Bimanual afunda imisebenzi yasekhishini ebucayi njengokuphenyisisa ukudla noma ukuhlanganisa izingxenye ezivela kumademo okusebenza ngocingo lwabantu Amathimba ngokuvamile athola imiphumela engcono lapho echaza ikhwalithi ephezulu ngaphambili, egcina indlela yokukhuphuka yomuntu yamakesi asemaphethelweni, futhi alandelele kokubili izinzuzo zokukhiqiza nezindleko zamaphutha ngokuhamba kwesikhathi.
Inqubomgomo Yokusabalalisa Yokulawulwa Kwerobhothi isebenza
Ukukhetha imigqomo eyimfuhlumfuhlu lapho ukubamba okuningi okuvumelekile kukhona futhi inqubomgomo izibophezela kokukodwa esikhundleni sokulinganisa.
Ukukhetha imigqomo eyimfuhlumfuhlu lapho ukubamba okuningi okuvumelekile okukhona futhi inqubomgomo izibophezela kokukodwa esikhundleni sokulinganisa Amathimba ngokuvamile athola imiphumela engcono uma echaza izilinganiso zekhwalithi ngaphambili, egcina indlela yokukhuphuka yomuntu yamakesi asemaphethelweni, futhi elandelela kokubili izinzuzo zokukhiqiza nezindleko zamaphutha ngokuhamba kwesikhathi.
Inqubomgomo Yokusabalalisa Yokulawulwa Kwerobhothi isebenza
Imojuli ye-Action-head ngaphakathi kwesistimu yesenzo solimi-ulimi ekhiqiza ukunyakaza okushelelayo kwemvamisa ephezulu kwezandla ezinobuhlakani.
Imojula ye-Action-head ngaphakathi kwezinhlelo zesenzo solimi lombono ekhiqiza ukunyakaza okushelelayo kwemvamisa ephezulu kwezandla ezinobuqili Amaqembu ngokuvamile athola imiphumela engcono lapho echaza imingcele yekhwalithi ngaphambili, egcina indlela yokukhuphuka komuntu yamakesi asemaphethelweni, futhi alandelele kokubili izinzuzo zokukhiqiza nezindleko zamaphutha ngokuhamba kwesikhathi.
Izingozi & Guardrails
Amalungelo ezithombe kanye nemvume kungaba ubungozi bezomthetho uma ukuvela kungacacile.
Ukusebenza kwemodeli kungahluka kukho konke ukukhanya, izibalo zabantu, kanye nezindawo.
Okuhle okungelona iqiniso kungase kungabonakali ngaphandle uma izinga lokuzethemba liqashelwa.
Ukuqalisa Umhlahlandlela
Chaza indlela yokwamukela yokunemba, ukukhumbula, nezindleko zamaphutha.
Chaza indlela yokwamukela yokunemba, ukukhumbula, nezindleko zamaphutha. Phatha isinyathelo ngasinye njengesango lobufakazi: uma imibandela ingafinyelelwa, misa ukukhishwa, vala igebe, bese unweba ukusetshenziswa.
Hlola ngedatha efana nezimo zangempela zokukhiqiza.
Hlola ngedatha efana nezimo zangempela zokukhiqiza. Phatha isinyathelo ngasinye njengesango lobufakazi: uma imibandela ingafinyelelwa, misa ukukhishwa, vala igebe, bese unweba ukusetshenziswa.
Engeza isibuyekezo somuntu ukuze uthole ukuzethemba okuphansi noma izibikezelo zomthelela omkhulu.
Engeza isibuyekezo somuntu ukuze uthole ukuzethemba okuphansi noma izibikezelo zomthelela omkhulu. Phatha isinyathelo ngasinye njengesango lobufakazi: uma imibandela ingafinyelelwa, misa ukukhishwa, vala igebe, bese unweba ukusetshenziswa.
Landelela ukukhukhuleka kwemodeli bese uqinisekisa kabusha ngemva kwezinguquko zekhamera noma zesethi yedatha.
Landelela ukukhukhuleka kwemodeli bese uqinisekisa kabusha ngemva kwezinguquko zekhamera noma zesethi yedatha. Phatha isinyathelo ngasinye njengesango lobufakazi: uma imibandela ingafinyelelwa, misa ukukhishwa, vala igebe, bese unweba ukusetshenziswa.