Xool-Làkk-Jëf Royuwaayi Robotik
Modèlu Vision-Language-Action (VLA) ay reso neuronal yu mag lañu yuy jël nataali kamera boole ci ay tegtal yuñ bind ba noppi di génne ci saasi ndigalu motëru robot bi.
Résumé
They matter because they bring the broad common sense of foundation models to physical machines, letting one model control a robot across many tasks instead of hand-coding each behavior.
Plongeur bu xóot
Benn xeetu VLA dafay boole ñatti mbir: gis-gis (kadre kamera), làkk (mébet bu melni 'def koppu bi ci evier'), ak jëf (angle yu boole, ubbiku/tëjug gripper, wala gaawaayu efektër bu mujj). Google DeepMind's RT-2 nekkoon na lu am solo: jël na xeetu làkku gis-gis buñ tàggat ci nataali web ak mbind, ba noppi ñu boole ko ci yoon wi robo yi di jaar, suko defee benn reso bi mëna tontu 'ban meññeef la lii?' itam génne ay jëf yuñ màndargaale ni mbind. Royuwaay yu ubbeeku yu melni OpenVLA (parametre 7B) ak pi-0 bu Xarañteg Yaram topp ci. Ci anam wu gëna am solo, xeetu yii dañuy wane toxal 'emergence': xam-xam web (xam logo màrk, xam 'ki gëna ndaw') dafay yóbbu ci manipulation, suko defee robot bi dafay generalise ci mbir ak tegtal yu mu musul gis ci tàggat robot.
Gis-gis xarala
VLA yu bari dañuy tàqale jëf yu wéy yi ci ay token suko defee transformateur bi mëna leen wax luy waaja xew, ni kàddu yi. RT-2 dafay boole dimension jëf bu nekk ci benn ci 256 bin yi ba noppi bàyyi leen ñu nekk chaîne text. Design yu bees yu melni pi-0 dañuy takk boppu 'ekspert ci jëf' bu méngoo ak diffusion wala flow ci yaxu ndigg bu lakk wi ñuy gis, di defar ay piyeesu jëf yu fréquence yu kawe (lu melni, 50 Hz) ci barabu benn jéego bu wuute, gëna baaxal dexterity.
njeextalu pexe
Gaawaay ak yaatuwaay
Visual IA mën na otomatise saytu, gis ak etiketu liggéey ci eskaal.
Tabax tànneef
Ekipu kreatif yi mën nañu defar konsept yu gëna gaaw te duñu def lu bari ci loxo.
Ekip ak def liggéey
Liggéeyukaay yi mën nañu jëfandikoo siñaal nataal wala wideo yu jafewoon lool ci liggéey.
Ëlëgu gis-gis-làkk-jëf xeetu robotik
Xaarandi ay done yu gëna mag (Open X-Embodiment dafay boole ay done yu bawoo ci 22+ xeeti robot) suko defee benn model di dawal ay loxo, ay humanoid ak ay base mobile. Gëstu dafay puus ci inference bu gëna gaaw ngir doxal ci jamono dëgg, 3D bu gëna riis ak dugal tactile, ak chaine yu xalaat fu model bi di 'xalaat' balaa muy jëfandikoo. Luñu bëgga mooy benn politik generalist boo mëna laaj ci Àngle bu leer, ak seet ci saasi, lu mel ni waxtaan ak assistant.
Doxal ci àdduna dëgg
RT-2 dafay yoree ab robo waañ ngir 'toxal banaan bi ci nimero 3' di jëfandikoo ay sifar yu mu jàngee ci mbindu web bi, du ay demo robot
OpenVLA, xeetu 7B bu ubbeeku, lab yi defar ko bu baax ngir doxal tànneef ak-place ci kaw loxo yu yomb njëg
Xarañteg yaram pi-0 mooy fóotu yéere ak fomp taabal ci boole ay xam-xam yu bari ci benn njàngale
Benn loxo depot dafa wax 'tannal mbir mi gëna yomba dagg' nga xool ban mbir moo ci melokaanam
Risk yi ak balustrade yi
Yelleefi nataal ak nangu mën na nekk risku yoon sudee fi ñu bawoo leerul.
Performance model bi mën na wuute ci leeraay bi, demographie bi ak environmaa bi.
Njuumteg positive yi mën nañu dem te kenn duko seetlu fileek xool wuñu buntu wóolu sa bopp.
Roadmap ngir samp gi
Mandargal kritërium nangug njub, woowaat ak njëgu njuumte.
Saytu ak done yu méngoo ak anam yi ñuy liggéeyee dëgg.
Yokk jàngat nit ngir xam fu wóorul dara wala am njeexital yu rëy.
Toppal model drift bi nga baaxal ko ginaaw bi kamera bi wala done yi soppeekoo.
Weyal di banneexu
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Vision-Language-Action Models for Robotics quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gis bi ci topp
CLIP ak misaali làkku gis-gis
Laaj yi ñuy faral di laaj
What is Vision-Language-Action Models for Robotics?
Modèlu Vision-Language-Action (VLA) ay reso neuronal yu mag lañu yuy jël nataali kamera boole ci ay tegtal yuñ bind ba noppi di génne ci saasi ndigalu motëru robot bi. Dañu am solo ndax dañuy indi xam-xam bu yaatu ci modelu fondaasioŋ yi ci masin physique yi, bàyyi benn model mu yoree robot ci liggéey yu bari ci plaasu kode loxo bu nekk ci doxalin bu nekk.
Yan ñatti xeeti dugal/génne ñooy màndargaal xeetu gis-gis-làkk-jëf (VLA)?
VLA dafay jël gis-gis (nataal) boole ci mébetu làkk ba noppi génne ay jëf (komando motër), boole gis-gis, topp tegtal ak doxal.
Ak naka la RT-2 bu DeepMind mënee wane jëfi robot yi suko defee ab transformatër mëna leen defar?
RT-2 dafa boole dimension jëf bu nekk ci ay token yu wuute (lu melni, 256 bins) ba noppi génne leen ni string, bàyyi masin modelu làkk bi ñu wax luy am ci jëf yu melni kàddu.
Luy 'toxal buy jameet' ci wàllu VLAs?
Ndax VLAs yi dañu tàmbalee ci xeetu làkku gis-gis biñ tàggat ci web bi, xam-xam bu melni xàmmee logo yi wala xam 'ki gëna ndaw' dafay toxal ci liggéey yu ñuy jaay xel yuñ musul gis ci done robot.
Lan mooy njariñu pi-0 diffusion/flow-matching boppu jëf buñu ko méngale ak benn token jëf diskret?
Lu moy benn jéego bu wuute ci saa si, pi-0 dafay defar ay piyeesu jëf yuy wéy ci fréquence bu kawe, loolu mooy tax mouvement bi gëna nooy te gëna xarañ.
Lan moo waral done yu ubbeeku X-Embodiment am solo ci VLA yi?
Open X-Embodiment dafay boole ay trajectoire yu bawoo ci robot yu bari te wuute, di jàppale ci tàggat ay politik yuy toxal ci loxo yi, base mobile yi ak yeneen anam yu ñuy jëfandikoo.