Kwenzekeni
Abacwaningi bahlongoze i-AffectOmni, uhlaka lokuqeqesha amamodeli ezilimi ezinkulu ze-multimodal ukubonisana ngemizwa, izinhloso kanye nezinye izimpawu ezithintekayo ezithombeni noma ezigcawini ezihlobene nomphakathi nezobuciko. I-preprint ithi amamodeli kwesinye isikhathi angafinyelela izimpendulo ezilungile ngokuthembela kumongo ozungezile ngenkathi enganaki ubufakazi obugxile kubantu obungenza ukucabanga kwabo kube lula ukububheka.
Ukuphrinta kusengaphambili okuhanjiswe ku-arXiv ngo-Aug. 24 kuchaza i-AffectOmni, uhlaka lokufunda oluqinisayo lokubonisana okuthintekayo okuqinisekiswayo kumamodeli ezilimi ezinkulu. Leli phepha ligxile ezigcawini eziphathelene nomphakathi nezobuciko lapho isistimu ingase idinge ukucabangela imizwa, inhloso noma ukuhleleka kwemicimbi. Ababhali bahlonza ubuthakathaka obuthile ezinhlelweni ezikhona: imodeli inganikeza impendulo efanele kuyilapho isebenzisa izinqamuleli ezisuselwe kumongo wesigcawu esibanzi esikhundleni sokubheka izimpawu ezigxile kubantu ezifana nezinkulumo ezincane nolimi lomzimba.
Uhlaka lwengeza izingxenye ezimbili zomklomelo ezibizwa ngokuthi i-People Focus kanye ne-Temporal Order. Ngokusho komthombo, i-People Focus ikhuthaza imodeli ukuthi ikhethe ubufakazi obuhlanganisa abantu, kanti i-Temporal Order ikhuthaza ukucabanga okuhlelekile lapho kwenzeka imicimbi. Leli phepha lithi lezi zimpawu zihloselwe ukunciphisa ukuziphatha kwezinqamuleli futhi kuthuthukiswe ukuxhumana phakathi kwempendulo yemodeli kanye nobufakazi obubonakalayo obusekelayo. Umthombo awunikezi imininingwane eyanele ukuze uthole ukuthi le miklomelo iziphatha kanjani kuzo zonke izinhlobo zesigcawu noma ingabe ivimbela isinqamuleli ekuhlolweni okuzimele.
I-AffectOmni iphinde isebenzise amaphuzu wokuqhathanisa angaphakathi kweqembu ngesikhathi sokufunda kokuqinisa. Ababhali bathi lokhu kuhloselwe ukubhekana nokuhlanganisa amaphuzu ekwahluleleni kwemodeli yolimi olukhulu, lapho izimpendulo zamakhandidethi eziningi zithola amaphuzu afanayo futhi ngenxa yalokho zikhiqiza izimpawu zokuqeqesha ezibandlululayo. Ngemuva kokuthi imodeli ikhiqize isisekelo sefomu lamahhala, Isifinyezo Esicabangayo siguqula leso sizathu sibe imiyalo yobufakazi obusebenzisekayo. Leyo miyalelo ibe isigxiliswa ezifundeni zobufakazi obusezingeni le-pixel kusetshenziswa i-SAM3, okwenza lokho ababhali bakuchaza njengesixhumi esibonakalayo esihlolekayo ngaphandle kweluphu yokuqeqeshwa.
Umthombo ubika izivivinyo kumabhentshimakhi amathathu: I-IntentBench, i-Daily Omni ne-WorldSense. Ngokuphathelene namamodeli omthombo ovulekile esikalini se-7B, ababhali babika intuthuko engaguquki, okuhlanganisa inzuzo engu-4.66% ekuqashelweni kwemizwelo kanye nenzuzo engu-14.29% emisebenzini ebucayi yesikhashana. Umthombo awucacisi ukuthi lezi zibalo ziyizinzuzo zamaphoyinti aphelele noma ukuthuthukiswa kwephesenti elihlobene, futhi awunikezi ukubalwa kwesampula, izikhawu zokuzithemba, imigoqo yamaphutha noma ukuqhathanisa namasistimu obunikazi aqine kakhulu. I-preprint ithi ikhodi iyatholakala, kodwa umthombo awunikezi isixhumanisi sekhosombe esisebenzisekayo noma uchaze imigomo yelayisensi.
Imininingwane yomthombo: arxiv.org ↗
Kungani kubalulekile
Ukuqashelwa komthelela kuyikhono elilandelanayo lamasistimu ahumusha ukusebenzelana komphakathi, kodwa ilebula elungile ayibonisi ukuthi imodeli ibone umuntu ofanele, inkulumo noma ukuthinta. Indlela ye-AffectOmni ibhekana naleyo nkinga yokulandeleka ngokumatanisa ukucabanga okuyimodeli nezifunda zobufakazi ezingahlolwa ngaphandle kwenqubo yokuqeqesha. Uma izinzuzo ezibikiwe zibambelela ngaphezu kwamabhentshimakhi ababhali, indlela ingase inikeze indlela yokuziphendulela yokuhlola amasistimu e-multimodal ahumusha ukuziphatha komuntu.
Udaba olusebenzayo akukhona nje ukuthi imodeli ye-multimodal ingasho umzwelo. Kuzinhlelo zokusebenza ezihumusha izigcawu zomphakathi, abasebenzisi bangadinga ukwazi ukuthi ibuphi ubufakazi obubonakalayo obuholele ekwahluleleni. Isistimu efaka ilebula yesehlakalo ngendlela efanele ngesizathu esingalungile ingahluleka uma ingemuva, okokugqoka, isilungiselelo noma ezinye izimpawu zomongo zishintsha. Ukugcizelela kwephepha ebufakazini obugxile kubantu bubhekana nalelo gebe lokwethembeka ngokuqondile, nakuba umthombo ubika incazelo yababhali kunokuthola okuqinisekisiwe okuzimele.
Isinyathelo sokusekela ubufakazi singenza imiphumela yemodeli ethintekayo ibe lula ukuyicwaninga. Ngokuhumusha isizathu sibe yimiyalelo futhi siyihlobanise nezifunda ezisezingeni lamaphikseli, i-AffectOmni inikeza i-artifact ekhonkolo umhloli angayihlola. Lokhu kusebenza kakhulu kunesicelo esijwayelekile sokuthi imodeli izichaze yona ngokwayo: isimangalo siboshelwe ezindaweni ezisesithombeni esifakiwe. Noma kunjalo, izifunda ezigqanyisiwe akufanele zithathwe ngokuzenzakalelayo njengobufakazi bokucabanga okuyimbangela. Umthombo awuqinisekisi ukuthi izifunda zembula ngokwethembeka inqubo yangaphakathi ekhiqize impendulo.
Ukuthuthukiswa okubikiwe kungase kube usizo ngenxa yokuthi ukuqonda kwesikhashana nokuhumusha ngokomzwelo kungamaphuzu avamile okuhluleka kumasistimu we-multimodal. Ukuthuthukiswa okungu-14.29% emisebenzini ebucayi yesikhashana, uma kukhiqizwe kabusha ngokuzimele futhi kuchazwe ngokucacile, kungabaluleka kumasistimu ahlaziya ukulandelana kunezithombe ezimile ezingazodwa. Umthombo awusho ukuthi inzima kangakanani imisebenzi, ukuthi amabhentshimakhi akhiwe kanjani noma ukuthi izinzuzo zihumushela ekusetshenzisweni okulandelanayo okufana nemfundo, ukufinyeleleka, ukulinganisela noma ukusebenzisana kwekhompuyutha yomuntu.
Umsebenzi futhi ubonisa ukukhetha okubanzi komklamo ekuhlolweni kwe-AI: ukuvuza hhayi kuphela okuphumayo kodwa ukukhethwa nokuhlelwa kobufakazi obusekelayo. Leyo ndlela ingasiza abacwaningi bahlukanise isisekelo sangempela sokubuka ezimpendulweni ezikhiqizwa ngokuhlangana kwedathasethi. Kodwa-ke, izinqumo ezithintekayo zizwela ngokwemvelo kumongo nokuhumusha. Indlela evuza ngokugxila ezindleleni zomuntu ezibonakalayo isengafaka ukuqagela mayelana nokuvezwa kwemizwelo, izinkambiso zomphakathi noma incazelo yokuthinta, ikakhulukazi uma lokho kuqagela kubonakala kudatha yokuqeqeshwa.
I-Interactive Mechanism: Indlela Esebenza Ngayo Ngempela
Hlola ubuchwepheshe obuyisisekelo ngemuva kwalokhu kuthuthukiswa ngokuhlanganyela.
A route planner searches possible journeys using explicit rules. What does this illustrate about AI?
Ongakubuka ngokulandelayo
Imiphumela iphuma ekuphrintiwe ngaphambili okunamakhasi ayi-12 arXiv futhi ibikwa ngababhali bayo; umthombo awusunguli ukuphindaphinda okuzimele, ukusetshenziswa komhlaba wangempela noma ukusebenza phakathi kwabantu nezilungiselelo ezibanzi. Umsebenzi wokulandelela kufanele uhlole ukuthi uhlaka luhlala luthembekile yini namasiko angajwayelekile, ukusebenzelana okungacacile, ikhwalithi yesithombe ephansi nezigcawu lapho izimpawu zemizwa zicashile noma ziphikisana. Kuzobaluleka futhi ukunquma ukuthi izifunda zobufakazi ziyibonisa ngempela yini inqubo yesinqumo semodeli noma zihlinzeka ngezindawo ezibambekayo zokusekela ngemva kokwakhiwa kwempendulo.
Umbuzo wokuqala ukuphindaphinda. I-AffectOmni yethulwa njengephrinta ye-arXiv, hhayi njengomphumela obuyekezwe ngontanga noma oqinisekiswe ngokuzimele. Abacwaningi kufanele bahlole ukuthi ingabe izinzuzo ezibikiwe ziyaqhubeka yini ngaphansi kwephrothokholi yokuhlola efanayo, kungakhathaliseki ukuthi zihlala ngemva kokulawula idatha yokuqeqeshwa eyengeziwe noma usayizi wemodeli, nokuthi ingabe indlela ithuthukisa ukulinganisa nokutholwa kwamaphutha esikhundleni sezikolo zokubonisa kuphela.
Ububanzi bebhentshimakhi buzobalulekile. Amagama omthombo i-IntentBench, i-Daily Omni ne-WorldSense kodwa ayichazi ukutholakala kwezibalo zabantu, izilimi, ikhwalithi yesithombe, izilungiselelo zamasiko noma ibhalansi yezimo zomphakathi. Ukuhlola okuzayo kufanele kuhlole imizwa engacacile, izimpawu ezixubile, ubuso obuvalekile, ukuma komzimba okungajwayelekile nezigcawu lapho umuntu ogqame kakhulu engeyena umthombo wobufakazi obufanele. Kufanele futhi babike imiphumela yeqembu elincane lapho ukuhumusha kungahluka kuwo wonke amasiko noma ukukhubazeka.
Isimangalo sokucwaningwa kwamabhuku sifanele ukuhlolwa okuqondiwe. Umhloli angase aqhathanise izifunda ezigqanyisiwe nezichasiselo zomuntu, aphazamise izifunda ezikhethiwe, azifihle, noma ashintshe umongo wangemuva kuyilapho ebambe ubufakazi obugxile kubantu ngokungaguquki. Ukuhlola okunjalo kungasiza ekunqumeni ukuthi izifunda zobufakazi ziyadingeka yini esinqumweni semodeli noma zenziwa ngemuva kweqiniso. Umthombo awukubiki lokhu kuhlolwa, ngakho-ke izinga lokuqinisekiswa okumangalelwe lihlala lingaziwa.
Okokugcina, imingcele esebenzayo ayikamiswa. Alukho ulwazi emthonjeni mayelana nokubambezeleka, izindleko zokubala, ukulayisensa, ukutholakala kokuthunyelwa, izivikelo zobumfihlo noma ukusetshenziswa kumasistimu abukhoma. Ababhali babika imiphumela ngokumelene nezisekelo zesikali se-7B somthombo ovulekile, kodwa umthombo awubonisi ukuthi i-AffectOmni iyaqhudelana namamodeli amakhulu noma iqinile ngaphandle kwamabhentshimakhi amathathu aqanjiwe. Kuze kube yilapho leyo mibuzo iphendulwa, umsebenzi uqondwa kangcono njengesiphakamiso socwaningo esinemiphumela ebikiwe ethembisayo, hhayi njengesixazululo esiqinisekisiwe sokuhumusha imizwelo yabantu.