Kwenzekeni
Abacwaningi bahlongoza i-Segment-to-Video Supervision, noma i-S2V, yezinhlelo ze-AI ze-multimodal eziphendula imibuzo mayelana namavidiyo amade. Indlela idala izibonelo zemibuzo nezimpendulo kusukela kumasegimenti amafushane, evidiyo enziwe ngendawo, bese isebenzisa lezo zibonelo ukuqeqesha imodeli kumavidiyo ahambisanayo aphelele. Ababhali babika intuthuko kuwo wonke ama-benchmarks wokuqonda amavidiyo amaningi kusetshenziswa amasampula e-VQA ayi-10,000, ngokudlula okukodwa okuya phambili kanye namathokheni okukhiphayo anomkhawulo ekuqondeni.
Iphepha, elithunyelwe ku-arXiv ngomhla ka-Aug. 21, 2026, likhuluma ngokuqondwa kwevidiyo ende ngamamodeli wezilimi ezinkulu. Isiqalo sayo ukuthi amavidiyo amade nayinkimbinkimbi aqukethe izinto eziphazamisayo ezingafihla imininingwane yendawo. Ngokusho kwababhali, lokhu kungaholela ekutheni imodeli igxile ebufakazini obungalungile futhi ikhiqize impendulo engalungile. Ngakho-ke ucwaningo lugxile emandleni athile e-AI: ukuxhuma umbuzo mayelana nevidiyo nesikhathi esifanele noma ingxenye engaphakathi kwaleyo vidiyo.
Indlela ehlongozwayo ibizwa nge-Segment-to-Video Supervision, noma i-S2V. Abacwaningi baqale bakhiqize izibonelo ezibukwayo zokuphendula imibuzo ezigabeni zasendaweni, ezimfushane. Lezo zibonelo zibe sezidluliselwa emuva kusilungiselelo sevidiyo egcwele ngesikhathi sokuqeqeshwa. Isizathu esishiwo ukuthi amasegimenti amafushane enza imininingwane enezinhlamvu ezinhle kube lula ukubonwa, kuyilapho ukuqeqeshwa kuvidiyo egcwele kufundisa imodeli ukuhlobanisa leyo mininingwane nemibuzo naphezu kokuba khona kokuqukethwe okungahlobene. Indlela isebenzisa ukufunda okuqiniswayo nalokho okuchazwa yi-abstract njengomvuzo olula osuselwe ekunembeni kanye namasampula e-VQA ayi-10,000.
Ngesikhathi sokunquma, imodeli ye-S2V ewumphumela iklanyelwe ukuphendula ngokudlula okukodwa okuya phambili kanye nenani elilinganiselwe lamathokheni okukhiphayo. Ababhali bathi ukuhlola kwabo kukhombisa ukuthuthuka okungaguquki kumabhentshimakhi okuqonda amavidiyo amaningi uma kuqhathaniswa nawo womabili amamodeli ajwayelekile e-multimodal nezindlela ezisuselwe ekucabangeni. Baphinde bafune izinzuzo ekuqeqeshweni nasekusebenziseni izinkomba. Umthombo awuwahlonzi amabhentshimakhi noma unikeze usayizi wezinzuzo ezibikiwe ku-abstract, ngakho-ke lezo zimangalo kufanele zithathwe njengemiphumela ebikwe yiphepha esikhundleni samaqiniso asungulwe ngokuzimele.
Imininingwane yomthombo: arxiv.org ↗
Kungani kubalulekile
Amavidiyo amade aqukethe inani elikhulu lezinto ezingabalulekile, okwenza kube nzima ngezinhlelo ze-AI ukuthola ubufakazi obuthile obudingekayo ukuphendula umbuzo. Indlela yephepha iqondise kuleyo nkinga ngenkathi ifuna ukugwema izindleko eziphezulu zezichasiselo, ukuklanywa komvuzo oyinkimbinkimbi kanye nokubambezeleka okuhlotshaniswa nezindlela zokucabanga eziyinkimbinkimbi. Uma iqinisekiswa ngokuzimele, indlela yokwenza ingenza ukuhlaziya kwevidiyo ende kusebenze kakhulu kuzilungiselelo lapho ukubala, isikhathi sokuphendula noma izinsiza zokulebula ziboshelwe.
Ukuhlaziya ividiyo ende kunzima ngesizathu esiqondile: ulwazi oludingekayo ukuphendula umbuzo lungathatha ingxenye encane yomongo omkhulu kakhulu. Uhlelo lwe-AI olucubungula yonke ividiyo kufanele luhlukanise ubufakazi obubalulekile kumsebenzi wangemuva, izigcawu eziphindaphindiwe nemicimbi engahlobene. Indlela ye-S2V igxile ekuqeqesheni ebufakazini bendawo kuqala, bese isebenzisa ividiyo ephelele njengesilungiselelo lapho lobo bufakazi kumele bubuyiselwe khona. Leyo yimpendulo eqondiwe emkhawulweni omaphakathi ezinhlelweni ze-AI zamavidiyo.
Isimangalo sokusebenza kahle singase sibaluleke kakhulu ngoba ukuthuthukisa ukucabanga ngokuvamile kuza nezindleko ezengeziwe. Leli phepha lithi izindlela zangaphambili zingadinga ukuqinisa-ukuhlela kahle phezulu, izichasiselo ezibizayo kanye nemiklamo eyinkimbinkimbi yemivuzo. Iphinde ithi ezinye izinhlelo zokuzicabangela noma zokuzibona zikhiqiza izimpendulo ezinde futhi zikhuphule ukubambezeleka kokuqondisisa. I-S2V ihlose ukunciphisa leyo mithwalo ngesethi yokuqeqeshwa ye-VQA encane, umvuzo olula wokunemba kanye nenqubo yempendulo yephasi elilodwa. Uma ukusebenza okubikiwe kuphindwa, indlela ingaba nendaba konjiniyela abacubungula amaqoqo amavidiyo amakhulu noma basebenze ngaphansi kwesikhathi sokuphendula kanye nemikhawulo yokubala.
Ukubaluleka okungokoqobo kuhlala kunemibandela. Ukusebenza kwebhentshimakhi okungcono ngeke ngokwako kusungule ukuqonda okuthembekile kuvidiyo yomhlaba wangempela, lapho imibuzo ingase idideke, ubufakazi obufanele bungasatshalaliswa ezikhathini ezikude, futhi amaphutha angaba nemiphumela ehlukile kuye ngohlelo lokusebenza. Umthombo awubiki imiphumela yokusebenzisa, ukuhlola komuntu, ukuhlola okuqondene nesizinda noma amanani okuhluleka. Futhi ayiqinisekisi ukuthi indlela yehlisa izindleko eziphelele kuzo zonke izilungiselelo, njengoba ukudala izibonelo zokuqeqeshwa zendawo kanye nokulungiselela okokufaka kwevidiyo egcwele kungase kubeke umthwalo wabo siqu.
I-Interactive Mechanism: Indlela Esebenza Ngayo Ngempela
Hlola ubuchwepheshe obuyisisekelo ngemuva kwalokhu kuthuthukiswa ngokuhlanganyela.
Which component of an AI application is the machine-learning model itself?
Ongakubuka ngokulandelayo
Iphepha liyi-arXiv eliphrintiwe ngaphambili, futhi i-abstract yalo ayinikezi izinzuzo zokunemba okunembile, amagama ebhentshimakhi, izimfuneko zokubala noma ukuqhathanisa ngemininingwane yezinombolo. Ukucutshungulwa okwengeziwe kufanele kuhlole ukuthi ingabe ukuthuthukiswa okubikiwe kubamba ubude bevidiyo yonkana, izizinda nezinhlobo zemibuzo, ukuthi ukukhetha kwesegimenti kwazisa izindawo eziyizimpumputhe, nokuthi ingabe indlela ihlala isebenza kahle yini uma isetshenziswa namamodeli amakhulu noma ahlukene e-multimodal.
Ubufakazi obulandelayo obubalulekile yimininingwane yezinombolo kanye neyendlela ephuma ephepheni eligcwele. Abafundi kufanele babheke ubunikazi nosayizi bamabhentshimakhi evidiyo ende, amamodeli ayisisekelo, izinguquko zokunemba kwangempela kanye nezilinganiso zokuqeqesha neziqondiso. Kuzophinde kube nendaba ukuthi iziqhathaniso zisebenzisa imindeni eyimodeli efanayo, okokufaka kwevidiyo kanye nebhajethi yokubala. Ngaphandle kwaleyo mininingwane, i-abstract isekela ukuba khona kwendlela ehlongozwayo kanye nesiqondiso esibikiwe sababhali semiphumela, kodwa hhayi isilinganiso esinembile senzuzo yayo.
Ukugadwa okusekelwe esigabeni kungase kwethule ukuhwebelana uma iziqeshana ezikhethiwe zingaqukethe umongo owanele. Ukuhlola kufanele kuhlole imibuzo izimpendulo zayo ezincike ezenzakalweni zangaphambi nangemuva kwesegimenti yendawo, ukusebenzelana ezindaweni ezikude zevidiyo, noma ubudlelwano besikhashana obucashile. Kufanele futhi ihlole ukuthi ingabe ukukhiqizwa kwesegimenti kuthanda imininingwane ebonakalayo ngenkathi ishoda ngomsindo, ukulandelana kwezikhathi, ubunikazi besikhulumi noma umongo wokulandisa obanzi. Umthombo uchaza impendulo yemibuzo yevidiyo kodwa awucacisi ukuthi lezi zindlela noma amacala anzima ayesingathwa kanjani.
Ukuphindaphinda okuzimele kuzonquma ukuthi i-S2V iyindlela yokuqeqesha ewusizo kabanzi noma umphumela oxhumene nedatha ethile nokukhetha kwemodeli. Umsebenzi wokulandelela owusizo ungaqhathanisa indlela namanye amasu evidiyo ende asebenzayo, ihlole ezizindeni ezingabonwa futhi ilinganise amaphethini wamaphutha ngokuhambisana nokunemba. Iphepha lisanda kuthunyelwa futhi alinakho ukuqinisekiswa kwangaphandle okubikiwe emthonjeni onikeziwe. Kuze kube yilapho lawo masheke etholakala, umnikelo wawo osekelwa ngokuqinile uyisiphakamiso esiphathekayo esinophawu lokulinganisa olubikwe umbhali kanye nokuthuthukiswa kokusebenza kahle, hhayi ubufakazi bokuthi i-AI yevidiyo ende ixazulule ukucabanga kwendawo.