Buyela Ezindabeni
UkuqambaAI Understanding ukwaziswa

I-GapSight ifundisa amamodeli olimi lombono ukuthi kufanele ubheke nini futhi kuphi futhi

Iphrinta entsha ye-arXiv yethula i-GapSight, indlela evumela amamodeli olimi lombono ngokukhetha ukuthi avakashele izifunda zesithombe lapho ukubuka komhlaba okunokulungiswa okuphansi kulahlekelwa imininingwane ebalulekile. Ababhali babika amaphuzu aphezulu amaphakathi kuwo wonke amabhentshimakhi ayisithupha, ngenkathi bephawula ukuthi imiphumela iphuma ekuhlolweni kwabo futhi ihlala…

5 min readRead the primary source
Primary-source image accompanying GapSight teaches vision-language models when and where to look again
Idokhumenti yomthombo oyinhlokoUmthombo urekhodiwe
Umshicileli
arxiv.org
Isixhumanisi somthombo
arxiv.orghttps://arxiv.org/abs/2608.21762
Uhlobo lomthombo
Idokhumenti eyisisekelo — isimemezelo esisemthethweni, iphepha, ukugcwalisa, noma ikhasi lomuntu wokuqala esilifunda ngokuqondile.
UmongoQonda lokhu ngemizuzwana engama-60

Qala lapha

Imigomo ebalulekile

I-OCR (Ukubonwa Kwezinhlamvu Ezikhanyayo)
Ubuchwepheshe obuguqula umbhalo ezithombeni noma izikena zibe umbhalo ofundeka ngomshini.
Imodeli Yolimi Lombono (VLM)
Imodeli ye-multimodal ecubungula ngokuhlanganyela ulwazi olubonakalayo nolubhaliwe.
Inkumbulo (Inkumbulo yomenzeli)
Ingqikithi egciniwe umenzeli we-AI usebenzisa ezinyathelweni zonke noma izikhathi ukuze athuthukise ukuqhubeka.
ZihloleImibuzo Ecacisiwe yamamodeli e-AI

Kwenzekeni

Abacwaningi bahlongoza i-GapSight, uhlaka lokuqeqeshwa kanye nencazelo yamamodeli olimi lombono oluqala ngokuhlola isithombe emhlabeni wonke bese lunquma ukuthi luzosihlola yini isitshalo esikhululekile. Iphepha libika ukuthi le ndlela ithuthukise ukusebenza kuwo wonke amabhentshimakhi ayisithupha ahlanganisa i-OCR, amadokhumenti, amashadi, i-infographics, i-VStarBench ne-MME-RealWorld-Lite.

Iphepha likhuluma ngomkhawulo othile kumamodeli olimi lombono: imodeli ingathola isithombe, kodwa imininingwane edingekayo ukuze kuphenduleke umbuzo inganyamalala lapho isithombe sicindezelwa sibe ukumelelwa komhlaba okunokulungiswa okuphansi. Ababhali baphikisa ngokuthi ukwaba amathokheni abonakalayo amaningi kuwo wonke umbuzo akuphumelelanga futhi kungaphazamisa imisebenzi encike esigcawini sisonke noma umongo wedokhumenti.

I-GapSight isebenzisa inqubo enezigaba ezimbili. Imodeli yolimi lombono kuqala ithatha umbono womhlaba wonke. Irutha yokunqampuna engasindi ibese ibikezela ukuthi ingabe umbuzo udinga ukubukeka kwesibili, ilinganisela ukusizakala okulindelekile kwalokho kubuyekezwa, bese ikhetha ibhokisi lokunqampuna eliqhubekayo. Umbono wesibili uhloselwe ukunikeza ubufakazi bendawo kuphela lapho ukumelwa komhlaba kunganele.

Ukugadwa kuvela ekulahlekelweni kwempendulo yemodeli eqondiwe noma imajini yenketho yokukhetha okuningi. Ngesikhathi sokuqeqeshwa okungaxhunyiwe ku-inthanethi, abacwaningi baqhathanisa ukusebenza kwemodeli besebenzisa ukubuka komhlaba kuphela nokusebenza kwayo ngemva kokwengeza izitshalo zamakhandidethi. Isitshalo esithuthukisa impendulo eqondiwe siba ilebula yokubuyekeza yokuqeqesha umzila. Lokhu kwenza indlela iqonde ngqo kunokuthembela kuphela kuzichasiselo zesifunda esijwayelekile, ngokusho kwephepha.

Ababhali babika izivivinyo nge-LLaVA-1.5-7B, InternVL2.5-8B kanye ne-Qwen2-VL-2B-Yala kumabhentshimakhi ayisithupha ahlanganisa i-OCR, amadokhumenti, amashadi, i-infographics, i-VStarBench ne-MME-RealWorld-Lite. Ku-InternVL2.5-8B, isilinganiso samabhentshimakhi ayisithupha okubikiwe sikhuphuke sisuka ku-52.25 kusistimu yesisekelo sokungasondeza saya ku-64.29 nge-GapSight. Iphepha liqhathanisa lowo mphumela nezilinganiso ezibikiwe ezingu-57.16 ze-CropVLM, 55.84 ze-ViCrop kanye no-54.43 ze-ZoomRefine.

Umthombo uhlonza iphepha njengokuthunyelwe kwe-arXiv okwenziwa ngomhla ka-Aug. 22, 2026. Kuyi-preprint, futhi umthombo onikeziwe uqukethe ikhasi elingabonakali nelebhayibhilogi kunemininingwane egcwele yokuhlola. Ngakho-ke umthombo awuqinisekisi ukwakheka kwebhentshimakhi okuyiyonayona, ukuhlola kwezibalo, ukubala izindleko, ukutholakala kokusetshenziswa noma ukuthi izinzuzo zikhiqizwe kabusha ngokuzimele.

Imininingwane yomthombo: arxiv.org ↗

Kungani kubalulekile

Amasistimu olimi olubonakalayo angaphuthelwa ulwazi olubonakalayo esithombeni kodwa olulahlekile phakathi nokucubungula okunokulungiswa okuphansi. Uma izinzuzo ezibikiwe ziqhubeka, ukufunda kabusha okukhethiwe okukhethiwe kungathuthukisa imisebenzi ebucayi ngaphandle kokusebenzisa ukucubungula okubizayo kokuxazulula okuphezulu kuwo wonke umbuzo.

Inkinga ebonakalayo ibalulekile kumasistimu aphendula imibuzo mayelana nezinto ezibonakalayo eziminyene. Ilebula yeshadi, igama elincane, okufakiwe kwethebula noma isici esibonakalayo sendawo singase sibe khona esithombeni sokuqala kodwa singatholakali kumodeli esebenza ekubukeni komhlaba okucindezelwe. Indlela enganquma ukuthi ihlolwe nini isifunda ingase iqondise ekubalweni kwezimo okungenzeka zizuze.

Umnikelo omaphakathi wephepha awukona nje ukusebenzisa isithombe esikhulu. Iphakamisa isinqumo esifundiwe mayelana nokuthi uzophinde ubheke kuphi futhi. Lowo mehluko ubalulekile ngoba ukucutshungulwa kokucaca okuphezulu kungakhuphula ukusetshenziswa kwamathokheni futhi kungase kunciphise ukuba wusizo komongo womhlaba. Uma izinqumo ze-router zithembekile, amasistimu angakwaba ukubala okubukwayo ngokukhetha kakhudlwana.

Imiphumela ebikiwe iphakamisa ukuthi indlela ingase ithuthukise izinhlobo ezimbalwa zemibuzo ebonakalayo ephendulwa ngesikhathi esisodwa kunokuba ibhekane nomsebenzi owodwa ochazwe ngokuncane. Ababhali bathi i-router ihlenga izimpendulo ezingalungile zekhonkrithi, ishintsha izinga layo lesenzo ngomsebenzi futhi ikhiqize iphrofayili yethokheni evumayo. Lezi izimangalo ezivela ekuhlaziyweni kwemishini yephepha, hhayi okutholwe ngokuzimela.

Noma iyiphi inzuzo isazoncika ezindlekweni nasekuthembekeni kwesinyathelo sokubuyekeza esengeziwe. Umthombo awunikezi ukubambezeleka, inkumbulo noma izibalo zamandla, futhi awusho ukuthi irutha ikhetha ukunqampuna kangaki. Futhi ayibonisi ukuthi ukuthuthukiswa kubonisa ukusetshenziswa kobufakazi obubonakalayo obungcono ngokujwayelekile noma izinzuzo ezihambisana namabhentshimakhi akhethiwe kanye namamodeli okuqondiwe asetshenziselwa ukukhiqiza ukugada.

Ngakho-ke umphumela uqondwa kangcono njengesiqondiso socwaningo esingase sibe usizo, hhayi ubufakazi bokuthi amamodeli olimi lombono angafunda ngokuthembekile imininingwane emincane ekusetshenzisweni okuvamile. Umthombo ubika ukuhlola okulawulwa ngababhali, okungenalo ulwazi olumayelana nokusetshenziswa kokukhiqiza, imiphumela yomsebenzisi, ukuhlolwa kokuphepha noma ukusebenza ezithombeni ezingaphandle kwamabhentshimakhi asohlwini.

Interactive Mechanism

I-Interactive Mechanism: Indlela Esebenza Ngayo Ngempela

Hlola ubuchwepheshe obuyisisekelo ngemuva kwalokhu kuthuthukiswa ngokuhlanganyela.

Agent Lifecycle Stage:
1
User Intent & Planning: "Audit customer refund request #4092 and settle payment."
2
Tool Calling: Emits structured JSON call crm_get_transaction(id='4092').
3
Guardrail & Verification:🛡️ Paused: High-value action requires human operator sign-off.
4
Final Settlement: Refund recorded, email receipt dispatched, and audit log stored.
Core takeaway: An AI agent is not just a language model—it is a closed loop of planning, tool invocation, and environment feedback. Production systems require self-healing retries and strict human approval guardrails.
I-Interactive Concept Check+10 Points
AI Models Explained Quiz

Which component of an AI application is the machine-learning model itself?

Ongakubuka ngokulandelayo

Imibuzo ebalulekile ukuthi ingabe imiphumela iphindaphinda kuwo wonke amamodeli namasethi edatha akhethwe ngokuzimela, ingakanani isibalo esengeziwe esidingwa umzila wokunqampuna, nokuthi ingabe indlela ihlala ithembekile uma izithombe, imibuzo noma amafomethi empendulo ehluka kuzilungiselelo zokuhlola. Umthombo awusunguli ukusetshenziswa komhlaba wangempela, ukutholakala kwekhodi, ukuqhathanisa komuntu noma ukuqinisekiswa okuzimele.

Ukuphindaphinda ukuhlolwa okulandelayo okucacile. Abacwaningi abazimele bazodinga ukusebenzisa noma ukuthola indlela, bayihlole kumabhentshimakhi afanayo, futhi bayihlole kumadathasethi engeziwe nemindeni eyimodeli. Umthombo awusho ukuthi ikhodi, amarutha aqeqeshiwe noma ama-artifact okuhlola ayatholakala, ngakho ukukhiqizwa kabusha kuhlala kuwumbuzo ovulekile.

Ubudlelwano phakathi kokunemba nokubala kufanele buhlolwe ngokucophelela. I-GapSight yakhelwe ukugwema ukusebenzisa amathokheni abonakalayo engeziwe kuwo wonke umbuzo, kodwa i-abstract ayibali inani lezitshalo, amathokheni, isikhathi sokucubungula noma izimfuneko zezingxenyekazi zekhompuyutha. Indlela ekhuphula amaphuzu kuyilapho ibeka izindleko ezinkulu ezifihliwe ingaba nevelu elingokoqobo elihlukile kunaleyo enikeza ukubuyekezwa okukhethiwe okuphumelelayo.

Ukuqina ngaphandle kokuhlelwa kokuqeqeshwa kungenye into engaziwa. Umzila uqeqeshwe kusukela ekulahlekelweni kwemodeli eqondiwe noma izinguquko zemajini yempendulo, ezingase ziyisize ihambisane naleyo modeli kodwa ezingakhawulela ukudluliswa lapho imodeli, ukusatshalaliswa kwesithombe, isitayela sombuzo noma ifomethi yempendulo ishintsha. Umthombo awukubiki ukudluliswa kwamamodeli ahlukahlukene noma ukumelana nemininingwane ebonakalayo edukisayo.

Isimangalo sephepha sokuthi i-router ihlenga izimpendulo ezingalungile kufanele ihlolwe ezingeni lombuzo ngamunye, hhayi kuphela ngama-avareji. Umsebenzi wokulandelela owusizo ungahlola ukuthi yimaphi amaphutha alungiswayo, yimaphi amaphutha angeniswa ngokunqampuna, kanye nokuthi imodeli ingabona yini lapho isilimo singenabo ubufakazi obanele. Umthombo awunikezi lokho kuhlukaniswa.

Okokugcina, ukuthuthukiswa okubikiwe kudinga ukuhlukaniswa emiphumeleni eqondene nebhentshimakhi. The abstract gives aggregate scores and names comparison systems, but it does not provide per-benchmark results, uncertainty estimates or human baselines. Leyo mininingwane izonquma ukuthi lokho okutholakele kufanele kuhunyushwe kabanzi kangakanani.

Imihlahlandlela ehlobene nemibuzo

Amamodeli e-AI AchaziweAma-TransformersUkuqeqeshwa kwe-AIChatGPT ne-LLMsHlola okwaziyo — zama imibuzo ye-AI yamahhalaBheka igama le-AI kuhlu lwethu lwamagamaLandela i-tracker yokukhishwa kwemodeli ye-AI
Uthole lokhu kuwusizo?