Byagenze bite
Icapiro rya arXiv ryapimwe GPT-4o, Claude Sonnet 4.5, Gemini 2.5 Flash, Qwen-2.5 na Llama-3.1 kuri dataseti esheshatu zikubiyemo ururimi rwa Nastaliq Urdu, Icyongereza n'Icyongereza. Abanditsi bavuga ko ibyiciro byahinduwe hagati yumwimerere-Urdu-inyandiko n’ibisobanuro by’icyongereza, hamwe nibintu byangiza byanyuze nkibisanzwe mu Urdu. Uru rupapuro rutangaza kandi ko nta mpapuro zabugenewe zabigenewe mu mpapuro 205 mu cyenda cyakozwe na ALW / WOAH.
Uru rupapuro rwashyikirijwe arXiv ku ya 25 Kanama 2026, rwiga niba imiterere y’indimi nini ishyira mu majwi imvugo y’urwango mu buryo butandukanye. Ubushakashatsi bwarwo bukubiyemo moderi eshanu-GPT-4o, Claude Sonnet 4.5, Gemini 2.5 Flash, Qwen-2.5 na Llama-3.1 - na datasets esheshatu zikoresha ururimi rwa Nastaliq, Urdu, Icyongereza na Kode. Inkomoko yerekana akazi nkiperereza ryumutekano wambukiranya imipaka, aho kuba ibicuruzwa cyangwa raporo yibyabaye kuri platform.
Abanditsi bagereranya ibyiciro biri mubyanditswe byumwimerere hamwe nibyiciro byahinduwe mucyongereza. Bavuga ko umutekano udahwitse kuri dataseti eshanu zanditswemo Urdu-inyandiko ziri hagati ya 15.9% kuri Gemini 2.5 Flash kugeza 31.6% kuri Qwen-2.5. Mu magambo y’uru rupapuro, urubanza "rwabuze-mu-Urdu" rurimo ibirimo byerekanwa ko ari bibi mu buhinduzi bw’icyongereza ariko bigashyirwa nkibisanzwe mu myandikire yacyo ya Urdu.
Ibipimo byavuzwe byabuze-mu-Urdu biri hagati ya 2,4% na 9.9%, hamwe na mediani ya 4.3% murwego rwo hejuru. Abstract ivuga ko moderi ntoya ifunguye-uburemere yerekanaga cyane ihungabana ryinshi hamwe nigipimo cyabuze-cyangiritse kurenza imipaka ifunze imipaka yashyizwe mubizamini. Ibi ni ibisubizo byatangajwe n'abanditsi; isoko yatanzwe ntabwo itanga ingero zifatika, kuri gusenyuka, intera ikizere cyangwa ibizamini byibarurishamibare.
Uru rupapuro rusuzuma kandi ubushakashatsi bwakozwe ku isuzuma ry’umutekano wa Urdu. Dukurikije ibisobanuro, abanditsi bakoresheje ACL Anthology API kugira ngo babare impapuro 205 mu cyenda ALW / WOAH basanze impapuro zeru zabugenewe muri icyo gihe. Inkomoko ntisobanura ibipimo ngenderwaho byose muri iryo barura cyangwa ngo isobanure niba imirimo ifatika ishobora kuba yagaragaye ahandi hantu, ibirango cyangwa ibyiciro byubushakashatsi.
Ibisobanuro birambuye: arxiv.org ↗
Impamvu ari ngombwa
Ubushakashatsi bwerekana ko sisitemu yo kugereranya ibintu ishobora kugaragara neza iyo igeragejwe binyuze mu buhinduzi bw'icyongereza kuruta iyo ikora urdu mu buryo butaziguye. Ibyo bitera ikibazo gifatika cyo gusuzuma kurubuga na serivisi zikorera abakoresha indimi nyinshi: amanota yumutekano rusange arashobora guhisha kunanirwa kwibanda kumyandikire cyangwa ubwoko bwindimi. Ibyavuye mu bushakashatsi ni ibyasabwe na progaramu imwe ya arXiv kandi ntibigaragaza uburyo ibipimo byapimwe bikwirakwizwa muri rusange kurenza imibare na moderi zapimwe.
Intego nyamukuru ni uko isuzuma ryumutekano rishobora guterwa nuburyo ururimi ruhagarariwe, ntabwo rushingiye gusa kubyo inyandiko isobanura. Niba moderi igereranijwe igeragezwa cyane cyane mucyongereza cyangwa ku bikoresho byahinduwe, imikorere yapimwe ntishobora kwerekana kunanirwa kugaragara iyo abakoresha banditse mu nyandiko ya Urdu. Sisitemu rero irashobora kwakira ibisubizo byizeza ibisubizo mugihe ikora ibintu bitandukanye muburyo butandukanye.
Ibyo bifite akamaro kuko kugereranya ibintu bikoreshwa muguhitamo niba ibikoresho byahagaritswe, byongerewe cyangwa byemewe. Raporo yabuze-mu-Urdu isobanura kunanirwa kwerekanwa: inyandiko yagaragaye ko ari mbi nyuma yubusobanuro rimwe na rimwe bwanyujijwe nkuko bisanzwe mubyanditswe byumwimerere. Ubushakashatsi ntibwerekana inshuro nyinshi ibibazo nk'ibi bibera ku mbuga nkoranyambaga, ni ubuhe bwoko bw'imvugo zabigizemo uruhare, cyangwa ingaruka zakurikiyeho, bityo ingaruka zacyo zikwiye gusobanurwa nk'impungenge zakozwe aho kuba igereranyo cy’ibibi.
Akazi kandi gatanga amatsinda yumutekano amagambo asobanutse yo gupima. Muri rusange ubunyangamugayo cyangwa amasezerano birashobora guhisha ukutumvikana hagati yinyandiko, mugihe kugereranya inyandiko-mvugo hamwe nigipimo cyabuze-mu-Urdu gishobora kwerekana icyiciro kimwe cyibintu byangiritse byabuze. Izi ngamba zishobora gufasha amashyirahamwe kugenzura ururimi, kugereranya verisiyo yicyitegererezo no kumenya aho ibizamini bishingiye ku buhinduzi bidahagije. Inkomoko ntivuga ko amanota yatanzwe ari igipimo cyuzuye cyumutekano.
Ikinyuranyo cyatangajwe hagati yuburemere-buremereye nicyitegererezo gifunze bishobora guhindura uburyo amashyirahamwe ahitamo cyangwa agakoresha sisitemu yo kugereranya indimi nyinshi, ariko kugereranya bisaba ubwitonzi. Icyitegererezo kirashobora gutandukana mumyitozo yamakuru, guhuza, uburyo bwo kugera hamwe nimyitwarire isanzwe, kandi abstract ntabwo itanga ibisobanuro bihagije byuburyo bwo gutandukanya icyateye itandukaniro. Uru rupapuro rero rushyigikira igenzura ryikitegererezo hamwe ninyandiko ikwirakwizwa, ntabwo ari umwanzuro wuzuye ko icyiciro kimwe cyicyitegererezo gifite umutekano rusange.
Uburyo bukoreshwa: Uburyo bukora
Shakisha ikoranabuhanga ryihishe inyuma yiri terambere.
Why can ethical evaluation not be reduced to one model score?
Ibyo kureba
Ibibazo byingenzi bikurikiraho ni ukumenya niba ibisubizo bikomeza urungano rusubirwamo no kwigana, uko imibare yateranijwe ikanashyirwaho ikimenyetso, kandi niba uburyo bumwe bugaragara muri sisitemu yo gukora ubu. Abashakashatsi n'ababitanga bagomba kandi gusuzuma Ikirundi cy'ikiromani, guhinduranya kode, guhindagurika kw'akarere, ubwiza bw'ubuhinduzi, hamwe no kuvugurura icyitegererezo bitandukanye. Inkomoko ntishyiraho ibisobanuro, ibipimo ntarengwa, uburyo bwo gutoranya cyangwa uburyo bwo kugabanya ibicuruzwa byagabanya amakuru avuguruzanya.
Ikintu cya mbere kidashidikanywaho ni uburyo. Inkomoko yatanzwe ni progaramu ya arXiv v1, ntabwo ari ibimenyetso byerekana ko akazi karangije gusuzuma urungano. Basomyi bakeneye impapuro zuzuye za ingano, ibisobanuro bya label, inzira ya annotator, inzira yubusobanuro, ibisobanuro, igenamiterere rya decoding hamwe nisesengura ryibarurishamibare kugirango barebe uko ijanisha ryavuzwe rikomeye.
Kwigana bizaba ingenzi cyane. Ubushakashatsi bugerageza bitanu byitwa moderi hamwe nuruhererekane rwihariye rwa datasets, ariko ibisobanuro ntibisobanura niba ibiciro bigera kuri verisiyo nshya yicyitegererezo, ubundi buryo bwo kugereranya, ubwoko bwikinyarwanda cyangwa indimi zijyanye nabyo. Ntabwo yerekana kandi niba amakosa yibanze ku ngingo zihariye, uburyo bwo kuvuga urwango, imiterere yimyandikire cyangwa urwego rwo guhinduranya code.
Abakohereza bagomba kureba isuzuma ryerekana umwimerere-inyandiko yinjiza mu buryo butaziguye aho gufata igisobanuro cyicyongereza nka proksi ihagije. Igikorwa cyingirakamaro cyo gukurikirana cyagereranya ibisubizo byihariye byanditswe mugihe, gutanga raporo yibinyoma hamwe nibibi byabuze, no kugerageza niba impinduka zoroheje zitezimbere ururimi rumwe rutatesheje agaciro urundi. Inkomoko ntisobanura intervention yemewe.
Ubushakashatsi-bukubiyemo ubushakashatsi nabwo butanga igenzura. Abanditsi bavuga ko impapuro zeru zabugenewe zanditse mu mpapuro 205 hirya no hino icyenda ALW / WOAH, ariko ibisobanuro ntibitanga inyandiko zishakisha byuzuye cyangwa ngo bisobanure uko "Urdu" rwiyeguriye "rwasobanuwe. Igenzura ry'ejo hazaza rigomba gutuma ibyo bipimo byororoka kandi bigasuzuma imirimo hanze y'iburanisha. Kugeza icyo gihe, impapuro zishimangiwe cyane ni uko inyandiko ya Urdu ikwiye gupimwa neza mugupima umutekano wa LLM.