Byagenze bite
Abashakashatsi batangije uburyo bwo kubara inguzanyo zishingiye ku kubara, uburyo bwo gutanga inguzanyo-ishimangira-inguzanyo ku kimenyetso ku giti cyabo ukurikije ibarwa ryakozwe n'icyitegererezo cy'ururimi mu gihe cyo gusubiza. Ishyirwa mu bikorwa ryabo, CompPO, rikoresha ibitekerezo byibanze kugirango habeho irembo ryo kugumisha kuri buri kimenyetso kandi rikabihuza numunegura wanga leta zihishe umukinnyi hamwe namakuru yamakuru.
Inyandiko yashyikirijwe arXiv ku ya 21 Kanama itanga uburyo bushya bwo gutanga inguzanyo mugihe cyo kwiga gushimangira imiterere yindimi nini. Uru rupapuro rutandukanya inshingano zinguzanyo mubice bitatu: ibimenyetso byerekana niba umuzingo wagenze neza, umuyobozi ushinzwe gutwara abantu uhindura ibyo bimenyetso mubyiza byo murwego rwo hejuru, hamwe no kuvugurura geometrie ihindura izo nyungu impinduka za politiki. Abanditsi bavuga ko imirimo iherutse yateje imbere igice cya mbere n'icya gatatu, mu gihe amategeko akoreshwa mu gutwara abantu aguma ahanini atigenga ku myubakire y'icyitegererezo.
Urwego rwateganijwe, rwitwa comptabilite-itwara inguzanyo itwara inguzanyo, ikoresha imibare itandukanijwe na politiki yimyitwarire ibarwa imbere kugirango igaragaze uburyo agaciro kamanuka kanyuzwa mumuzingo. Algorithm ifatika, CompPO, ihindura ibitekerezo bya kavukire mumarembo agumanye kuri buri kimenyetso. Irembo rikoreshwa haba murwego rumwe rwo gukuramo intambwe no kumuhanda-ushingiye kumurongo rusange-inyungu yiswe Comp-GAE. Abanditsi bavuga ko irembo rihoraho rigabanya uburyo bwo kugereranya-coefficient rusange igereranya inyungu, itanga umurongo wibanze.
CompPO ikubiyemo kandi kunegura transport, cyangwa TAC. Aho kongeramo Transformer ya kabiri yubunini bumwe, TAC yongera gukoresha reta zihishe zabakinnyi no kuyobora amakuru. Uru rupapuro ruvuga ko igihembo cyibikorwa hamwe na politiki ya PPO yaciwe bidahinduka; impinduka zisabwa nuburyo inguzanyo zitwarwa nuburyo abanegura bahujwe nubwikorezi. Mu bushakashatsi bwakoresheje imbuto eshanu za Qwen3-4B, abanditsi bavuga ko 61.4% ya nyuma yakozwe neza, hamwe n’icyizere cya 95% hagati ya 60.8% na 62.0%, ugereranije na 53.8%, intera iri hagati ya 52.9% na 54.7%, kuri GRPO.
Impapuro zo gukuraho impapuro zerekana ibisubizo bivuye guhuza ibice. Comp-GAE ihujwe nabanegura bisanzwe yageze kuri 55.2%, mugihe TAC ihujwe n irembo rihamye ryageze kuri 56.4%; ntaho bihuriye na sisitemu yuzuye. Abanditsi bavuga ingaruka zikorana n amanota 2.4, hamwe 95% intera yicyizere hagati ya 1.9 na 2.9. Guhindura no kugenzura imyanya byavuzwe kugirango bishyigikire inzira yihariye. CompPO yari ihagaze neza muri 10 kuri 12 PPO-grid ikora, ugereranije na 3 kuri 12 yo kugereranya. Ku isuzuma ryahagaritswe, abanditsi bavuga ko hari byinshi byateye imbere kuri GRPO ya 4.3 na 3.9 umururumba watsinze @ 1 macro amanota kuri Qwen3-4B na Llama-3.1-8B-Amabwiriza.
Ibisobanuro birambuye: arxiv.org ↗
Impamvu ari ngombwa
Igisubizo gikemura ikibazo gikomeye muburyo bwo kwiga gushimangira imiterere yindimi nini: guhitamo ibice byigisubizo kirekire bikwiye inguzanyo cyangwa amakosa kubisubizo byanyuma. Abanditsi batangaza ko hari byinshi byahinduwe kuri GRPO, ariko ibimenyetso biva mubitegura hamwe nubushakashatsi buke, bityo uburyo rusange hamwe nigiciro cyibikorwa bikomeza kutamenyekana.
Kwiga gushimangira icyitegererezo cyururimi bigomba guhuza ibihembo byanyuma nibimenyetso byinshi kugiti cyawe hamwe nicyemezo hagati. Igisubizo kirashobora kuba kirimo intambwe zingirakamaro kandi zidafasha, ariko uburyo bwerekana imibare imwe y'ibisubizo mubisubizo byose birashobora gutanga ubuyobozi buke. Urupapuro rwibanze rwibanze ni uko kubara kwicyitegererezo ubwabyo bishobora gutanga ikimenyetso cyihariye cyo guhitamo uburyo inguzanyo igomba gukomeza kuva kumurongo kugeza ku kindi.
Niba ingaruka zavuzwe zifata mugari mugari, ubwubatsi-bumenyereye ubwikorezi bwinguzanyo bushobora gutuma imbaraga-yiga-ivugurura igamije cyane cyane idasabye ibisobanuro bitandukanye cyangwa intego ya politiki. Ibyo bifite akamaro kuko impinduka kumurimo w'inguzanyo zirashobora kwinjizwa mumiyoboro isanzwe ya PPO. Kugereranya byatangajwe na GRPO bifitanye isano cyane cyane na LLM yo kongera imbaraga-yiga kuko igena uburyo bwateganijwe nkimpinduka zerekana ibimenyetso byamahugurwa aho kuba umuryango mushya wicyitegererezo cyangwa ibicuruzwa bireba abakoresha.
Ivanwaho ryavuzwe ni ingirakamaro kuko ryerekana ko ibisubizo bidasobanuwe n irembo ryaturutse kubitekerezo cyangwa uwongeye kunegura wenyine. Uburyo bwuzuye bwakoze neza kurenza ibice byombi mubice byatanzwe, kandi abanditsi bavuga ko guhinduranya no kugenzura bishyigikira igitekerezo cyuko guhuza inzira byihariye. Kugereranya gushikamye byerekana kandi inyungu zifatika zishoboka: kwiruka muke bidahungabana bishobora kugabanya imyitozo yatakaye, nubwo isoko idatanga ibipimo cyangwa kubara ibiciro.
Ibimenyetso bigomba gusomwa nkibisubizo byubushakashatsi hakiri kare. Inkomoko ni inyandiko ya arXiv, ntabwo ari igitabo cyasuzumwe n’urungano, kandi ibisobanuro ntibisobanura imirimo ishingiyeho, ingano ya dataset, ibisobanuro byashyizwe mu bikorwa, ingengo yimari y’amahugurwa, cyangwa uburyo bw’ibarurishamibare burenze intera yatangajwe. Ntabwo ishyiraho kandi niba inyungu ziza hamwe nububiko bwinyongera, ubukererwe, ubuhanga bukomeye, cyangwa sensitivite kubitekerezo. Uburyo bukoreshwa muburyo rusange rero buterwa no kumenya niba abashakashatsi bigenga bashobora kubyara ibisubizo kandi niba inzira yimukira mubyitegererezo binini kandi intego zinyuranye zo gushimangira-kwiga.
Uburyo bukoreshwa: Uburyo bukora
Shakisha ikoranabuhanga ryihishe inyuma yiri terambere.
Which component of an AI application is the machine-learning model itself?
Ibyo kureba
Ibizamini byingenzi bizakurikiraho ni kwigana byigenga, icyitegererezo cyagutse no gukwirakwiza imirimo, no kugereranya birimo ikiguzi cyamahugurwa, gukoresha kwibuka, nigihe cyo gukora. Inkomoko ntisobanura niba CompPO ikora yizewe irenze Qwen3-4B na Llama-3.1-8B-Isuzuma ryisuzuma cyangwa niba ibimenyetso byayo bishingiye kubitekerezo bikomeza kuba ingirakamaro muburyo butandukanye bwububiko.
Icyambere cyambere ni kwigana kurenza imbuto eshanu za Qwen3-4B hamwe nisuzuma ryakonje. Amatsinda yigenga agomba kugerageza uburyo kubunini bw'icyitegererezo, imirimo yo guhugura, imiterere y'ibihembo, hamwe nururimi-rwerekana imiterere. Inkomoko yizina Qwen3-4B na Llama-3.1-8B-Amabwiriza, ariko ntivuga niba uburyo bwarasuzumwe muburyo butandukanye bwikitegererezo cyangwa niba ibimenyetso byitondewe bitwara kimwe mubyubatswe bifite inzira zitandukanye cyangwa ibishushanyo mbonera.
Abashakashatsi bagomba kandi gupima amahugurwa yuzuye. Uru rupapuro ruvuga ko TAC yongera gukoresha abakinnyi bahishe kandi bagahindura amakuru nta ntera ya kabiri ihwanye na Transformer, ariko inkomoko ntabwo igereranya imikoreshereze yibuka, igihe cyo gutoza amasaha yo ku rukuta, ibisabwa byuma, cyangwa compte yose. Ibyo bipimo bizagaragaza niba amakuru yatangajwe ku nyungu n’umutekano bihamye ari ingirakamaro ku mashyirahamwe asanzwe akora imiyoboro ihenze yo kongera imbaraga-yiga.
Ibindi bikorwa bigomba gusuzuma uburyo irembo ryo kugumya kwitabwaho rikomeye. Raporo ihindagurika hamwe nu mwanya bigenzura bishyigikira guhuza inzira yihariye mu bushakashatsi, ariko isoko ntisobanura uburyo irembo ryitabira ibintu birebire, ibimenyetso bidasanzwe byo gutekereza, ibihembo bidakabije cyangwa urusaku, cyangwa impinduka mukubabaza no gutoranya. Ntabwo bizwi kandi niba kwibanda kubitekerezo ari porokisi yizewe yo kubara akamaro cyangwa gusa ikimenyetso cyingirakamaro kubikorwa byihariye byageragejwe.
Hanyuma, imiterere yuburyo nkintangiriro yingirakamaro. Inkomoko ntabwo itanga igenzura ryigenga, kohereza umusaruro, kode iboneka, cyangwa ibisubizo byurungano. Ibyo kutirengagiza ntibitesha agaciro ibyagaragaye, ariko bisiga ibibazo byingenzi bidasubijwe kubyerekeranye no kubyara no kubishyira muri rusange. Urubanza rukomeye rwasaba ibisobanuro birambuye byashyizwe mubikorwa, bihuye-ibiciro-fatizo, ibisubizo hejuru yinyubako zubatswe, hamwe nibimenyetso byerekana ko iterambere rikomeza hanze yisuzuma ryabanditsi.