Horumarinta Siyaasadda Qaraabada ee Kooxda
Hagaajinta Siyaasadda Qaraabada Kooxda (GRPO) waa habka xoojinta-waxbarashada ee habaynta hab-dhaqanka hab-luuqadeed kaas oo ku qiimeeya jawaab kasta oo ka dhan ah koox jawaabaha walaalaha ah isla markiiba, meeshana ka saaraysa isku xidhka qiimaha gaarka ah ee ay isticmaasho PPO.
Dulmar
It became famous as the core training trick behind DeepSeek's reasoning models.
quusid qoto dheer
GRPO waa kala duwanaanshiyaha waxbarashada xoojinta-kordhinta ee loogu talagalay inay RL ka dhigto hagaajinta qaababka luqadda waaweyn mid jaban oo xasilloon. Heerka PPO wuxuu u baahan yahay 'naqdiye' (qaabka qiimaha) oo la bartay, qiyaas ahaan sida siyaasadda lafteeda, si loo qiyaaso sida calaamad kasta u wanaagsan tahay. GRPO gabi ahaanba meesha ayay ka saartay dhaleecayntaas. Mid kasta oo degdeg ah waxay muunad ka qaadaa koox dhammaystiran (dheh 8-64), kulligood waxay ku dhibaysaa calaamada abaal-marineed, ka dibna waxay xisaabisaa faa'iidada dhammaystir kasta iyada oo la jaan-qaadaysa abaal-marinteeda ka dhanka ah macnaha kooxda iyo leexashada heerka. Jawaabaha celcelis ahaan ka sarreeya waa la xoojiyay oo kuwa celcelis ahaan ka hooseeya waa la xakameeyay. Erayga kala-duwanaanshaha KL waxa uu qaabku ku ilaaliyaa siyaasadda tixraaca. Waxaa soo bandhigay DeepSeek, waxay xoojisay DeepSeekMath iyo moodooyinka sababaynta DeepSeek-R1.
Aragtida Farsamada
Fikradda muhiimka ah ayaa ah in lagu beddelo aasaaska qiimaha la bartay ee PPO laguna beddelo saldhigga kooxda Monte Carlo. Koox wax soo saar leh oo leh abaalmarin r_i, faa'iido kasta waa A_i = (r_i - mean(r)) / std(r). Dhibcahaas caadiga ah waxay badinaysaa saamiga ixtimaalka go'ay, si la mid ah PPO, iyo ganaaxa KL ee ka dhanka ah moodeel tixraaceedka la qaboojiyey ayaa xannibaya qulqulka. Sababtoo ah ma jiro dhaleecayn loo tababaray, xusuusta oo xisaabiya qiyaas ahaan kala badh, iyo caadi ka dhigista degdega ah waxay siisay si dabiici ah oo la miisaamay, faa'iidooyinka kala duwanaanshiyaha hoose.
Saamaynta Istiraatijiyadeed
Qiimaha iyo miisaaniyada
Go'aamada qaab-dhismeedku waxay horseedaan waxqabadka iyo kharashka hawlgalka sannadaha.
Go'aamo cad
Waxbarashada farsamada waxay ka caawisaa kooxaha inay doortaan xidhmo sax ah, ma aha oo kaliya kan ugu cusub.
Xakamaynta tayada
Doorashooyinka injineernimada ee wanaagsan waxay yareeyaan shilalka la isku halleyn karo ee wax soo saarka.
Mustaqbalka Siyaasadda Qaraabada ee Kooxda
GRPO waxa ay si degdeg ah u noqotay cunto karinta caadiga ah ee tababbarka moodooyinka caqliga furan, shaybaadhkana waxa ay ku celcelinayaan meelaha daciifka ah. Cilmi baadhayaashu waxay sahamiyaan hagaajinta dhererka iyo eexda dhibka (sida Dr. GRPO), heerka calaamada halkii ay ka ahaan lahaayeen isku xigxiga caadiga ah, iyo ka saarida ama dib u qaabaynta ereyga KL. Filo is dhexgal adag oo leh abaal-marin la xaqiijin karo (xisaab, koodhka, isticmaalka aaladda), maaraynta ka wanaagsan ee calaamadaha aan fiicneyn, iyo isku-dhafka isku-dhafka ah ee isku-dhafka kooxda iyo dhaleeceynta fudud ee hawl-qabad, hawlo badan.
Dhaqangelinta Adduunka-dhabta ah
Tababarka DeepSeek-R1 iyo DeepSeekMath si ay u soo saaraan silsilad-fikrad dheer oo fikrado ah iyadoo la adeegsanayo abaal-marinnada saxda ah ee qaanuunka ku salaysan ee dhibaatooyinka xisaabta
Qaababka jiilka code-ka-habaynta wanagsan halkaaso xal kasta oo muunad ah lagu qiimeeyay haddi ay ku gudubto imtixaanada cutubka, kooxdana ay caadi tahay in ay doortaan guulaystayaasha
Tubooyinka RLHF-ka furan (tusaale, TRL iyo maktabadaha verl) iyadoo la isticmaalayo GRPO si loo waafajiyo moodooyinka wada sheekaysiga adoon bixinayn shabakad qiime leh
Hagaajinta hab-dhaqanka edbinta ama hab-dhaqanka badbaadada iyadoo la tijaabinayo dhawr jawaabo isla markiiba lana abaalmariyo kuwa abaalgudka qiimaha ugu sarreeya marka loo eego asaagooda
Khatarta & Dariiqyada Ilaalada
Hagaajinta hal bartilmaameed waxay qarin kartaa daciifnimada nidaamka ballaaran.
Kaabayaasha dhaqaalaha iyo dayactirka inta badan waa la dhayalsadaa.
Nabadgelyada iyo daldaloolada u fiirsashada ayaa kori kara marka nidaamyadu noqdaan kuwo aad u adag.
Qorshe Hawleedka Dhaqangelinta
Qeex daahida, tayada, iyo bartilmaameedyada qiimaha ka hor inta aan la hirgelin.
Benchmark marka la eego culeyska dhabta ah iyo xaaladaha xogta.
La socodka qalabka khaladaadka, leexashada, iyo saamaynta isticmaalaha.
U diyaari dib-u-noqoshada iyo dariiqyada jawaab-celinta dhacdada ka hor inta aanad miisaan.
Sii wad Sahaminta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Group Relative Policy Optimization quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Hagaha xiga
Kordhinta Siyaasada Ku Dhaw
Su'aalaha soo noqnoqda
What is Group Relative Policy Optimization?
Hagaajinta Siyaasadda Qaraabada Kooxda (GRPO) waa habka xoojinta-waxbarashada ee habaynta hab-dhaqanka hab-luuqadeed kaas oo ku qiimeeya jawaab kasta oo ka dhan ah koox jawaabaha walaalaha ah isla markiiba, meeshana ka saaraysa isku xidhka qiimaha gaarka ah ee ay isticmaasho PPO. Waxay caan ku noqotay khiyaamada tababarka xudunta u ah moodooyinka sababaynta DeepSeek.
Waa maxay qaybta ugu muhiimsan ee PPO GRPO ma tirtirtaa?
Isbeddelka saxeexa GRPO waxa uu hoos u dhigayaa qiimaha la bartay ee PPO/qaabka dhaleecaynta, kaas oo caadiyan ku saabsan cabbirka siyaasadda, badbaadinta xusuusta la taaban karo iyo xisaabinta.
Sidee GRPO u xisaabisaa faa'iidada dhamaystirka la siiyay?
Dhammaystir kasta faa'iidadiisa waa (abaal-marin-koox celcelis ahaan) / weecasho heerka kooxeed, markaa kooxda jawaabaha isla markiiba waxay u dhaqmaan sida gundhigga.
Qaabkee ayaa GRPO si fiican u yaqaan?
GRPO waxaa soo bandhigay oo caan ku ah DeepSeek, gaar ahaan DeepSeekMath iyo sida matoorka RL ee ka dambeeya moodooyinka sababaynta DeepSeek-R1.
Doorkee ayuu ereyga KL-kala-duwanaanshaha ku leeyahay GRPO?
Ciqaabta KL ee ka dhanka ah tixraaca (sida caadiga ah ka hor-RL) moodelku waxa uu joogtaa tababarka si siyaasaddu u horumarto iyada oo aan la dumin ama u leexin wax soo saar xumaaday.
Waa maxay sababta GRPO ay si gaar ah ugu soo jiidanayso moodooyinka sababaynta tababarka ee xisaabta ama koodka?
Samaynta xalal badan iyo ku-sixinta iyaga oo si toos ah u hubinaya saxnaanta tooska ah ee lammaaneyaasha si nadiif ah faa'iidooyinka caadiga ah ee kooxda GRPO, dhaleecayn looma baahna.