Incamake
Igipimo cyiza cya Optimisiyoneri (ORPO) nuburyo bwiza bwo guhuza neza bwigisha icyitegererezo cyururimi imyitwarire myiza nibyifuzo byabantu mumwanya umwe wamahugurwa. Nibyingenzi kuko isimbuka bisanzwe bisanzwe ibihembo byigihembo nicyitegererezo, bigatuma guhuza bihendutse kandi byoroshye.
Igipimo cyiza cya Optimisiyonike ni igice cyururimi-AI ikoreshwa mu gusoma, kubyara, gushyira mu byiciro, no guhindura inyandiko n'imvugo ku gipimo.
Kwibira cyane
ORPO, yatangijwe na Hong, Lee, na Thorne mu 2024, ikomatanya kugenzura neza no guhuza ibyifuzo mu ntambwe imwe. Imiyoboro myinshi ihuza ibanza ikora SFT kurugero rwiza, hanyuma ikore uburyo bwa kabiri nka RLHF cyangwa DPO bisaba kopi yahagaritswe yicyitegererezo (reference) hiyongereyeho kubikwa kubiri. ORPO ikuraho moderi yerekana rwose. Igihombo cyayo kongeramo igihano kubisanzwe bikurikira-ikimenyetso: bizamura impinduka icyitegererezo giha igisubizo cyatoranijwe (gikunzwe) mugihe gisunika ibitagenda neza. Kuberako ikoresha ikigereranyo kidasanzwe aho kuba ikinyuranyo gikomeye-gishoboka, igihano kiritonda, nuko rero icyitegererezo cyiga gutanga ibisubizo byiza utibagiwe nibisekuru neza.
Ubushishozi
Igihombo cya ORPO nigihombo cya SFT cross-entropy hiyongereyeho log-sigmoid iremereye yikigereranyo cyibiti bitandukanya ibisubizo byatoranijwe kandi byanze. Impanuka zingana p / (1-p), igipimo rero kigereranya uburyo bishoboka cyane ko icyitegererezo kibona igisubizo cyiza nikibi. Gukoresha ibidashoboka aho kuba bibisi byoroshye bituma itandukaniro ryoroheje, ririnda gukabya gukabya ibimenyetso byangwa bishobora gutesha agaciro icyitegererezo.
Kumenya Odds Ikigereranyo cyo guhitamo
Igipimo cyiza cya Optimisiyoneri (ORPO) nuburyo bwiza bwo guhuza neza bwigisha icyitegererezo cyururimi imyitwarire myiza nibyifuzo byabantu mumwanya umwe wamahugurwa. Nibyingenzi kuko isimbuka bisanzwe bisanzwe ibihembo byigihembo nicyitegererezo, bigatuma guhuza bihendutse kandi byoroshye. Igipimo cyiza cya Optimisiyonike ni igice cyururimi-AI ikoreshwa mu gusoma, kubyara, gushyira mu byiciro, no guhindura inyandiko n'imvugo ku gipimo. Kugirango wubake byimbitse, fata Odds Ratio Preference Optimisation nkicyitegererezo gikora, ntabwo ari ikintu kimwe: gusobanura ibyagezweho, gusobanura ibitekerezo, no gutandukanya ibyo sisitemu ishobora gukora byizewe nibisaba guca imanza zinzobere.
Mu myitozo, amakipe akomeye akoresha Odds Ratio Ibyifuzo bya Optimisiyoneri igishushanyo mbonera, kugarura, no gusubiramo imirongo nka sisitemu imwe y'itumanaho ihuriweho. Bandika ibipimo ngenderwaho byerekana intsinzi, bagerageza kurwanya amakuru afatika hamwe nakazi keza, kandi bagasubiramo bashingiye kubikorwa byagaragaye ko batsinzwe aho gutsinda inshuro imwe. Aha niho imyumvire yubumenyi ihinduka mubushobozi burambye kubicuruzwa, politiki, nibikorwa.
Ururimi rwakazi rushobora kugenda byihuse nta gutamba guhuzagurika. Mugihe kimwe, ibintu bya Hallucinated birashobora kwinjiza bucece raporo, gushyigikira imigendekere, cyangwa ibisubizo byubushakashatsi. Uburyo bukomeye cyane ni uguhuza umuvuduko wikigereranyo hamwe na disipuline yimiyoborere: kuyobora abaderevu, gufata ibimenyetso, gutangaza ibyemezo byicyemezo, no gukomeza kuvugurura uburyo bwo kwirinda nkimyitwarire yicyitegererezo, ibyo abakoresha bategereje, nibisabwa n'amategeko bigenda bihinduka.
Ingaruka z'Ingamba
Ururimi rwakazi rushobora kugenda byihuse nta gutamba guhuzagurika.
Ururimi rwakazi rushobora kugenda byihuse nta gutamba guhuzagurika. Mubikorwa byujuje ubuziranenge, ibi bihindurwa mumategeko agenga imikorere, imipaka nyirubwite, hamwe n'imihango yo gusubiramo kenshi kugirango amakipe ashobore kwigirira ikizere aho gupima ibidasobanutse.
Yagura uburyo bwindimi nuburyo bwo gutumanaho.
Yagura uburyo bwindimi nuburyo bwo gutumanaho. Mubikorwa byujuje ubuziranenge, ibi bihindurwa mumategeko agenga imikorere, imipaka nyirubwite, hamwe n'imihango yo gusubiramo kenshi kugirango amakipe ashobore kwigirira ikizere aho gupima ibidasobanutse.
Amakipe arashobora kumara umwanya munini murubanza mugihe automatike ikora gusubiramo.
Amakipe arashobora kumara umwanya munini murubanza mugihe automatike ikora gusubiramo. Mubikorwa byujuje ubuziranenge, ibi bihindurwa mumategeko agenga imikorere, imipaka nyirubwite, hamwe n'imihango yo gusubiramo kenshi kugirango amakipe ashobore kwigirira ikizere aho gupima ibidasobanutse.
Gushyira mu bikorwa Isi
Kuringaniza neza-isoko ya 7B yo kuganira kubintu byombi utabanje gupakira kopi ya kabiri yerekana, kugabanya ububiko bwa GPU
Gutangira guhuza umufasha-wunganira umufasha kugirango ahitemo ikinyabupfura, kuri politiki ibisubizo mumahugurwa amwe akora aho kuba SFT-hanyuma-DPO
Abashakashatsi bagereranya ORPO na DPO kuri dataset imwe kugirango berekane guhuza kugereranya na compte yo hasi
Guhuza icyitegererezo cyibanze kumurongo wihariye (urugero, gutegura amategeko) aho ingero nziza kandi mbi zombi ziboneka ariko ingengo yimishinga ntangarugero ntabwo
Uburyo bwo Gushyira mu bikorwa
Ikigereranyo cya Odds Gukunda Optimisation mubikorwa
Kuringaniza neza-gufungura isoko-7B yerekana uburyo bwo kuganira utabanje gukuramo kopi ya kabiri yerekana, kugabanya ububiko bwa GPU.
Kuringaniza neza-isoko ya 7B yo kuganira kubintu byombi utabanje gupakira kopi ya kabiri yerekana, kugabanya ububiko bwa GPU Amakipe ubusanzwe abona ibisubizo byiza iyo asobanuye ibipimo byujuje ubuziranenge imbere, agakomeza inzira yo kuzamura abantu kubibazo, kandi akurikirana inyungu zibyara umusaruro hamwe nibiciro byamakosa mugihe runaka.
Ikigereranyo cya Odds Gukunda Optimisation mubikorwa
Gutangira guhuza umukiriya-wunganira umufasha kugirango ahitemo ikinyabupfura, kuri politiki ibisubizo mumahugurwa amwe akora aho kuba SFT-hanyuma-DPO.
Gutangira guhuza umufasha-wunganira umukiriya kugirango ahitemo ikinyabupfura, kuri politiki ibisubizo mumahugurwa amwe aho kuba SFT-hanyuma-DPO Amakipe ubusanzwe abona ibisubizo byiza iyo asobanuye ibipimo ngenderwaho byimbere, agakomeza inzira yo kuzamura abantu kubibazo byimbere, kandi akurikirana inyungu zibyara umusaruro nibiciro byamakosa mugihe.
Ikigereranyo cya Odds Gukunda Optimisation mubikorwa
Abashakashatsi bagereranya ORPO na DPO kuri dataset imwe kugirango berekane guhuza kugereranya na compte yo hasi.
Abashakashatsi bagereranya ORPO na DPO kuri dataset imwe kugirango berekane guhuza kugereranywa namakipe yo hasi ya compte mubisanzwe babona umusaruro ushimishije mugihe basobanuye ibipimo ngenderwaho byimbere, bagakomeza inzira yo kuzamura abantu kubibazo, kandi bagakurikirana inyungu zibyara umusaruro nibiciro byamakosa mugihe runaka.
Ikigereranyo cya Odds Gukunda Optimisation mubikorwa
Guhuza icyitegererezo cyibanze kumurongo wihariye (urugero, gutegura amategeko) aho ingero nziza kandi mbi zombi ziboneka ariko ingengo yimishinga ntangarugero ntabwo.
Guhuza icyitegererezo cyibanze kumurongo wihariye (urugero, gutegura amategeko) aho ingero nziza kandi mbi ziboneka ariko ingengo yimishinga ntangarugero ntabwo Amakipe asanzwe abona ibisubizo byiza iyo asobanuye ibipimo ngenderwaho byimbere, agakomeza inzira yo kuzamura abantu kubibazo byimbere, kandi akurikirana inyungu zibyara umusaruro nibiciro byigihe.
Ingaruka & Kurinda
Ibintu bifatika bishobora kwinjiza bucece raporo, gushyigikira imigendekere, cyangwa ibisubizo byubushakashatsi.
Kwihuta byihuse birashobora gukora ibisubizo bidahuye mubisabwa bisa.
Ibyanditswe byumvikana birashobora kugaragara niba kugenzura kugenzura ari ntege.
Igishushanyo mbonera
Sobanura imiterere isohoka, amajwi, hamwe nubuziranenge mbere yo gutangira.
Sobanura imiterere isohoka, amajwi, hamwe nubuziranenge mbere yo gutangira. Fata buri ntambwe nk irembo ryibimenyetso: niba ibipimo bitujujwe, hagarika kuzenguruka, funga icyuho, hanyuma noneho wagure imikoreshereze.
Ibisubizo byibanze hamwe nisoko yizewe igihe cyose ukuri kwingirakamaro.
Ibisubizo byibanze hamwe nisoko yizewe igihe cyose ukuri kwingirakamaro. Fata buri ntambwe nk irembo ryibimenyetso: niba ibipimo bitujujwe, hagarika kuzenguruka, funga icyuho, hanyuma noneho wagure imikoreshereze.
Komeza kugenzura abantu kugenzura ibisubizo byinshi.
Komeza kugenzura abantu kugenzura ibisubizo byinshi. Fata buri ntambwe nk irembo ryibimenyetso: niba ibipimo bitujujwe, hagarika kuzenguruka, funga icyuho, hanyuma noneho wagure imikoreshereze.
Kurikirana uburyo bwo kunanirwa no kongera imyitozo cyangwa akazi gahoraho.
Kurikirana uburyo bwo kunanirwa no kongera imyitozo cyangwa akazi gahoraho. Fata buri ntambwe nk irembo ryibimenyetso: niba ibipimo bitujujwe, hagarika kuzenguruka, funga icyuho, hanyuma noneho wagure imikoreshereze.