InoteveraGaidhi rinotevera
LLM Eval Frameworks: Promptfoo and DeepEval
Tekinoroji
Nhungamiro yehunyanzvi
Iyo LLM eval dataset seti yakadzivirirwa yezvakaiswa zvinogoneka, imwe neimwe yakapetwa nemhinduro yegoridhe, mareferensi echokwadi kana rubriki yegiredhi, yaunomhanyisa application yako kuyera kunaka uye kubata regressions.
Teerera kugwaro iriAI-yakarondedzerwa · ~3 min
Izvo zvine basa nekuti shanduko yekukasira, modhi kana kudzoreredza inogona kuvandudza imwe maitiro uku ichityora imwe. Pasina yakagadziriswa bvunzo seti, zvikwata zvinopedzisira zvichitonga shanduko nemienzaniso mishoma yakasarudzwa nemaoko.
Tanga nekushandisa chaiko. Marogi, matikiti ekutsigira, mibvunzo yekutsvaga uye kubvunzurudza kwevashandisi kunoratidza izvo vanhu vanonyatso kubvunza, kusanganisira iwo akashata zvirevo izvo zvakagadzira mienzaniso inopotsa. Usati watanga, ita kuti nyanzvi dzedomasi dzinyore zvinopinda. Iwe unogona kuwedzera makesi ekugadzira anogadzirwa neLLM, chero munhu achiaongorora uye nekubvisa zvinhu zvisingaite kana zvakapetwa. Zvadaro, tsanangura kuti ndezvipi zvinorehwa nechiitiko chimwe nechimwe. Mamwe mabasa ane mhinduro imwe yechokwadi, sekuronga, kuburitsa kana kutarisa kwechokwadi. Kune izvi, chengetedza mhinduro yegoridhe iyo inogona kufananidzwa chaizvo kana mushure mechiedza normalization. Pamabasa akazaruka, nyora rubriki yemaitiro chaiwo, anotarisika, senge 'taura zuva rekupedzisira rekudzosera mari' kana 'hakurudzire mukwikwidzi'. Mitemo isina kujeka yakaita se'inobatsira' inotungamira kugiredhi isingaenderane, angave giredhi ari munhu kana mutongi modhi. Wobva wavhara huwandu hwemapupu chaiwo nemaune. Tag makesi nechinangwa, kuomerwa, mutauro uye mhando yemushandisi, uye tarisa kuti mapoka akakosha anomiririrwa. Wedzera zvichemo zvemupendero uye zveanopikisa: - mibvunzo isina kujeka - zvikumbiro zvine ruzivo rusipo - mapindiro akareba - zvikumbiro zvekunze-zvikumbiro zvinofanirwa kurambwa - kuedza-jekiseni nekukurumidza Chengetedza chidimbu chekudzoka chakaitwa nekutadza kwekare. Saizi yakatarwa kune sarudzo yaanofanira kutsigira. Mamwe gumi nemaviri akanyatsosarudzwa makesi anogona kuratidza matambudziko makuru kutanga. Kuziva misiyano midiki nekuvimbika kunotora zvakawanda. Nemazana zana ekupasa/kufoira uye mwero wekupasa uri pedyo ne80 muzana, muganho wekukanganisa ungangowedzera kana kubvisa 8 muzana mapoinzi pa95 muzana chivimbo. Kuvandudzwa kwemapoinzi maviri hazvizivikanwe kubva kune ruzha pahukuru ihwohwo. Mhan'ara mibairo yechidimbu chega chega, kwete chete pakati peavhareji. Kukanganisa kwakajairika kubata dataset sekupedza. Zvigadzirwa nevashandisi zvinoshanduka, saka zorodza kubva kumatanda matsva. Chengetedza chikamu chakavharirwa chausingambopikisa, kuitira kuti zvirevo zvako zvisanyanya kukwana pabvunzo.
Chikwata chetsigiro chinotumira kunze mazana matatu echokwadi matikiti ane data rako pachako abviswa, anonyora gadziriso chaiyo kune yega yega, uye anowana vhezheni yega yega yekukurumidza kupokana navo.
Kupfupisa yemusangano umo mazwi mazhinji ari echokwadi, timu inonyora rubriki: inovhara sarudzo nhatu dzakakosha, haiite zvirevo zvisingatsigirwe, inogara pasi pemazwi zana nemakumi mashanu.
Chikwata chinowedzera makesi ekumucheto: yekupinza isina chinhu, mubvunzo muchiSpanish, gwaro rine kukurumidza-jekiseni kuyedza, uye kunze-kwe-scope chikumbiro mubatsiri anofanira kuramba.
Kana mushandisi akataura mhinduro yakaipa mukugadzirwa, nyaya yacho inopinda mudhata nemhinduro chaiyo, saka gadziriso inoramba ichiyedzwa kubva ipapo zvichienda mberi.
Eval tooling yakura nekukurumidza. Vhura-sosi masisitimu uye mapuratifomu akabatwa zvino anotonga madhaseti, mhanyisa magiredhi uye tarisa mhinduro nekufamba kwenguva. Synthetic data yekugadzira ingangoramba ichinatsiridza, izvo zvinoita kuti zvisingawanzo kesi zvidhure kuvhara. Izvo hazvigone kutsiva izvo zvemushandisi chaiwo sechibatiso chezvine basa. Sezvo maapplication achiwedzera agency, eval seti inowedzera inosanganisira akawanda-nhanho mabasa akaiswa pamagiredhi pamhedzisiro yekupedzisira uye matanho anotorwa munzira. Chirango chepakati chinoshanduka zvishoma nezvishoma: tsanangura budiriro zvakajeka, vhara mabiko chaiwo, shandisa makesi akakwana kuvimba nenhamba, uye zorodza seti nguva nenguva kubva mukugadzirwa.
Tsamba yemahara
Nhatu dzakasimbiswa dzeAI nyaya mangwanani ega ega pakati pevhiki, dzakanyorwa neChirungu chakajeka. Yemahara zvachose, hapana ads.
Imwe email zuva rega rega revhiki. Bvisa kunyoresa kamwe chete. Hatimbotengese kana kugovera kero yako.
Zviedze iwe pachako
Mhinduro yepakarepo pamhinduro yega yega, uye chitupa chinogovaniswa chine ID inobatika kana wapedza kosi.
Tsigira dzidzo yemahara yeAI. AI Understanding ndeye 501(c)(3) isingabatsiri - hapana ads, hapana paywall, zvachose. Ita chipo
Iyo LLM eval dataset seti yakadzivirirwa yezvakaiswa zvinogoneka, imwe neimwe yakapetwa nemhinduro yegoridhe, mareferensi echokwadi kana rubriki yegiredhi, yaunomhanyisa application yako kuyera kunaka uye kubata regressions. Izvo zvine basa nekuti shanduko yekukasira, modhi kana kudzoreredza inogona kuvandudza imwe maitiro uku ichityora imwe. Pasina yakagadziriswa bvunzo seti, zvikwata zvinopedzisira zvichitonga shanduko nemienzaniso mishoma yakasarudzwa nemaoko.
Kushandisa chaiko kunobata izvo vanhu vanonyatso kubvunza, kusanganisira kushatisa mazwi. Synthetic kesi ndeye yakaongororwa yekuwedzera, kwete yekutsiva.
Zvakavhurika-zvinobuda zvakaita sepfupiso hazvigone kuenderana chaizvo. Chaiyo, chinotarisika maitiro regai magiredhi avatonge nguva dzose.
Chiyero chakanaka chakananga uye chinotarisika. Mitemo isina kujeka senge 'inobatsira' inotungamira mukusaenderana magiredhi nevanhu uye vanotonga mhando zvakafanana.
Pa95 muzana chivimbo muganho ungangoita mapoinzi masere pahukuru uhu. Misiyano midiki pakati peshanduro haigone kuvimbwa pasina dzimwe nyaya.
Kuwedzera kutadza kwese kwakagadziriswa kudataset kunoreva kuti shanduko inotevera haigone kuunza chinyararire iyo bug.
Ramba uchidzidza
Mamwe madhairekitori akasarudzirwa nyaya iyi
InoteveraGaidhi rinotevera
LLM Eval Frameworks: Promptfoo and DeepEval
Tekinoroji