InoteveraGaidhi rinotevera
ROUGE uye BLEU Evaluation Metrics
Tekinoroji
Nhungamiro yehunyanzvi
RAG ongororo inowana kudzoreredza-yakawedzera chizvarwa sisitimu zvakasiyana pakutora uye pachizvarwa, kazhinji nemametric mana: kutendeka, kukosha kwemhinduro, kurongeka kwemamiriro ezvinhu uye kurangarira mamiriro.
Teerera kugwaro iriAI-yakarondedzerwa · ~3 min
RAGAS ndeye yakavhurika-sosi sisitimu inokokorodza aya metrics, zvakanyanya kushandisa LLM semutongi. Zvibodzwa zvakaparadzana zvine basa nekuti zvinoratidza kana mhinduro yakaipa yabva mukutora ndima dzisiridzo kana kushandisa zvisirizvo.
RAG system inogona kukundikana munzvimbo mbiri: kudzoreredza kunogona kutora ndima dzisiridzo, kana chizvarwa chinogona kufuratira kana kukanganisa dzakakodzera. Chiyero chimwe chete chemhando yepamusoro chinovanza zvakaitika, saka ongororo yeRAG inotsemura pombi kuita zvikamu zvinoyerwa. RAGAS, yakavhurika-sosi sisitimu yakaunzwa mubepa ra2023 naEs nevamwe vaaishanda navo, yakasimudzira seti yepakati metrics inovhara mahafu ese. Kutendeseka kunobvunza kana chirevo chiri chose chiri mumhinduro chichitsigirwa namashoko akapoteredza akadzorerwa; inobata kuona kunyange kana mhinduro ichiita seine musoro. Mhinduro inokoshera inobvunza kuti mhinduro yacho inonyatso gadzirisa mubvunzo here, kuranga mhinduro dzinonzvenga kana dzakapetwa. Context precision inobvunza kana machunks akadzoserwa akakosha akaiswa padhuze nepamusoro pane kuvigwa pakati peruzha. Context recall inobvunza kana mamiriro akadzoserwa aine zvese zvinodiwa kuburitsa mhinduro yereferensi, saka zvinoda mhinduro yechokwadi pamubvunzo wega wega webvunzo. Zvizhinji zvezvibodzwa izvi zvinoverengerwa neLLM ichiita semutongi, yakaputswa kuita nhanho diki. Nekuda kwekutendeka, mutongi anobvisa zvirevo zvega kubva mumhinduro uye anotarisa chimwe nechimwe maererano nezviri mukati; chibodzwa ndicho chikamu chinotsigirwa. Nekukoshera kwemhinduro, RAGAS inogadzira mibvunzo iyo mhinduro yaizopindura uye inoienzanisa nemubvunzo wekutanga uchishandisa zvakamisikidzwa. Mazano akafanana anoonekwa mune mamwe maturusi, akadai seTruLens RAG triad (context relevance, groundedness and answer relevance) uye DeepEval. Mafungiro maviri asiri iwo akajairika. Chekutanga, kutendeka hakurevi kururamisa: mhinduro inogona kudzokorora zvinyorwa zvekare kana zvisiri izvo. Chechipiri, LLM-mutongi zvibodzwa hachisi chokwadi chepasi. Ivo vanosiyana nemuenzaniso wemutongi uye nekukasira, saka zvikwata zvinovaona-zvitarise zvichipesana nemavara evanhu uye enzanisa masisitimu ari pasi pezvirongwa zvakafanana pane kubata zvibodzwa senge 0.85 segiredhi rakakwana.
Mubatsiri wezvemitemo webhangi anoita zvibodzwa 0.95 pakurangarira kwemamiriro ezvinhu asi 0.70 pakutendeka, izvo zvinoudza kuti kudzosa kwechikwata kwakanaka uye modhi iri kuwedzera zvinhu zvisina kutsigirwa, saka vanosimbisa kukurumidza kwete kuvaka patsva index.
Chikwata chetsigiro chinoenzanisa mamodheru maviri ekumisikidza pane imwecheteyo mibvunzo mazana maviri ebvunzo seti uye inochengeta iyo ine yepamusoro yekuyeuka mamiriro uye nemazvo.
Chishandiso cheruzivo rwechipatara chinomhanyisa kutendeka chibodzwa pamuenzaniso wemazuva ese wemibvunzo chaiyo uye mireza mhinduro nezvisina kutsigirwa zveongororo yevanhu isati yapararira.
Kutanga kunowedzera reranker mushure mekuona yakakwira mamiriro ekurangarira asi yakaderera mamiriro, zvichireva kuti chunks chaidzo dzakatorwa asi dzakaiswa pazasi zvisina basa.
Ongororo yeRAG iri kusimuka kubva pane dzimwe nguva mishumo isina mhepo kuenda kune inoenderera mberi cheki, uko sampuli yekugadzira traffic inopihwa uye kudzoreredzwa kunokonzeresa ziviso. Maframeworks anga achiwedzera mametrics ehurukuro dzakawanda, vamiririri uye mashandisirwo ezvishandiso, uye tsananguro yemetric inoshanduka pakati peshanduro dzeraibhurari, saka pinning shanduro zvine basa kune mhedzisiro yakafanana. Nyaya huru yakavhurika ndeyekuvimbika kwemutongi: vatongi veLLM vanogona kuratidza kusarura sekufarira mhinduro dzakareba, uye kubvumirana nevanoongorora vanhu kunosiyana nedomasi. Kugadzirisa vatongi vachipesana nemaseti madiki-akanyorwa nevanhu uye kuabatanidza nemametriki ekutsvagisa angangoramba ari maitiro akanaka.
Tsamba yemahara
Nhatu dzakasimbiswa dzeAI nyaya mangwanani ega ega pakati pevhiki, dzakanyorwa neChirungu chakajeka. Yemahara zvachose, hapana ads.
Imwe email zuva rega rega revhiki. Bvisa kunyoresa kamwe chete. Hatimbotengese kana kugovera kero yako.
Zviedze iwe pachako
Mhinduro yepakarepo pamhinduro yega yega, uye chitupa chinogovaniswa chine ID inobatika kana wapedza kosi.
Tsigira dzidzo yemahara yeAI. AI Understanding ndeye 501(c)(3) isingabatsiri - hapana ads, hapana paywall, zvachose. Ita chipo
RAG ongororo inowana kudzoreredza-yakawedzera chizvarwa sisitimu zvakasiyana pakutora uye pachizvarwa, kazhinji nemametric mana: kutendeka, kukosha kwemhinduro, kurongeka kwemamiriro ezvinhu uye kurangarira mamiriro. RAGAS ndeye yakavhurika-sosi sisitimu inokokorodza aya metrics, zvakanyanya kushandisa LLM semutongi. Zvibodzwa zvakaparadzana zvine basa nekuti zvinoratidza kana mhinduro yakaipa yabva mukutora ndima dzisiridzo kana kushandisa zvisirizvo.
Kuvimbika kunoyera chidimbu chemhinduro dzezvirevo zvinotsigirwa nezviri mukati, izvo zvinonangana nekufungidzira.
Context recall inotarisa kana chirevo chakadzoserwa chine izvo zvinodiwa pamhinduro yereferensi, saka referensi inofanira kuvapo.
Context chaiyo mibairo inoisa anobatsira chunks kumusoro pane chinzvimbo pane kuviga pakati pezvisina basa.
Kana mibvunzo inogadzirwa kubva mumhinduro yakafanana nemubvunzo chaiwo, mhinduro iri pamusoro; embeddings inoyera kufanana ikoko.
Iwo machunks ekurudyi ari kutorwa asi akaiswa zvisizvo, rinova dambudziko renzvimbo iyo reranker inowanzogadzirisa.
Ramba uchidzidza
Mamwe madhairekitori akasarudzirwa nyaya iyi
InoteveraGaidhi rinotevera
ROUGE uye BLEU Evaluation Metrics
Tekinoroji