Nhungamiro yehunyanzvi

RAG Evaluation Metrics uye RAGAS

RAG ongororo inowana kudzoreredza-yakawedzera chizvarwa sisitimu zvakasiyana pakutora uye pachizvarwa, kazhinji nemametric mana: kutendeka, kukosha kwemhinduro, kurongeka kwemamiriro ezvinhu uye kurangarira mamiriro.

Teerera kugwaro iriAI-yakarondedzerwa · ~3 min

Pa peji ino3 min verenga
  1. Pfupiso
  2. Kudzika Kwakadzika
  3. Kuitwa muNyika yeChokwadi
  4. Ramangwana reRAG Evaluation Metrics uye RAGAS
  5. Kuzvishandisa
  6. Ramba Uchiongorora
  7. Mibvunzo inowanzo bvunzwa

Pfupiso

RAGAS ndeye yakavhurika-sosi sisitimu inokokorodza aya metrics, zvakanyanya kushandisa LLM semutongi. Zvibodzwa zvakaparadzana zvine basa nekuti zvinoratidza kana mhinduro yakaipa yabva mukutora ndima dzisiridzo kana kushandisa zvisirizvo.

Kudzika Kwakadzika

RAG system inogona kukundikana munzvimbo mbiri: kudzoreredza kunogona kutora ndima dzisiridzo, kana chizvarwa chinogona kufuratira kana kukanganisa dzakakodzera. Chiyero chimwe chete chemhando yepamusoro chinovanza zvakaitika, saka ongororo yeRAG inotsemura pombi kuita zvikamu zvinoyerwa. RAGAS, yakavhurika-sosi sisitimu yakaunzwa mubepa ra2023 naEs nevamwe vaaishanda navo, yakasimudzira seti yepakati metrics inovhara mahafu ese. Kutendeseka kunobvunza kana chirevo chiri chose chiri mumhinduro chichitsigirwa namashoko akapoteredza akadzorerwa; inobata kuona kunyange kana mhinduro ichiita seine musoro. Mhinduro inokoshera inobvunza kuti mhinduro yacho inonyatso gadzirisa mubvunzo here, kuranga mhinduro dzinonzvenga kana dzakapetwa. Context precision inobvunza kana machunks akadzoserwa akakosha akaiswa padhuze nepamusoro pane kuvigwa pakati peruzha. Context recall inobvunza kana mamiriro akadzoserwa aine zvese zvinodiwa kuburitsa mhinduro yereferensi, saka zvinoda mhinduro yechokwadi pamubvunzo wega wega webvunzo. Zvizhinji zvezvibodzwa izvi zvinoverengerwa neLLM ichiita semutongi, yakaputswa kuita nhanho diki. Nekuda kwekutendeka, mutongi anobvisa zvirevo zvega kubva mumhinduro uye anotarisa chimwe nechimwe maererano nezviri mukati; chibodzwa ndicho chikamu chinotsigirwa. Nekukoshera kwemhinduro, RAGAS inogadzira mibvunzo iyo mhinduro yaizopindura uye inoienzanisa nemubvunzo wekutanga uchishandisa zvakamisikidzwa. Mazano akafanana anoonekwa mune mamwe maturusi, akadai seTruLens RAG triad (context relevance, groundedness and answer relevance) uye DeepEval. Mafungiro maviri asiri iwo akajairika. Chekutanga, kutendeka hakurevi kururamisa: mhinduro inogona kudzokorora zvinyorwa zvekare kana zvisiri izvo. Chechipiri, LLM-mutongi zvibodzwa hachisi chokwadi chepasi. Ivo vanosiyana nemuenzaniso wemutongi uye nekukasira, saka zvikwata zvinovaona-zvitarise zvichipesana nemavara evanhu uye enzanisa masisitimu ari pasi pezvirongwa zvakafanana pane kubata zvibodzwa senge 0.85 segiredhi rakakwana.

Kuitwa muNyika yeChokwadi

Mubatsiri wezvemitemo webhangi anoita zvibodzwa 0.95 pakurangarira kwemamiriro ezvinhu asi 0.70 pakutendeka, izvo zvinoudza kuti kudzosa kwechikwata kwakanaka uye modhi iri kuwedzera zvinhu zvisina kutsigirwa, saka vanosimbisa kukurumidza kwete kuvaka patsva index.

Chikwata chetsigiro chinoenzanisa mamodheru maviri ekumisikidza pane imwecheteyo mibvunzo mazana maviri ebvunzo seti uye inochengeta iyo ine yepamusoro yekuyeuka mamiriro uye nemazvo.

Chishandiso cheruzivo rwechipatara chinomhanyisa kutendeka chibodzwa pamuenzaniso wemazuva ese wemibvunzo chaiyo uye mireza mhinduro nezvisina kutsigirwa zveongororo yevanhu isati yapararira.

Kutanga kunowedzera reranker mushure mekuona yakakwira mamiriro ekurangarira asi yakaderera mamiriro, zvichireva kuti chunks chaidzo dzakatorwa asi dzakaiswa pazasi zvisina basa.

Ramangwana reRAG Evaluation Metrics uye RAGAS

Ongororo yeRAG iri kusimuka kubva pane dzimwe nguva mishumo isina mhepo kuenda kune inoenderera mberi cheki, uko sampuli yekugadzira traffic inopihwa uye kudzoreredzwa kunokonzeresa ziviso. Maframeworks anga achiwedzera mametrics ehurukuro dzakawanda, vamiririri uye mashandisirwo ezvishandiso, uye tsananguro yemetric inoshanduka pakati peshanduro dzeraibhurari, saka pinning shanduro zvine basa kune mhedzisiro yakafanana. Nyaya huru yakavhurika ndeyekuvimbika kwemutongi: vatongi veLLM vanogona kuratidza kusarura sekufarira mhinduro dzakareba, uye kubvumirana nevanoongorora vanhu kunosiyana nedomasi. Kugadzirisa vatongi vachipesana nemaseti madiki-akanyorwa nevanhu uye kuabatanidza nemametriki ekutsvagisa angangoramba ari maitiro akanaka.

Kuzvishandisa

Nei zvichikosha

  • Mutengo uye bhajeti. Zvisarudzo zvezvivakwa zvinotyaira kuita uye mutengo wekushandisa kwemakore.
  • Sarudzo dzakajeka. Dzidzo yehunyanzvi inobatsira zvikwata kusarudza murwi wakakodzera, kwete iwo mutsva chete.
  • Kudzora kwemhando yepamusoro. Sarudzo dzeinjiniya dziri nani dzinoderedza zviitiko zvekuvimbika mukugadzira.

Chenjera

  • Kugadzirisa imwe bhenji kunogona kuvanza yakafara system kushaya simba.
  • Infrastructure uye mari yekugadzirisa inowanzotarisirwa pasi.
  • Chengetedzo uye kucherechedzwa mapundu anogona kukura sezvo masisitimu anowedzera kuoma.

Matanho anotevera

  1. Tsanangura latency, mhando, uye mutengo zvinangwa usati waitwa.
  2. Benchmark pasi pechokwadi mutoro uye data mamiriro.
  3. Chishandiso chekutarisa zvikanganiso, kudonha, uye mushandisi maitiro.
  4. Gadzirira nzira dzekudzosera kumashure uye dzezviitiko usati wawedzera.

Ramba Uchiongorora

Tsamba yemahara

Tora iyo yemazuva ese AI pfupiso

Nhatu dzakasimbiswa dzeAI nyaya mangwanani ega ega pakati pevhiki, dzakanyorwa neChirungu chakajeka. Yemahara zvachose, hapana ads.

Imwe email zuva rega rega revhiki. Bvisa kunyoresa kamwe chete. Hatimbotengese kana kugovera kero yako.

Zviedze iwe pachako

Take the RAG Evaluation Metrics and RAGAS quiz

Mhinduro yepakarepo pamhinduro yega yega, uye chitupa chinogovaniswa chine ID inobatika kana wapedza kosi.

Tanga mibvunzo

Tsigira dzidzo yemahara yeAI. AI Understanding ndeye 501(c)(3) isingabatsiri - hapana ads, hapana paywall, zvachose. Ita chipo

Mibvunzo inowanzo bvunzwa

Chii chinonzi RAG Evaluation Metrics uye RAGAS?

RAG ongororo inowana kudzoreredza-yakawedzera chizvarwa sisitimu zvakasiyana pakutora uye pachizvarwa, kazhinji nemametric mana: kutendeka, kukosha kwemhinduro, kurongeka kwemamiriro ezvinhu uye kurangarira mamiriro. RAGAS ndeye yakavhurika-sosi sisitimu inokokorodza aya metrics, zvakanyanya kushandisa LLM semutongi. Zvibodzwa zvakaparadzana zvine basa nekuti zvinoratidza kana mhinduro yakaipa yabva mukutora ndima dzisiridzo kana kushandisa zvisirizvo.

Ndeipi metric inotarisa kana kuda kwese mumhinduro kunotsigirwa neyakadzoserwa mamiriro?

Kuvimbika kunoyera chidimbu chemhinduro dzezvirevo zvinotsigirwa nezviri mukati, izvo zvinonangana nekufungidzira.

Ndeipi yeaya metric inoda mhinduro yechokwadi yepasi pamubvunzo wega wega webvunzo?

Context recall inotarisa kana chirevo chakadzoserwa chine izvo zvinodiwa pamhinduro yereferensi, saka referensi inofanira kuvapo.

Chii chinoyeresa kurongeka kwechirevo?

Context chaiyo mibairo inoisa anobatsira chunks kumusoro pane chinzvimbo pane kuviga pakati pezvisina basa.

RAGAS inofungidzira sei kukosha kwemhinduro?

Kana mibvunzo inogadzirwa kubva mumhinduro yakafanana nemubvunzo chaiwo, mhinduro iri pamusoro; embeddings inoyera kufanana ikoko.

Sisitimu inoratidza kurangarira kwemamiriro ekunze asi yakaderera mamiriro chaiwo. Chii chinonyanya kugadzirisa?

Iwo machunks ekurudyi ari kutorwa asi akaiswa zvisizvo, rinova dambudziko renzvimbo iyo reranker inowanzogadzirisa.