InoteveraGaidhi rinotevera
Gwaro Revabereki uye Diki-ku-Kukuru Kudzosa
Tekinoroji
Nhungamiro yehunyanzvi
Contextual kudzoreredza inzira yeRAG inofanogadzirira tsananguro ipfupi, gwaro-inoziva kune imwe neimwe chunk isati yaiswa uye yakarongedzerwa yekutsvaga keyword, saka chunk ichiri kuita zvine musoro kana ichiverengwa yega.
Teerera kugwaro iriAI-yakarondedzerwa · ~3 min
Izvo zvine basa nekuti chunks anowanzo rasikirwa neruzivo rwakakosha kana akapatsanurwa kubva mugwaro ravo, sekuti ndeipi kambani, chigadzirwa kana nguva yavanoreva, uye kurasikirwa ikoko kunoita kuti mutoreri atadze nzira chaiyo.
Standard RAG inotsemura magwaro kuita chunks, inomisikidza chunk yega yega, uye inotora chunks zvakanyanya kufanana nemubvunzo. Dambudziko nderekuti kupatsanurana kunobvisa mamiriro. Chunk inoti "mari yekambani yakakura ne3%" haitaure kuti ndeipi kambani kana kuti inguva ipi, saka hapana kutsvaga semantic kana kutsvaga kwemazwi akakosha kunogona kuenderana nemubvunzo unovatumidza. Anthropic yakatsanangura kudzoserwa kwemamiriro ekunze munaGunyana 2024. Pachunk yega yega, modhi yemutauro inopiwa gwaro rose uye chunk, uye inokumbirwa kunyora muchidimbu mamiriro (kazhinji akatenderedza 50 kusvika ku100 tokens) iyo inoisa chunk mukati megwaro. Chirevo chacho chakagadzirirwa kune chunk. Mavara akasanganiswa anobva ashandiswa muma indexes maviri: indekisi yekumisikidza (contextual embeddings) uye BM25 keyword index (contextual BM25). Panguva yekubvunza, mibairo kubva kune zvese inosanganiswa, uye nesarudzo reranker inoodha vakwikwidzi vepamusoro. MuAnthropic zvakashumwa zviedzo pamaseti akati wandei, kumisikidzwa kwemamiriro ezvinhu pamwe nemamiriro ezvinhu BM25 yakaderedza mwero wepamusoro-makumi maviri wekutadza kudzoreredza nezvikamu makumi mana nemapfumbamwe muzana zvichienzaniswa nekumisikidzwa kwakajairwa, uye kuwedzera kudzoreredza kwakaunza kudzikiswa kusvika pamakumi matanhatu nenomwe muzana. Mhedzisiro pane yako pachako data inogona kusiyana, saka zvakakodzera kuyerwa. Mutengo uri panguva yekunongedza: imwe modhi yekufona pane chunk, imwe neimwe inosanganisira yakazara gwaro. Kurumidza caching kunoita kuti izvi zvidhure zvakanyanya, nekuti gwaro rakachengetwa kamwe chete uye rinoshandiswazve pamativi aro ese. Mafungiro asina kunaka ekudzivirira: iyo yakawedzerwa mamiriro anoshandiswa kudzoreredza, uye iwe unogona kusarudza kana kuiratidza kune jenareta. Izvo hazvitsive zvakanaka chunking, uye kune mabhesi ezivo madiki anoenderana neiyo modhi yemamiriro hwindo, kutumira zvese zvakananga kungave nyore pane chero pombi yekutora.
Chikamu chinobva kumushumo wekota chinoverenga chete "Revenue yakawedzera 3% pamusoro pekota yapfuura"; kudzoreredza kwemamiriro ezvinhu kunofanogadzirira kuti inobva kune imwe kambani Q2 faira, saka kutsvaga kutumidza iyo kambani nekota inoiwana.
Mubhuku revashandi, chunk ichiti "Izvi zvinoshanda mushure memazuva makumi mapfumbamwe" inowana prefix inotsanangura kuti inobva muchikamu chekukodzera basa kure.
A software zvinyorwa mubatsiri anowedzera mamiriro achiona kuti gadziriso chunk ndeye vhezheni 3 yeAPI, ichideredza kuvhiringidzika nemavara akafanana kubva muvhezheni 2.
Ruzivo rwerutsigiro rwechivakashure chekugadzirisa matambudziko nemhando yechigadzirwa yavanovhara, saka mibvunzo nezve imwe modhi inomira kutora matanho eimwe.
Contextual kudzoreredza kunoratidza maitiro akakura ekushandisa yakawanda computation panguva ye indexing kuita kuti nguva yemubvunzo ivimbike. Sezvo mafoni emodhi achichipa uye caching inova yakajairwa, kuwedzera mamiriro kune chunks kungangoita nhanho yekusarudzika mumapaipi mazhinji eRAG. Yakareba mahwindo mahwindo anoderedza kudiwa kwekudzoserwa muzvikamu zvidiki asi kwete muhombe, saka matekiniki akadai anofanirwa kugara akakodzera kune mabhesi makuru eruzivo. Tarisira kusanganisa kwakawanda nekusefa kwemetadata, magirafu ezivo uye kunonoka-kudzokorodza kwekudyidzana, nezvikwata zvinosarudza zvichibva pakuyeukwa kwakayerwa kwete fashoni.
Tsamba yemahara
Nhatu dzakasimbiswa dzeAI nyaya mangwanani ega ega pakati pevhiki, dzakanyorwa neChirungu chakajeka. Yemahara zvachose, hapana ads.
Imwe email zuva rega rega revhiki. Bvisa kunyoresa kamwe chete. Hatimbotengese kana kugovera kero yako.
Zviedze iwe pachako
Mhinduro yepakarepo pamhinduro yega yega, uye chitupa chinogovaniswa chine ID inobatika kana wapedza kosi.
Tsigira dzidzo yemahara yeAI. AI Understanding ndeye 501(c)(3) isingabatsiri - hapana ads, hapana paywall, zvachose. Ita chipo
Contextual kudzoreredza inzira yeRAG inofanogadzirira tsananguro ipfupi, gwaro-inoziva kune imwe neimwe chunk isati yaiswa uye yakarongedzerwa yekutsvaga keyword, saka chunk ichiri kuita zvine musoro kana ichiverengwa yega. Izvo zvine basa nekuti chunks anowanzo rasikirwa neruzivo rwakakosha kana akapatsanurwa kubva mugwaro ravo, sekuti ndeipi kambani, chigadzirwa kana nguva yavanoreva, uye kurasikirwa ikoko kunoita kuti mutoreri atadze nzira chaiyo.
Kupatsanura kunobvisa mamiriro akatenderedza, saka chunk yakasarudzika inogona kusataura chinhu kana nguva iyo mubvunzo unobvunza nezvayo.
Iyo modhi inopiwa iyo yakazara gwaro uye chunk uye inodzosera pfupi mamiriro ayo anoisa chunk mukati megwaro.
Iyo nzira inoshandisa embeddings yemamiriro uye mamiriro BM25, yobva yabatanidza mhedzisiro yavo.
Anthropic yakataura nezve kudzikisira kwe49 muzana, ichikwira kusvika pamakumi matanhatu nenomwe muzana nekugadzirisa zvakare. Migumisiro pane imwe data inogona kusiyana.
Imwe neimwe chunk call inosanganisira iyo yakazara gwaro; caching iyo yakagovaniswa prefix inodzivirira kubhadhara yakazara mutengo kakawanda.
Ramba uchidzidza
Mamwe madhairekitori akasarudzirwa nyaya iyi
InoteveraGaidhi rinotevera
Gwaro Revabereki uye Diki-ku-Kukuru Kudzosa
Tekinoroji