Nhungamiro yehunyanzvi

Gwaro Revabereki uye Diki-ku-Kukuru Kudzosa

Kudzosa magwaro evabereki, kunonziwo kutsvagisa kudiki-kusvika-kukuru, kunotsvaga pazvimedu zvidiki, zvemavara chaizvo asi inopa modhi yemutauro chikamu chikuru kana gwaro chikamu chimwe nechimwe chakabva.

Teerera kugwaro iriAI-yakarondedzerwa · ~3 min

Pa peji ino4 min verenga
  1. Pfupiso
  2. Kudzika Kwakadzika
  3. Kuitwa muNyika yeChokwadi
  4. Ramangwana reRugwaro Rwevabereki uye Kudzoserwa Kwediki-ku-Kukuru
  5. Kuzvishandisa
  6. Ramba Uchiongorora
  7. Mibvunzo inowanzo bvunzwa

Pfupiso

Izvo zvine basa nekuti inobvisa yakajairwa chunking kutengeserana-kure: madiki chunks anoenderana nemibvunzo nenzira kwayo, uye mubereki mukuru anopa iyo modhi yakakwana mamiriro ekupindura nemazvo.

Kudzika Kwakadzika

Fixed-size chunking inomanikidza kutengeserana. Zvidiki zvidiki, taura mitsara mishoma, inoburitsa zvakapinza: iyo vector inomiririra imwe pfungwa, saka mubvunzo nezve iyo pfungwa inofanana nayo zvakanyanya. Asi chunk yemitsara miviri haiwanzo kuve nemamiriro akakwana kuti modhi ipindure zvakanaka; inogona kusiya tsananguro yendima nhatu kare kana kusarudzika mumutsara unotevera. Makuru machunks anotakura iwo mameseji, asi mabatirwo awo anodzima mazano akati wandei pamwechete, saka kudzoreredza kunoita kushoma uye ndima yakakodzera inogona kuburitswa nechunk inongori pamusoro-musoro. Kutora kudiki-ku-kukuru kunotsemura mabasa maviri. Iyo sisitimu inonongedza matinji evana ekutsvaga, uye mwana wega wega anochengeta chinongedzo kumubereki mukuru: chikamu, peji, kana gwaro rose. Panguva yekubvunza inowana vana vakanaka, yobva yatarisa kumusoro uye inodzosera vabereki vavo. Kana vana vakati wandei vachigovana mubereki mumwe, mubereki anotumirwa kamwe chete, izvo zvinobvisawo zvakapetwa. LangChain's ParentDocumentRetriever inoshandisa izvi nechitoro chevana uye chitoro chemagwaro chakasiyana chevabereki. LlamaIndex inopa mapatani ane hukama: mutsara-hwindo kudzoreredza, iyo inodzosa mutsara unofananidzwa pamwe nenhamba yakatarwa yemitsara yakavakidzana, uye otomatiki-kubatanidza kudzoreredza, iyo inotsiva akawanda akadzoserwa machunks eshizha neakagovaniswa mubereki node kana zvakakwana zvawo zvakaenzana. Maitiro anowanzo kurova yakagadziriswa-saizi chunking pamagwaro akarongwa senge manyoro, zvibvumirano, marongero uye mabhuku ezvinyorwa, uko zvinoreva zvinoenderana nechikamu chakatenderedza. Zvinobatsira zvishoma kana magwaro atove mapfupi, kana kana vabereki vakakura zvekuti vashoma vavo vanofashukira hwindo remukati kana kuviga mhinduro mumagwaro asina basa. Imwe pfungwa isiriyo ndeyekuti uku kuri kungoshandisa machunks makuru. Haisi: iyo yekudzoreredza unit uye chizvarwa yuniti yakasiana nemaune saizi, saka unoramba uchinyatsoenderana pasina kufa nenzara muenzaniso wemamiriro.

Kuitwa muNyika yeChokwadi

Iyo yemukati IT inobatsira bot inonongedza nhanho yega yega yekugadzirisa senge yakaparadzana mwana chunk, asi kana nhanho ichienderana inodzosera maitiro ese, saka modhi haiudze vashandisi kuti vaite nhanho 4 pasina nhanho 1 kusvika 3.

Chishandiso chekutsvagisa zviri pamutemo chinoenderana nechikamu chimwe chete chenguva dzezviziviso zvekugumisa, chobva chapfuudza modhi yacho chikamu chekondirakiti yakazara, kusanganisira tsananguro nezvimwe zvinoshandura zvinorehwa nendima.

Mubatsiri wekosi yeyunivhesiti anotsvaga mitsara-chikamu chechunks kubva pane zvinyorwa zvehurukuro uye anodzosera peji yakatenderedza, saka mhinduro nezve fomula inosanganisirawo mamiriro ainoshanda pasi pawo.

Mubatsiri weinishuwarenzi anotora machisi akati wandei kubva mugwaro repolicy rimwe, ovaisa nemubereki, uye anotumira chikamu ichocho chepolicy kamwe chete pane zvimedu zvishanu zvinopindirana.

Ramangwana reRugwaro Rwevabereki uye Kudzoserwa Kwediki-ku-Kukuru

Diki-kusvika-hombe iri kuita yekusarudzika pane hunyengeri, uye makuru masisitimu atofumura hierarchical kana hwindo rekudzoreredza semagadzirirwo sarudzo. Mahwindo makuru emamiriro ezvinhu anoderedza chirango chekutumira vabereki vakakura, izvo zvinoita kuti pateni idhure kutora. Mibvunzo yakavhurika inoramba iri pamusoro pekusarudza miganhu yevabereki otomatiki, semuenzaniso kushandisa ongororo yedhizaini kana chikamu chakadzidzwa pane kuvimba nemisoro. Iri kuwedzera kusanganiswa nevanogadzirisa uye nemaitiro-ekumisikidza maitiro ayo anoedza kupa madiki chunks kuziva zvakavatenderedza panguva ye indexing. Kuedza pamagwaro ako pachako, kwete mabhenji akajairwa, inoramba iri nzira yakavimbika yekusarudza saizi yemwana nemubereki.

Kuzvishandisa

Nei zvichikosha

  • Mutengo uye bhajeti. Zvisarudzo zvezvivakwa zvinotyaira kuita uye mutengo wekushandisa kwemakore.
  • Sarudzo dzakajeka. Dzidzo yehunyanzvi inobatsira zvikwata kusarudza murwi wakakodzera, kwete iwo mutsva chete.
  • Kudzora kwemhando yepamusoro. Sarudzo dzeinjiniya dziri nani dzinoderedza zviitiko zvekuvimbika mukugadzira.

Chenjera

  • Kugadzirisa imwe bhenji kunogona kuvanza yakafara system kushaya simba.
  • Infrastructure uye mari yekugadzirisa inowanzotarisirwa pasi.
  • Chengetedzo uye kucherechedzwa mapundu anogona kukura sezvo masisitimu anowedzera kuoma.

Matanho anotevera

  1. Tsanangura latency, mhando, uye mutengo zvinangwa usati waitwa.
  2. Benchmark pasi pechokwadi mutoro uye data mamiriro.
  3. Chishandiso chekutarisa zvikanganiso, kudonha, uye mushandisi maitiro.
  4. Gadzirira nzira dzekudzosera kumashure uye dzezviitiko usati wawedzera.

Ramba Uchiongorora

Tsamba yemahara

Tora iyo yemazuva ese AI pfupiso

Nhatu dzakasimbiswa dzeAI nyaya mangwanani ega ega pakati pevhiki, dzakanyorwa neChirungu chakajeka. Yemahara zvachose, hapana ads.

Imwe email zuva rega rega revhiki. Bvisa kunyoresa kamwe chete. Hatimbotengese kana kugovera kero yako.

Zviedze iwe pachako

Take the Parent Document and Small-to-Big Retrieval quiz

Mhinduro yepakarepo pamhinduro yega yega, uye chitupa chinogovaniswa chine ID inobatika kana wapedza kosi.

Tanga mibvunzo

Tsigira dzidzo yemahara yeAI. AI Understanding ndeye 501(c)(3) isingabatsiri - hapana ads, hapana paywall, zvachose. Ita chipo

Mibvunzo inowanzo bvunzwa

Chii chinonzi Gwaro Revabereki uye Diki-ku-Ku-Kukuru Kudzosa?

Kudzosa magwaro evabereki, kunonziwo kutsvagisa kudiki-kusvika-kukuru, kunotsvaga pazvimedu zvidiki, zvemavara chaizvo asi inopa modhi yemutauro chikamu chikuru kana gwaro chikamu chimwe nechimwe chakabva. Izvo zvine basa nekuti inobvisa yakajairwa chunking kutengeserana-kure: madiki chunks anoenderana nemibvunzo nenzira kwayo, uye mubereki mukuru anopa iyo modhi yakakwana mamiriro ekupindura nemazvo.

Ndeupi musimboti wekutengeserana kunogadzirisa kudzoreredza kudiki-kusvika-kukuru?

Zvimedu zvidiki zvinopa zvakanangana zvekumisikidza asi zvakanyanyisa mamiriro; zvimedu zvakakura zvinopa mamiriro asi zvakadzika zvakadzika. Zvidiki-kusvika-zvikuru zvinotsvaga zvidiki uye zvinodzoka zvakakura kuti uwane zvese.

Mune diki-kusvika-hombe system, chii chakamisikidzwa nekutsvagirwa panguva yekubvunza?

Vana vanoiswa indexed kuti vanyatsoenderana; vabereki vanongotariswa mushure memutambo wemwana.

Kana matatu akadzoreredzwa machunks ese ari echikamu chimwe chevabereki, system yacho inofanira kuitei?

Kuronga nemubereki kunoreva kuti chikamu chinosanganisirwa kamwe chete, chinochengetedza tokeni uye chinobvisa zvakapetwa zvidimbu.

LangChain's ParentDocumentRetriever inochengeta sei iwo mazinga maviri ezvinyorwa?

Vana vakaiswa muchitoro chevector kutsvaga, nepo vabereki vakazara vakagara muchitoro chemagwaro uye vanotorwa neID.

Chii chinodzosa mutsara-window?

Mutsara-hwindo rekudzoreredza rinoenderana pamwero wemutsara wobva wawedzera kusvika pafafitera remitsara yakatenderedza kugovera mamiriro.