Chii chaitika
Bepa idzva rekutsvagisa rinounza nzira yekumusoro yekuona kufungidzira muMakuru Mutauro Models (LLMs) nekuongorora kuyerera kweruzivo mukati mekutarisa magirafu. Nekuverenga iyo Forman-Ricci curvature yemagirafu aya, vaongorori vakaratidza zvimiro-kunyanya ruzivo mabhodhoro-anoenderana nezvakaburitswa. Iyo nzira inobata zvese semi-yemunharaunda uye yepasirese ruzivo-kuyerera maitiro, ichibvumira kune imwechete-pass yekuona nzira iyo inodarika iripo yakawanda-mhinduro uye yekutarisisa-yakavakirwa nheyo.
Chidzidzo ichi, chakanzi 'Detecting muLLMs: Kutsvaga Iyo Topological Siginecha yeImpaired Context Kugovera,' inotarisa kumakanika emukati emisoro yekutarisisa. Vanyori vanopa zano kuti iyo topology yekuyerera kweruzivo chiratidzo chakavimbika chekuti modhi iri kuburitsa zvechokwadi zvemukati kana kufungidzira.
Nekushandisa Forman-Ricci curvature kune yekutarisisa magirafu, vaongorori vakaratidza chaiwo masiginicha. Aya masiginicha anosimbisa 'mabhodhoro eruzivo' apo modhi inotadza kunyatso kubatanidza mamiriro kubva kumatokeni apfuura. Nzira iyi inotsanangurwa senzira ye 'single-pass', iyo inonzi nevanyori inoshanda kupfuura nzira dziripo dzinoda zvizvarwa zvekupindura zvakawanda kana matanho akaoma kunzwisisa ekunze.
Iyo empirical ongororo yakaitwa pane akawanda maLLM ekuvaka uye maviri akamiswa ekuona-yekuona mabhenji. Mhedzisiro yacho inoratidza kuvandudzwa kwemaitiro anowirirana pamusoro pezvivakwa zvemazuva ano, zvichiratidza kuti kuongororwa kwetopological chiratidzo chakasimba chekuvimbika kwemuenzaniso.
Kwakabva mashoko: arxiv.org ↗
Nei zvichikosha
Tsvagiridzo iyi inopa tsananguro yekuti sei maLLM achinyengedza, achibatanidza kukanganisa kwechokwadi nekutadza chaiko mukugovaniswa kwechiratidzo-chikamu. Nekuziva kuti fungidziro dzinowanzobva mukunyanya kuvimba nekuzvitarisisa, kudzoreredzwa kwemamiriro ezvinhu, kana ruzivo rwakanyanya-squashing mune yekupedzisira transformer layer, chidzidzo chinopa chakanyanya chekuongorora chishandiso pane dema-bhokisi rekuongorora. Izvi zvinogona kutungamira kune yakavimbika modhi yezvivakwa uye yakagadziridzwa chengetedzo mirairi inotarisisa kuyerera kwemukati mukati-chaiyo-nguva pane kuvimba chete nekusimbisa kwekunze.
Ikozvino kuona kwekufungidzira kunowanzoenderana nekutarisa kwekunze kwechokwadi kana kuenzanisa akawanda mamodhi anobuda, ayo ari computationally anodhura uye anowanzo kune ayo anokanganisa. Tsvagiridzo iyi inoshandura tarisiro kune mamiriro emukati memuenzaniso, ichipa chishandiso chekuongorora chinozivisa 'nei' kuseri kwekufungidzira.
Kuwana kuti kufungidzira kwakabatana ne'kusakanganiswa kugovana mamiriro' - kunyanya kuwedzeredza-kupwanya kana kudzoreredzwa kudzoreredzwa mune yekupedzisira transformer layer - inopa chinangwa chekongiri kune vanogadzira modhi. Panzvimbo pekutarisa kwakafara, vagadziri vanogona kushandisa aya epamusoro ekuona kuti agadzirise nzira dzekutarisisa kana nzira dzekuchekerera kuvandudza kuenderana kwechokwadi.
Iyi nzira inomiririra danho rakananga 'kududzirwa kwemichina,' uko chinangwa chiri chekunzwisisa pfungwa yemukati yemodhi pane kuitora sebhokisi dema. Kana iyi nzira ikaratidza scalable, inogona kuve yakajairwa chikamu cheAI chekuchengetedza bvunzo, ichibvumira kuzivikanwa kwemamodheru anotadze kuonesa kuona asati aiswa munzvimbo dzakakwirira.
Interactive Mechanism: Iyo Inonyatsoshanda
Ongorora ari pasi tekinoroji kuseri kwekusimudzira uku uchipindirana.
In AI, what are a model's "parameters"?
Zvekutarisa zvinotevera
Vatsvagiri vakaratidza nzira iyi pane akati wandei eLLM zvivakwa, asi iko kubatanidzwa kweiyo topological ongororo mumapaipi ekugadzira-giredhi inference inoramba iri dambudziko rakavhurika. Zvichaitika mune ramangwana zvingangotarisa kuti nzira iyi inogona kushandiswa kugadzirisa zvine simba mamodheru panguva yechizvarwa kana ichingogumira pakuonekwa kwepashure-hoc. Izvozvi hazvizivikanwe kana nzira iyi inogona kuyerwa kune yakakura kwazvo modhi pasina kuunza yakakosha latency pamusoro panguva yechizvarwa maitiro.
Chekutanga chisingazivikanwe mutengo wekuverengera weForman-Ricci curvature panguva chaiyo-nguva inference. Nepo vanyori vachiitsanangura senzira ye 'single-pass', kuomarara kwemasvomhu ekuongorora kwepamusoro kunogona kuunza latency isingagamuchirwe kune chaiyo-nguva yekutaura maapplication.
Chidzidzo chacho hachitsananguri kana nzira yacho iri modhi-agnostic kana kana ichida zvigadziriso zvemagadzirirwo kuti zvinyatsoshanda pane zvakasiyana-siyana zvekushandura. Imwe tsvagiridzo inodiwa kuona kana nzira iyi inopesana nezvinokurudzirwa nevavengi zvakagadzirirwa kukonzeresa zvionekwe mumhando dzakanyanyisa, dzakakura.
Vatsvakurudzi havana kupa ruzivo maererano nekuvepo kwekodhi kana mabhenji chaiwo anoshandiswa kune veruzhinji kuti vaone mhedzisiro iyi vakazvimirira. Nharaunda inofanirwa kutarisa kuburitswa kwekuita kuti vaone kana kuita kwacho kunobata mune yakakura, chaiyo-yepasirese yekuyedza mamiriro.