InoteveraGaidhi rinotevera
Logit Lens uye Yepakati Layer Decoding
Mutauro AI
Nhungamiro yehunyanzvi
Logit Lens uye Tuned Lens
Iyo logit lens uye tuned lens inzira dzekududzira dzinotarisa kune yakavanzika nyika ye transformer ne layer kuti uone kuti modhi iri 'kufunga' chii isati yaburitsa mhinduro yekupedzisira.
Teerera kugwaro iriAI-yakarondedzerwa · ~2 min
Pa peji ino2 min verenga
Pfupiso
They reveal how a prediction gradually forms as information flows up through the network.
Kudzika Kwakadzika
Shanduko inovaka mhinduro yayo nekuwedzera: imwe neimwe nhanho inowedzera kune inomhanya 'inosara rukova' iyo inongoshandurwa kuita mukana wemazwi kumagumo chaiko. Iyo logit lens, yakaunzwa ne nostalgebraist muna 2020, inodimbudzira izvi nekushandisa iyo yekupedzisira yemodhi isingachinjiki (uye layer yakajairwa) yakananga kumatanho epakati, kuti iwe uverenge kunze kweiyo network fungidziro pakadzika kwese. Izvi zvinowanzoratidza mhinduro ichipenya muzvikamu zvepakati-kusvika-zvinononoka. Iyo lenzi yakamisikidzwa (Belrose nevamwe vaanoshanda navo, 2023) inovandudza pairi nekudzidzisa diki diki probe padanho kushandura nyika dzakavanzwa munheyo yekupedzisira, kugadzirisa kurerekera uye kusarongeka iyo mbishi logit lens inotambura, kunyanya mumatanho ekutanga uye kumhuri dzakasiyana dzemhando.
Kuitwa muNyika yeChokwadi
Kushandisa logit lens kuona mhinduro yechokwadi senge guta guru rinobuda muzvikamu zvepakati zvemodhi
Kushandisa lens yakatuniwa kuenzanisa kuti mhuri dzemhando dzakasiyana dzinosangana sei pakufungidzira pakadzika
Kuona kuti modhi yave nemukati 'yasarudza' mhinduro akati wandei akaturikidzana isati yabuda
Kuongorora maseru apo fungidziro yechiratidzo inokuvadza kana yakarerekera inotanga kuve nesimba murukova rwasara.
Ramangwana reLogit Lens uye Tuned Lens
Maitiro eLens ave kuita chiyero chekutsvaga kuti chokwadi, kuramba, kana kusarerekera kunobuda sei pakadzika, uye nekuona kana modhi 'yakaziva' mhinduro nekukurumidza. Tarisira ivo vakasanganiswa ne sparse autoencoder uye causal patching kuti ifambe kubva pakutsanangura kufanotaura kuenda kutsananguro nzira. Tsvagiridzo iri kuongororawo kana kuverenga kwepakati kunoratidza ruzivo rwakadzika kana hunyengeri modhi inovanza mune yayo yekupedzisira kuburitsa, ichiita lenzi chivako chekuvakira chekuchengetedza ongororo uye yekukurumidza-yambiro.
Kuzvishandisa
Nei zvichikosha
- Mutengo uye bhajeti. Zvisarudzo zvezvivakwa zvinotyaira kuita uye mutengo wekushandisa kwemakore.
- Sarudzo dzakajeka. Dzidzo yehunyanzvi inobatsira zvikwata kusarudza murwi wakakodzera, kwete iwo mutsva chete.
- Kudzora kwemhando yepamusoro. Sarudzo dzeinjiniya dziri nani dzinoderedza zviitiko zvekuvimbika mukugadzira.
Chenjera
- Kugadzirisa imwe bhenji kunogona kuvanza yakafara system kushaya simba.
- Infrastructure uye mari yekugadzirisa inowanzotarisirwa pasi.
- Chengetedzo uye kucherechedzwa mapundu anogona kukura sezvo masisitimu anowedzera kuoma.
Matanho anotevera
- Tsanangura latency, mhando, uye mutengo zvinangwa usati waitwa.
- Benchmark pasi pechokwadi mutoro uye data mamiriro.
- Chishandiso chekutarisa zvikanganiso, kudonha, uye mushandisi maitiro.
- Gadzirira nzira dzekudzosera kumashure uye dzezviitiko usati wawedzera.
Ramba Uchiongorora
Tsamba yemahara
Tora iyo yemazuva ese AI pfupiso
Nhatu dzakasimbiswa dzeAI nyaya mangwanani ega ega pakati pevhiki, dzakanyorwa neChirungu chakajeka. Yemahara zvachose, hapana ads.
Imwe email zuva rega rega revhiki. Bvisa kunyoresa kamwe chete. Hatimbotengese kana kugovera kero yako.
Zviedze iwe pachako
Take the Logit Lens and Tuned Lens quiz
Mhinduro yepakarepo pamhinduro yega yega, uye chitupa chinogovaniswa chine ID inobatika kana wapedza kosi.
Tsigira dzidzo yemahara yeAI. AI Understanding ndeye 501(c)(3) isingabatsiri - hapana ads, hapana paywall, zvachose. Ita chipo
Mibvunzo inowanzo bvunzwa
What is Logit Lens and Tuned Lens?
Iyo logit lens uye tuned lens inzira dzekududzira dzinotarisa kune yakavanzika nyika ye transformer ne layer kuti uone kuti modhi iri 'kufunga' chii isati yaburitsa mhinduro yekupedzisira. Ivo vanoratidza kuti kufanotaura kunogadzirwa sei zvishoma nezvishoma sezvo ruzivo rwunoyerera kuburikidza netiweki.
Logit lens inoita sei?
Iyo logit lens mapurojekiti epakati asara-rukova nyika kuburikidza neiyo iripo yekumisikidza kuti uone iyo modhi yakafanotaurwa tokeni pakadzika kwega kwega.
Ndeipi kuvandudza kwakakosha iyo lens yakaturikirwa inowedzera pamusoro peiyo mbishi logit lens?
Iyo lenzi yakamisikidzwa inodzidza muturikiri wemutsara wega wega, kugadzirisa zvakarerekera uye kupa akatendeka, akaderera-anovhiringika kuverenga pane mbishi logit lens.
Ndechipi chimiro mumatransformer chinoita kuti lenzi idzi dzigoneke?
Imwe neimwe nhanho inonyora yekuwedzera inogadziridza kune yakagovaniswa yakasara rukova, saka kufungidzira iwo mafambiro ekutanga anofungidzira kufanotaura kwepakati.
Ndiani akaunza yekutanga logit lens, uye riini?
Iyo logit lens yakaunzwa ne nostalgebraist mune 2020 blog post inoongorora GPT-2's yepakati fungidziro.
Ndekupi iyo logit lens inowanzoratidza mhinduro yekupedzisira 'crystallizing'?
Maverengero anowanzo ratidza chiratidzo chakakodzera kuwana mukana kubva pakati-kusvika-kunonoka maseru sezvo komputa inoungana.
Ramba uchidzidza
Mazano anoenderana
Mamwe madhairekitori akasarudzirwa nyaya iyi