Logit Lens uye Tuned Lens
Iyo logit lens uye tuned lens inzira dzekududzira dzinotarisa kune yakavanzika nyika ye transformer ne layer kuti uone kuti modhi iri 'kufunga' chii isati yaburitsa mhinduro yekupedzisira.
Pfupiso
They reveal how a prediction gradually forms as information flows up through the network.
Kudzika Kwakadzika
Shanduko inovaka mhinduro yayo nekuwedzera: imwe neimwe nhanho inowedzera kune inomhanya 'inosara rukova' iyo inongoshandurwa kuita mukana wemazwi kumagumo chaiko. Iyo logit lens, yakaunzwa ne nostalgebraist muna 2020, inodimbudzira izvi nekushandisa iyo yekupedzisira yemodhi isingachinjiki (uye layer yakajairwa) yakananga kumatanho epakati, kuti iwe uverenge kunze kweiyo network fungidziro pakadzika kwese. Izvi zvinowanzoratidza mhinduro ichipenya muzvikamu zvepakati-kusvika-zvinononoka. Iyo lenzi yakamisikidzwa (Belrose nevamwe vaanoshanda navo, 2023) inovandudza pairi nekudzidzisa diki diki probe padanho kushandura nyika dzakavanzwa munheyo yekupedzisira, kugadzirisa kurerekera uye kusarongeka iyo mbishi logit lens inotambura, kunyanya mumatanho ekutanga uye kumhuri dzakasiyana dzemhando.
Technical Insight
Maitiro ese ari maviri anoshandisa iyo yasara yerukova maonero: yega yega inonyora yekuwedzera zvigadziriso kune yakagovaniswa vheti iyo yekusamisa matrix gare gare mapurojekiti kune mazwi emazwi. Iyo logit lens inoshandisazve izvo chaizvo kusamira pamatunhu epakati pasina kumwe kudzidziswa. Iyo lenzi yakamisikidzwa panzvimbo pacho inodzidza mepu yemutsara-mutsetse (yakadzidzwa 'muturikiri') kuti mamiriro ega ega anoshandurwa kuita fomati iyo yekupedzisira layer inotarisirwa, ichipa fungidziro yakapfava, yakatendeka uye yakaderera-kuvhiringika.
Strategic Impact
Mutengo uye bhajeti
Zvisarudzo zvezvivakwa zvinotyaira kuita uye mutengo wekushandisa kwemakore.
Sarudzo dzakajeka
Dzidzo yehunyanzvi inobatsira zvikwata kusarudza murwi wakakodzera, kwete iwo mutsva chete.
Kudzora kwemhando yepamusoro
Sarudzo dzeinjiniya dziri nani dzinoderedza zviitiko zvekuvimbika mukugadzira.
Ramangwana reLogit Lens uye Tuned Lens
Maitiro eLens ave kuita chiyero chekutsvaga kuti chokwadi, kuramba, kana kusarerekera kunobuda sei pakadzika, uye nekuona kana modhi 'yakaziva' mhinduro nekukurumidza. Tarisira ivo vakasanganiswa ne sparse autoencoder uye causal patching kuti ifambe kubva pakutsanangura kufanotaura kuenda kutsananguro nzira. Tsvagiridzo iri kuongororawo kana kuverenga kwepakati kunoratidza ruzivo rwakadzika kana hunyengeri modhi inovanza mune yayo yekupedzisira kuburitsa, ichiita lenzi chivako chekuvakira chekuchengetedza ongororo uye yekukurumidza-yambiro.
Real-World Implementation
Kushandisa logit lens kuona mhinduro yechokwadi senge guta guru rinobuda muzvikamu zvepakati zvemodhi
Kushandisa lens yakatuniwa kuenzanisa kuti mhuri dzemhando dzakasiyana dzinosangana sei pakufungidzira pakadzika
Kuona kuti modhi yave nemukati 'yasarudza' mhinduro akati wandei akaturikidzana isati yabuda
Kuongorora maseru apo fungidziro yechiratidzo inokuvadza kana yakarerekera inotanga kuve nesimba murukova rwasara.
Njodzi & Guardrails
Kugadzirisa imwe bhenji kunogona kuvanza yakafara system kushaya simba.
Infrastructure uye mari yekugadzirisa inowanzotarisirwa pasi.
Chengetedzo uye kucherechedzwa mapundu anogona kukura sezvo masisitimu anowedzera kuoma.
Implementation Roadmap
Tsanangura latency, mhando, uye mutengo zvinangwa usati waitwa.
Benchmark pasi pechokwadi mutoro uye data mamiriro.
Chishandiso chekutarisa zvikanganiso, kudonha, uye mushandisi maitiro.
Gadzirira nzira dzekudzosera kumashure uye dzezviitiko usati wawedzera.
Ramba Uchiongorora
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Logit Lens and Tuned Lens quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gaidhi rinotevera
Logit Lens uye Yepakati Layer Decoding
Mibvunzo inowanzo bvunzwa
What is Logit Lens and Tuned Lens?
Iyo logit lens uye tuned lens inzira dzekududzira dzinotarisa kune yakavanzika nyika ye transformer ne layer kuti uone kuti modhi iri 'kufunga' chii isati yaburitsa mhinduro yekupedzisira. Ivo vanoratidza kuti kufanotaura kunogadzirwa sei zvishoma nezvishoma sezvo ruzivo rwunoyerera kuburikidza netiweki.
Logit lens inoita sei?
Iyo logit lens mapurojekiti epakati asara-rukova nyika kuburikidza neiyo iripo yekumisikidza kuti uone iyo modhi yakafanotaurwa tokeni pakadzika kwega kwega.
Ndeipi kuvandudza kwakakosha iyo lens yakaturikirwa inowedzera pamusoro peiyo mbishi logit lens?
Iyo lenzi yakamisikidzwa inodzidza muturikiri wemutsara wega wega, kugadzirisa zvakarerekera uye kupa akatendeka, akaderera-anovhiringika kuverenga pane mbishi logit lens.
Ndechipi chimiro mumatransformer chinoita kuti lenzi idzi dzigoneke?
Imwe neimwe nhanho inonyora yekuwedzera inogadziridza kune yakagovaniswa yakasara rukova, saka kufungidzira iwo mafambiro ekutanga anofungidzira kufanotaura kwepakati.
Ndiani akaunza yekutanga logit lens, uye riini?
Iyo logit lens yakaunzwa ne nostalgebraist mune 2020 blog post inoongorora GPT-2's yepakati fungidziro.
Ndekupi iyo logit lens inowanzoratidza mhinduro yekupedzisira 'crystallizing'?
Maverengero anowanzo ratidza chiratidzo chakakodzera kuwana mukana kubva pakati-kusvika-kunonoka maseru sezvo komputa inoungana.