Lens Logit ak decodage couche intermédiaire
Lens logit bi ab pexe tekki la buy dekode nekkinu transformatër bu nëbbu ci bepp layer ci vocabulaire buy wax luy waaja am, loolu dina tax nga mëna seetaan formu guess ci xóotaayu.
Résumé
It matters because it turns an opaque stack of math into a readable, layer-by-layer story of how the model arrives at its answer.
Plongeur bu xóot
Transformatër dafay tabax ab wax luy waaja am ci ay fukki-fukki diisaay, bu nekk di yokk ci vecteur 'residual stream' buñ bokk. Lens logit bi dafay jël stade bu nëbbu bi ci diggu couche bi, jëfandikoo couche bu mujj bi ci model bi ak matrix biy génne, ba noppi jàng ci token yi stade partiel bi bëgg. Ndax bépp couche dafay bind ci benn stream residual, mën nga ko teela decoder doonte couche bi mujj lañu ko jagleel. Gëstukat yi gis nañu ni ci laaj yu bari yu dëggu, token bu dëggu bi dafay feeñ ci diggu layers yi ba noppi ñu setal ko, ci layers yu njëkk yi dañuy faral di feeñ ci surface-niveau wala ñu kopie-the-input guesses. Variante yu melni 'lens tuned' dafay tàggat ab sonde bu ndaw ci couche bu nekk ngir saafara jafe-jafe yi, di joxe liñ jàng bu gëna set te gëna néew bruit.
Gis-gis xarala
Ci anam wu mekanik: jëlal h_L bi des ci couche L, nga yokk ko ci unembedding (dafay faral di nekk transpose buñ boole) ginaaw LayerNorm bi mujj, ginaaw ga softmax. Loolu dafay dox ndax residual stream bi dafay yokk ba noppi bokk benn base ak espace biy génn ci diisaay yi. Lens plain bi dafay teela tëju; lens biñ akorde dafay jàng soppi affine A_L h_L + b_L ci couche bu nekk ngir mëna xayma diggante yi ci kaadar decodage bu mujj bi ci anam wu gëna dëggu.
njeextalu pexe
Gaawaay ak yaatuwaay
Liggéeyukaay yi ci làkk yi mën nañu gëna gaaw te duñu yàq deggoo gi.
Dugg ak yegg
Dafay yaatal jëfandikoo gi ci làkk yi ak ci anam yi ñuy jokkoo.
dogal yu gëna leer
Ekip yi mën nañu gëna yàgg ci àtte ci jamono ji otomatisation di liggéey ci baamtu.
Ëlëgu Logit Lens ak dekodaasu diggante couche
Dekodaasu stil lens Logit mingi nekk sonde buñ miin ci tekkib mekanik ak saytu kaaraange IA. Xaarandil lëkkaloo bu gëna dëgër ak autoencoders yu néew ak diksoneer yu am solo, suko defee jàngat yi mëna joxe tur ci konsept yi benn layer di fësal, duñu lim token yi kese. Lu model yi di màgg, tablo lenti otomatik yi mën nañu màndargaal fi hallucinations yi wala matt yu wóorul yi njëkka kristalise, ba noppi etalonnage lens-style mën na nekk jumtukaayu debugging ci biir pipeline tàggat.
Doxal ci àdduna dëgg
Xalaatal ci ban layer la model bi njëkka 'xam' kapitaalu France balaa tontu bu mujj bi.
Saytu hallucinations ci gis couche bi token bu jaarul yoon waaye wóolu boppam moo njëkka jiital stream residuel bi.
Teg lens logit bu woyof ak lens tuned ngir natt ni gëm-gëmu diggu model bi di kalibree.
Saytu ndax token buy bañ bu jëm ci kaaraange mingi teela feeñ wala dañu koy yokk ci diisaay yu mujj yu néew yi.
Risk yi ak balustrade yi
Lépp lu jaarul yoon mën na dugg ci rapoor yi, jàppale ci liggéey bi, wala ci njariñu gëstu bi.
Sensibilite bu gaaw mën na jur njariñ yu wuute ci laajte yu noonu mel.
Done yu am solo mën nañu feeñ sudee seytu jëfandikoo gi néew doole.
Roadmap ngir samp gi
Mandargal formaa génne gi, melokaan bi, ak standard kalite yi laata ngay dugal ko.
Tontu yu am solo ak balluwaay yu wóor saa yu dëggu bi di am solo.
Fexeel am barabu xool nit ñi ngir am njariñ yu am solo.
Toppal anami gacce yi ak di faral di tàggataat ay laaj wala def-liggéey.
Weyal di banneexu
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Logit Lens and Intermediate Layer Decoding quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gis bi ci topp
Lentille Logit et Lentille Tune
Laaj yi ñuy faral di laaj
What is Logit Lens and Intermediate Layer Decoding?
Lens logit bi ab pexe tekki la buy dekode nekkinu transformatër bu nëbbu ci bepp layer ci vocabulaire buy wax luy waaja am, loolu dina tax nga mëna seetaan formu guess ci xóotaayu. Dafa am solo ndax dafay soppi pilu math bu opaque nekk luñu mëna jàng, jaar-jaar buy wane ni model bi amee ci tontu am.
Lan la lens logit bi di jëfandikoo ci diggu nekkin bu nëbbu ngir jàng li ñuy wax ci token?
Lens logit bi dafay jëfandikoowaat matrix bi mujjee ci model bi ak matrix bi amul benn xët ngir wane vecteur residuel bu digg bi ci biir logits vocabulaire yi.
Lan moo tax ñu mëna sax dekode diggu diggu yi ak unembedding bu mujj bi?
Transformatër yi dañuy yokk génnug couche bu nekk ci benn stream residuel buñ bokk, suko defee etaa yu digg yi nekk ci barab bu méngoo ak decodeur bi mujj.
Lan mooy jafe-jafe bi 'lens tuned' bi di saafara suñu ko méngale ak lens logit bi?
Lens biñ akorde dafay tàggat ab kàrtu affine bu ndaw ci couche bu nekk suko defee etaa yi ci digg yi gëna mëna decode ci anam wu gëna dëggu, wàññi biais ak bruit bi ci couche yu njëkk yi ci lens plain bi.
Ci laaj yu bari yu am solo, fan la token bu dëggu bi di njëkka feeñ ci suufu logit lens bi?
Jàngat yi dañu wane ni tontu bu dëggu bi dafay faral di feeñ ci diggu couche yi, ñu koy gëna ñaw ci couche yu ci topp yi, waaye couche yu njëkk yi dañu taamu surface wala copy guesses.
Lan mooy lens logit bi gëna am njariñ?
Valeur bi gëna am solo mooy mën nañu ko tekki: wane jëm kanam bu couche par couche ci distribution token biñ séentu ci model bi.