Lens Logit na Nchọpụta oyi akwa dị n'etiti
Igwe oghere logit bụ aghụghọ ịkọwapụta nke na-akọwapụta ọnọdụ zoro ezo nke transformer na oyi akwa ọ bụla n'ime amụma okwu, na-ahapụ gị ka ị lelee ụdị ntugharị n'ofe omimi.
Nchịkọta
It matters because it turns an opaque stack of math into a readable, layer-by-layer story of how the model arrives at its answer.
Ime miri emi
Ihe ngbanwe na-ewulite amụma site n'ọtụtụ ọkwa, nke ọ bụla na-agbakwunye na vector ' residual stream'. Igwe oghere logit na-ewe ọnọdụ zoro ezo na oyi akwa etiti, na-etinye ụkpụrụ ikpeazụ oyi akwa-norm na mmepụta ya na-enweghị ntụpọ matriks, wee gụpụta akara ngosi nke steeti ele mmadụ anya n'ihu masịrị ya. N'ihi na oyi akwa ọ bụla na-ede n'ime otu iyi fọdụrụnụ, ị nwere ike decoded ya n'oge n'agbanyeghị na e mere ya maka oyi akwa ikpeazụ. Ndị nchọpụta na-achọpụta na maka ọtụtụ eziokwu na-akpalite akara ziri ezi na-apụta na etiti etiti wee nụchaa ya, ebe ọkwa mbụ na-adịkarị elu elu-larịị ma ọ bụ detuo-ntụgharị ntinye. Ndị dị iche iche dị ka 'lens a na-ege ntị' na-azụ obere nyocha nke otu oyi akwa iji mezie maka ndakọrịta ahụ, na-enye ọhụụ dị ọcha na obere mkpọtụ.
Nghọta nka nka
Mechanically: were residual stream activation h_L na oyi akwa L, mụbaa site na unmbedding (mgbe a na-ekekọta ntinye-mbedding transpose) mgbe ikpeazụ LayerNorm, wee softmax. Nke a na-arụ ọrụ n'ihi na mmiri fọdụrụnụ bụ ihe mgbakwunye ma na-ekerịta ntọala na oghere mmepụta n'ofe ọkwa. Igwe oghere dị larịị na-eleda anya n'isi n'isi; oghere a na-ege ntị na-amụta mgbanwe affine A_L h_L + b_L kwa oyi akwa ka ịdepụta steeti etiti n'ime usoro ngbanwe ikpeazụ n'ikwesị ntụkwasị obi.
Mmetụta atụmatụ
Ọsọ na ọnụ ọgụgụ
Usoro ọrụ asụsụ nwere ike ịga ngwa ngwa n'achụghị nkwụsi ike.
Nweta na iru
Ọ na-agbasawanye ohere n'ofe asụsụ na ụdị nzikọrịta ozi.
Mkpebi doro anya
Ndị otu nwere ike itinyekwu oge na ikpe ebe akpaaka na-ejikwa nkwughachi.
Ọdịnihu nke oghere Logit na Ndozi oyi akwa etiti
Ndozi ụdị oghere Logit na-aghọ ihe nyocha ọkọlọtọ na nkọwa ọrụ yana nyocha nchekwa AI. Na-atụ anya njikọta siri ike na ndị na-enweghị autoencoders na akwụkwọ ọkọwa okwu, yabụ ndị nyocha nwere ike ịkpọ echiche ndị oyi akwa na-akwalite kama ịdepụta naanị akara ngosi. Ka ụdị na-eto eto, bọọdụ lens akpaaka nwere ike ịpụta ebe a na-ahụ anya ma ọ bụ mmecha na-adịghị mma ga-ebu ụzọ buru ụzọ mee ya, na nhazi ụdị oghere nwere ike ibu ụgbọ mmiri dị ka ngwa nbibi n'ime pipeline ọzụzụ.
Mmejuputa n'ezie n'ụwa
Ịhụ na nke oyi akwa ihe nlereanya mbụ 'maara' isi obodo France tupu azịza ikpeazụ ya.
Ịchọpụta ihe nhụsianya site n'ịhụ oyi akwa ebe akara ngosi na-ezighi ezi mana ntụkwasị obi na-ebu ụzọ na-achịkwa iyi ndị fọdụrụnụ.
Na-atụnyere oghere oghere dị larịị vs. len anya elere anya iji tụọ ka e si hazie nkwenye etiti ihe nlereanya.
Na-enyocha ma akara ọjụjụ dị mkpa maka nchekwa ọ pụtara n'oge ma ọ bụ na agbakwụnye ya site na ọkwa ole na ole ikpeazụ.
Ihe ize ndụ & okporo ụzọ nche
Eziokwu ndị e chepụtara echepụta nwere ike jiri nwayọ tinye akụkọ, nkwado nkwado, ma ọ bụ nsonaazụ nyocha.
Mmetụta ngwa ngwa nwere ike ịmepụta nsonaazụ na-ekwekọghị ekwekọ n'ofe arịrịọ ndị yiri ya.
Enwere ike ikpughe data ederede nwere mmetụta ma ọ bụrụ na njikwa ohere adịghị ike.
Map mmejuputa
Kọwaa usoro mmepụta, ụda, na ụkpụrụ ịdịmma tupu ibugharị.
Weghachite nzaghachi site na isi mmalite ntụkwasị obi mgbe ọ bụla izi ezi dị mkpa.
Debe ebe nleba anya mmadụ maka mpụta dị elu.
Sochie ụkpụrụ ọdịda ma na-azụghachi mkpali ma ọ bụ usoro ọrụ mgbe niile.
Nọgide na-eme nchọpụta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Logit Lens and Intermediate Layer Decoding quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ntuziaka na-esote
Lens Logit na oghere elegharịrị
Ajụjụ a na-ajụkarị
What is Logit Lens and Intermediate Layer Decoding?
Igwe oghere logit bụ aghụghọ ịkọwapụta nke na-akọwapụta ọnọdụ zoro ezo nke transformer na oyi akwa ọ bụla n'ime amụma okwu, na-ahapụ gị ka ị lelee ụdị ntugharị n'ofe omimi. Ọ dị mkpa n'ihi na ọ na-atụgharị nchịkọta mgbakọ na mwepụ na-adịghị ahụkebe ka ọ bụrụ akụkọ a na-agụ, nke oyiri-na-oyi nke otu ihe nlereanya ahụ si enweta azịza ya.
Kedu ihe oghere logit na-emetụta steeti zoro ezo dị n'etiti iji gụpụta amụma token?
Igwe oghere logit na-ejigharị usoro oyi akwa ikpeazụ nke ihe nlereanya ahụ na matriks na-enweghị ihe ọ bụla iji wepụta vector fọdụrụnụ na-erugharị n'ime usoro okwu.
Kedu ihe kpatara na ọ ga-ekwe omume iji decode etiti n'ígwé na enweghị njedebe ikpeazụ?
Ndị ngbanwe na-agbakwunye mmepụta oyi akwa nke ọ bụla n'ime iyi nke na-ekekọrịta, yabụ steeti etiti adịlarị na oghere dakọtara na ihe ngbanwe ikpeazụ.
Kedu nsogbu 'lens a na-emegharị anya' na-edozi n'ihe gbasara oghere logit dị larịị?
Igwefoto a na-ege ntị na-azụ obere maapụ affine nke ọ bụla ka etiti etiti steeti mebie koodu n'ikwesị ntụkwasị obi karịa, na-ebelata nhụsianya na mkpọtụ nke oghere oghere mbụ.
N'ọtụtụ mkpali dị adị, kedụ ebe akara ngosi ziri ezi na-ebu ụzọ pụta n'okpuru oghere logit?
Nnyocha na-egosi azịza ziri ezi na-apụtakarị na etiti etiti ma ndị ọzọ na-amụba ya, ebe ọkwa mbụ na-akwado elu ma ọ bụ detuo ịkọ nkọ.
Kedu ihe lens logit kacha baa uru ozugbo?
Isi uru ya bụ nkọwa: na-ekpughe mgbanwe oyibo-site-layier nke nkesa akara amụma amụma.