Mpempe akwụkwọ akpaaka maka mwepụta atụmatụ
Mpempe akwụkwọ akpaaka na-agbawa na-emepe ihe mgbagwoju anya n'ime netwọkụ akwara n'ime ọtụtụ puku atụmatụ mmadụ nwere ike ịgụ.
Nchịkọta
They are the leading tool for understanding what concepts a language model has actually learned.
Ime miri emi
N'ime ihe ngbanwe, otu neuron na-agbakarị ọkụ maka ọtụtụ echiche na-enweghị njikọ - ihe a na-akpọ superposition, ebe ihe nlereanya ahụ na-ejikọta atụmatụ karịa ka o nwere akụkụ. A zụrụ obere autoencoder (SAE) ka ọ rụgharịa vector activation nke oyi akwa site n'ịgafe ya na oyi akwa zoro ezo nke ukwuu yana ntaramahụhụ na-adịghị mma, yabụ naanị nkeji ole na ole na-arụ ọrụ ozugbo. Nkeji ndị ahụ na-adabakarị n'otu echiche nwere ike ịkọwa. Ọrụ 'Scaling Monosemanticity' nke 2024_AIU_PROTECTED_4__ weputara nde atụmatụ sitere na Claude 3 Sonnet, gụnyere ihe ama ama 'Golden Gate Bridge'. Ịkwalite ya mere ka ihe nlereanya ahụ jiri nlezianya kwuo banyere akwa mmiri ahụ - ihe akaebe na-egosi na njirimara ahụ bụ ihe kpatara ya, ọ bụghị ihe ndabara.
Nghọta nka nka
Otu SAE nwere koodu ntinye nke na-esetịpụ ọrụ d-akụkụ n'ime oghere dị ukwuu (dịka, 10-100x), mgbochi L1 ma ọ bụ n'elu-k na-amanye ọtụtụ latent ka ọ bụrụ efu, yana ihe ngbanwe nke na-ewulite ọrụ mbụ. Ọzụzụ na-ebelata mperi nrụgharị gbakwunyere ntaramarị dị nta. N'ihi na akwụkwọ ọkọwa okwu ezughị oke ma dịkwa ụkọ, latent nke ọ bụla na-aghọ 'monosemantic' - na-agbapụ maka otu echiche - na-eme ka ha nwee ike ịkọwapụta karịa neurons raw.
Mmetụta atụmatụ
Ọsọ na ọnụ ọgụgụ
Usoro ọrụ asụsụ nwere ike ịga ngwa ngwa n'achụghị nkwụsi ike.
Nweta na iru
Ọ na-agbasawanye ohere n'ofe asụsụ na ụdị nzikọrịta ozi.
Mkpebi doro anya
Ndị otu nwere ike itinyekwu oge na ikpe ebe akpaaka na-ejikwa nkwughachi.
Ọdịnihu nke Sparse Autoencoders maka mwepụta atụmatụ
Ndị SAE na-etolite n'ime ngwaọrụ nchekwa bara uru: ịchọpụta aghụghọ, nhụsianya, ma ọ bụ echiche adịghị mma, yana omume nduzi site na njigide njirimara. Ihe ịma aka ka dị - atụmatụ nkewa, mweghachi nrụpụta, yana ịkwado njirimara ndị ahụ zuru oke. Na-atụ anya ụzọ ọzụzụ dị ọnụ ala karịa (top-k na gated SAEs), akara njirimara akpaaka, yana ntinye n'ime dashboards nlekota ihe nlereanya ka ndị na-arụ ọrụ wee nwee ike nyochaa ihe ụdị ewepụrụ 'na-eche' ozugbo.
Mmejuputa n'ezie n'ụwa
Anthropic na-eweputa njirimara 'Golden Gate Bridge' site na Claude 3 Sonnet na ijikwa ihe nlereanya site n'ịkwalite ya.
Ịmata njirimara dị mkpa nchekwa dị ka aghụghọ, sycophancy, ma ọ bụ adịghị ike koodu n'ime mmemme ihe nlereanya.
Na-emebi neurons polysemantic n'ime ọtụtụ njirimara monosemantic iji dozie nnukwu ọnọdụ
Ntugharị njiri mara: ịgbanye ma ọ bụ gbanyụọ njirimara echiche iji jikwaa mpụta ihe nlereanya na-azụghachighị ya
Ihe ize ndụ & okporo ụzọ nche
Eziokwu ndị e chepụtara echepụta nwere ike jiri nwayọ tinye akụkọ, nkwado nkwado, ma ọ bụ nsonaazụ nyocha.
Mmetụta ngwa ngwa nwere ike ịmepụta nsonaazụ na-ekwekọghị ekwekọ n'ofe arịrịọ ndị yiri ya.
Enwere ike ikpughe data ederede nwere mmetụta ma ọ bụrụ na njikwa ohere adịghị ike.
Map mmejuputa
Kọwaa usoro mmepụta, ụda, na ụkpụrụ ịdịmma tupu ibugharị.
Weghachite nzaghachi site na isi mmalite ntụkwasị obi mgbe ọ bụla izi ezi dị mkpa.
Debe ebe nleba anya mmadụ maka mpụta dị elu.
Sochie ụkpụrụ ọdịda ma na-azụghachi mkpali ma ọ bụ usoro ọrụ mgbe niile.
Nọgide na-eme nchọpụta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Sparse Autoencoders for Feature Extraction quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ntuziaka na-esote
Mpempe akwụkwọ akpaaka maka nkọwa
Ajụjụ a na-ajụkarị
What is Sparse Autoencoders for Feature Extraction?
Mpempe akwụkwọ akpaaka na-agbawa na-emepe ihe mgbagwoju anya n'ime netwọkụ akwara n'ime ọtụtụ puku atụmatụ mmadụ nwere ike ịgụ. Ha bụ ngwá ọrụ na-eduga n'ịghọta echiche nke ụdị asụsụ amụtala n'ezie.
Kedu nsogbu dị n'ime netwọkụ akwara ozi ka ndị na-akpachapụ anya autoencoders na-enyere aka dozie?
Netwọk na-ebu atụmatụ karịa akụkụ site na superposition, na-eme otu neurons polysemantic; Ndị SAE na-atụgharị ndị a ka ọ bụrụ njiri dị iche.
Kedu njirimara ụkpụrụ ụlọ na-eme ka latent SAE nwee ike ịkọwapụta?
Ntaramahụhụ sparsity (L1 ma ọ bụ top-k) na-amanye ọtụtụ nkeji latent ka ọ bụrụ efu, na-akpali otu nkeji gaa na otu, echiche monosemantic.
N'ime ọrụ 'Scaling Monosemanticity'' Anthropic, kedu ihe atụ ama ama?
Ịkwalite atụmatụ àkwà mmiri Golden Gate mere Claude jiri nlezianya na-ezo aka na akwa mmiri ahụ, na-egosi njirimara ahụ bụ ihe kpatara ya.
Kedu ihe kpatara oyi akwa zoro ezo nke SAE ji gbasaa karịa ka ntinye ntinye?
Oghere adịghị agwụ agwụ (dịka, 10-100x wider) na-enye ohere maka echiche ọ bụla iji were akụkụ nke ya.
Kedu ihe 'monosemantic' pụtara n'okwu a?
Akụkụ monosemantic na-anabata otu echiche, n'adịghị ka neurons polysemantic na-agwakọta ọtụtụ echiche ọnụ.