Superposition na Polysemanticity na AI nkọwa
Superposition na ntụgharị uche AI bụ ụzọ netwọkụ akwara na-achịkọta ọtụtụ atụmatụ n'ime ntụzịaka nkekọrịta, nke na-eme ka neurons n'otu n'otu yie polysemantic ma sie ike ịkọwa.
Ime miri emi
Data ụwa nwere njiri mara oke uru karịa oyi akwa nwere akụkụ, yabụ netwọk na-akpakọba ha. Na superposition, ihe nlereanya na-anọchi anya atụmatụ dị ka ihe fọrọ nke nta-orthogonal ntụzịaka na oghere oghere kama ịrara otu neuron kwa njiri mara. Nke a na-arụ ọrụ n'ihi na ọtụtụ njiri mara obere (adịghị adịkebe n'otu oge), yabụ nnyonye anya oge ụfọdụ bụ ọnụ ahịa anabatara. Nsonaazụ bụ neurons polysemantic: Anthropic's 'Toy Models of Superposition' (2022) gosiri otu neuron na-agba maka, sịnụ, ihu pusi, ihu ụgbọ ala, na ụfọdụ ụkpụrụ ederede. N'ụzọ dị mkpa, netwọk nwere ike ịrụ ọtụtụ mgbakọ karịa ka ọ nwere neurons, mana ọ bụ naanị mgbe njirimara na-adịchaghị nke ọma na nkukota dị ụkọ.
Nghọta nka nka
Geometrically, ọ bụrụ na ị ga-echekwabara n atụmatụ n'akụkụ m nwere n ka m karịa m, ị nweghị ike idobe ha niile n'usoro. Ihe nlereanya ahụ na-ahazi ha dị ka ọtụtụ ihe fọrọ nke nta ka ọ bụrụ-orthogonal vectors, na-anabata obere nnyonye anya. Ụdị ihe egwuregwu ụmụaka na-ekpughe geometry a haziri ahazi dị ka ụzọ abụọ antipodal na pentagon. Sparsity bụ ọnọdụ na-enye ohere: mgbe naanị njirimara ole na ole na-ere ọkụ n'otu oge, nnyonye anya a na-atụ anya na-adị ala, ya mere uru nke ịnọchite anya atụmatụ ndị ọzọ na-akawanye ụda.
Mmetụta atụmatụ
Ọnụ ego na mmefu ego
Mkpebi ihe owuwu ụlọ na-akwalite arụmọrụ yana ọnụ ahịa ọrụ ruo ọtụtụ afọ.
Mkpebi doro anya
Nkà mmụta nka na-enyere ndị otu egwuregwu aka ịhọrọ nchịkọta ziri ezi, ọ bụghị naanị nke kachasị ọhụrụ.
Quality akara
Nhọrọ injinia ka mma na-ebelata ihe omume ntụkwasị obi na mmepụta.
Ọdịnihu nke Superposition na Polysemanticity na AI nkọwa
Ịghọta ihe dị elu bụ ntọala maka ịkọwapụta: ndị na-akpachapụ anya autoencoders dị kpọmkwem iji megharịa ya. Ọrụ ga-eme n'ọdịnihu bu n'obi ịkọ mgbe na otu ụdị si abanye superposition, chepụta architectures nke na-ebelata nnyonye anya na-emerụ ahụ, na tụọ oke njirimara ole enwere ike ịjupụta n'enweghị nsogbu. Ọ bụrụ na ndị na-eme nchọpụta nwere ike ịtụkwasị obi 'gbasawanye' superposition n'ime njirimara monosemantic n'ọtụtụ, ụdị nyocha maka sekit adịghị mma na-aghọwanye traktị karịa, na-atụgharị igbe ojii ka ọ bụrụ ihe dị nso na koodu enwere ike ịgụ.
Mmejuputa n'ezie n'ụwa
_AIU_PROTECTED_4__'s 2022 'Ụdị egwuregwu ụmụaka nke Superposition' na-egosi mkpọkọta atụmatụ a na-achịkwa ka ọ na-abawanye.
Ọhụụ neurons na InceptionV1 na-anabata ọtụtụ ihe na-enweghị njikọ, ikpe mara mma nke polysemanticity
N'ịkọwa ihe kpatara nyocha neuron otu ụdị asụsụ na-enye nsonaazụ mgbagwoju anya, agwakọtara n'ofe isiokwu
Na-akpali akpali autoencoders, nke dị kpọmkwem iji mebie ọrụ ndị a na-ebuli elu azụ n'ime otu echiche.
Ihe ize ndụ & okporo ụzọ nche
Ịkwalite otu akara ngosi nwere ike zoo adịghị ike sistemụ sara mbara.
A na-eledakarị ihe akụrụngwa na ụgwọ ọrụ anya.
Ọdịiche nchekwa na nleba anya nwere ike itolite ka sistemu na-adịwanye mgbagwoju anya.
Map mmejuputa
Kọwaa latency, ịdịmma na ebumnuche ọnụ ahịa tupu mmejuputa ya.
Benchmark n'okpuru ibu dị adị na ọnọdụ data.
Nleba anya akụrụngwa maka mperi, ịkpafu na mmetụta onye ọrụ.
Kwadebe ụzọ nzaghachi azụghachi azụ na ihe omume tupu ịchachaa.
Nọgide na-eme nchọpụta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Superposition and Polysemanticity in AI Interpretability quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ntuziaka na-esote
DeepSpeed na Megatron Ọzụzụ Stacks
Ajụjụ a na-ajụkarị
What is Superposition and Polysemanticity in AI Interpretability?
Superposition na ntụgharị uche AI bụ ụzọ netwọkụ akwara na-achịkọta ọtụtụ atụmatụ n'ime ntụzịaka nkekọrịta, nke na-eme ka neurons n'otu n'otu yie polysemantic ma sie ike ịkọwa.
Kedu ihe ' superposition' na-ezo aka na netwọkụ akwara?
Superposition bụ atụmatụ nke idebe njirimara dị iche iche karịa akụkụ site na iji ihe fọrọ nke nta ka ọ bụrụ orthogonal, ntụzịaka kpuchiri ekpuchi na oghere mmalite.
Kedu ọnọdụ na-eme superposition na-arụ ọrụ nke ọma n'agbanyeghị nnyonye anya atụmatụ?
N'ihi na ọtụtụ atụmatụ anaghị arụ ọrụ n'otu oge, nkukota anaghị adịkarị, yabụ ọnụ ahịa nnyonye anya na-adị ala.
Kedu ihe bụ neuron 'polysemantic'?
Polysemantic neurons na-ere ọkụ maka ọtụtụ njirimara na-enweghị njikọ, nke bụ ihe a na-ahụ anya nke superposition.
Kedu akwụkwọ Anthropic ewepụtara ọmụmụ ihe a na-achịkwa nke ọma na 2022?
'Ụdị egwuregwu ụmụaka nke Superposition' jiri obere netwọk nwere ike ịchịkwa iji gosi mgbe na otu esi ejikọta atụmatụ ọnụ.
Ọ bụrụ na oyi akwa ga-echekwa atụmatụ karịa ka o nwere akụkụ, kedu ihe geometrically na-agaghị ekwe omume?
Ịnweghị ike ịkwado vectors orthogonal karịa akụkụ, yabụ njirimara na-ejedebe dị ka ntụziaka nke fọrọ nke nta ka ọ bụrụ orthogonal na ụfọdụ ndakọrịta.