Autoencoder yu néew ngir dindi màndarga
Autoencoder yu néew yi dañuy ubbi mbir yu jaxasoo yi ci biir reso neuronal ci ay junni man-mani yu nit ñi mëna jàng.
Résumé
Mooy jumtukaay bi gëna am solo ngir xam ban konsept la modelu làkk jàng.
Plongeur bu xóot
Ci biir transformatër bi, benn neuron lay faral di tàkk ngir gis-gis yu bari yu wuute — muy lu ñuy woowe superposition, muy lu model bi di def ay man-man yu ëpp limu am dimension. Benn autoencoder bu bariwul (SAE) dañu koy tàggat ngir mu defaraat vecteur biy aktive ab couche ci jaare ko ci ab couche bu nëbbu bu gëna yaatu ak penalti bu bariwul, suko defee ay unité yu néew rek ñooy aktive benn yoon. Unit yooyu dañuy méngoo ak benn konsept buñ mëna tekki. Liggéeyu 2024 bi tuddu 'Escalement Monosemanticité' génne na ay milioŋ ciy màndarga ci 3 Sonnet, boole ci màndarga bu siiw bii di 'Pont de Porte d'Or'. Yokkatal ko moo tax model bi wax bu baax ci pont bi - firnde bu jub ci màndarga bi nekkoon sabab, nekkul coincidence.
Gis-gis xarala
SAE amna encoder buy màndargaal ab aktivasioŋ d-dimensionnel ci barab bu gëna mag (lu melni, 10-100x) espace latent, ab L1 wala top-k sparsity constraint buy forse latents yu bari ñu ba zero, ak ab decoder buy defaraat ab aktivasioŋ bu njëkk. Taggat yaram dafay wàññi njuumti tabaxaat boole ci daanu sparsity. Ndax diksoneer bi dafa mat te bariwul ay mbir, ay latente yu benn-benn dañuy nekk 'monosemantic' - di wut benn konsept - loolu moo tax ñu gëna mëna tekki neuron yu ñor yi.
njeextalu pexe
Gaawaay ak yaatuwaay
Liggéeyukaay yi ci làkk yi mën nañu gëna gaaw te duñu yàq deggoo gi.
Dugg ak yegg
Dafay yaatal jëfandikoo gi ci làkk yi ak ci anam yi ñuy jokkoo.
dogal yu gëna leer
Ekip yi mën nañu gëna yàgg ci àtte ci jamono ji otomatisation di liggéey ci baamtu.
Ëlëgu Autoencoder yu néew ngir dindi ay màndarga
SAEs ñu ngi màgg ci jumtukaayi kaaraange yu am solo: gis njuuj njaaj, njuuj njaaj, wala konsept yu wóorul, ak doxal doxalin ci man-mani tëju. Jafe-jafe yi des nañu - xaaj man-man yi, ñàkka tabaxaat, ak firndeel ni man-man yi mat nañu. Xaarandil pexe tàggat yu gëna yomb (top-k ak gated SAEs), etiketu màndarga otomatik, ak boole ci tablo dashboard yiy saytu model suko defee operatër yi mëna xool li model biñ dugal di 'xalaat' ci jamono dëgg.
Doxal ci àdduna dëgg
AIU_PROTECTED_4_ dindi màndarga 'Pont bu Buntu Wurus' ci Claude 3 Sonet ak di doxal model bi ci yokk ko
Xamme man-mani yi am solo ci wàllu kaaraange lu ci melni njuuj njaaj, sycophancy, wala kode vulnérabilités ci biir model activations
Dajale neuron yu am maana yu bari ci màndarga yu bari yu am benn maana ngir saafara jafe-jafe yi
Direksioŋ bu am màndarga: dafay taal wala dindi màndarga konsept ngir doxal model biy génne te doo tàggataat
Risk yi ak balustrade yi
Lépp lu jaarul yoon mën na dugg ci rapoor yi, jàppale ci liggéey bi, wala ci njariñu gëstu bi.
Sensibilite bu gaaw mën na jur njariñ yu wuute ci laajte yu noonu mel.
Done yu am solo mën nañu feeñ sudee seytu jëfandikoo gi néew doole.
Roadmap ngir samp gi
Mandargal formaa génne gi, melokaan bi, ak standard kalite yi laata ngay dugal ko.
Tontu yu am solo ak balluwaay yu wóor saa yu dëggu bi di am solo.
Fexeel am barabu xool nit ñi ngir am njariñ yu am solo.
Toppal anami gacce yi ak di faral di tàggataat ay laaj wala def-liggéey.
Weyal di banneexu
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Sparse Autoencoders for Feature Extraction quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gis bi ci topp
Autoencoder yu néew ngir mëna tekki
Laaj yi ñuy faral di laaj
Luy Autoencoder yu néew ngir dindi màndarga?
Autoencoder yu néew yi dañuy ubbi mbir yu jaxasoo yi ci biir reso neuronal ci ay junni man-mani yu nit ñi mëna jàng. Mooy jumtukaay bi gëna am solo ngir xam ban konsept la modelu làkk jàng.
Ban jafe-jafe ci biir reso neuronal yi autoencoder yu néew yi di jàppale ci saafara?
Reseau yi dañuy àndaale ak man-man yu ëpp dimension yi jaaraleko ci superposisioŋ, loolu mooy tax neuron yu benn yi am polysemantik; SAEs yi dañu leen tàqale ci màndarga yu wuute.
Lan mooy màndarga architecture biy tax ñu mëna tekki latente SAE yi?
Penalti sparsity (L1 wala top-k) dafay forse yunit yu nëbbu yi ci nul, di puus yunit yi ci benn konsept bu am benn maana.
Ci liggéeyu Anthropic ci 'Eskalaasioŋ bu benn maana', lan mooy misaalu man-man bi siiw?
Yokkateg man-manu Pont bu Golden Gate moo tax Claude di royuwaay ci pont bi, di wane ni man-man bi dafa am sabab.
Lan moo waral ñu defar layer bu nëbbu bi ci SAE mu gëna yaatu ci dugal bi?
Barab bu nëbbu bu mat te ëpp (lu melni, 10-100x lu gëna yaatu) dafay may konsept bu nekk palaas ngir jël dimension boppam.
Luy 'monosemantik' ci anam yii?
Màndarga monosemantik dafay tontu benn konsept, wuute na ak neuron polysemantik yiy jaxase konsept yu bari ñoom ñaar.