Imashini zidasanzwe zo gusobanura
Imodoka zidasanzwe (SAEs) nigikoresho gikurura ibikorwa byimbere byimbere byurusobe rwimitsi mumurongo munini cyane wibintu bisukuye, bisobanurwa nabantu.
Incamake
They are one of the leading techniques for opening the 'black box' and seeing what concepts a model actually represents.
Kwibira cyane
Imbere ya transformateur, icyerekezo kimwe cyo gukora ivanga hamwe nibihumbi nibihumbi icyarimwe, bigatuma bigorana gusoma. Autoencoder ni gake ni urusobe ruto rwibice bibiri byatojwe kongera kubaka ibyo bikorwa binyuze murwego rwagutse rwihishe, ariko hamwe nigihano gike gihatira bake gusa muri neuron nyinshi zacyo kurasa icyarimwe. Kubera uwo muvuduko, buri gice cyihishe gikunda kuba umuhanga mubitekerezo bimwe, nka 'kuvuga ikiraro cya Zahabu Zahabu' cyangwa 'code ya Python'. Muri 2024 Anthropic yapimye ibi kuri Claude 3 Sonnet, ikuramo ibintu bigera kuri miriyoni 34, kandi OpenAI na DeepMind byasohoye imirimo ibangikanye SAE. Abashakashatsi barashobora noneho gufunga ikintu hejuru cyangwa hepfo kugirango bagerageze neza icyo ikora.
Ubushishozi
SAE ishushanya ibikorwa bya d-bipima muburyo bwagutse cyane (akenshi 8x kugeza 100x binini), hanyuma ikubaka umwimerere. Amahugurwa agabanya ikosa ryo kwiyubaka hiyongereyeho L1 ibihano kubikorwa byihishe, bitera imbaraga nke kuburyo ibice byinshi biguma hafi ya zeru. Ibihinduka nka TopK SAEs byubahiriza muburyo butaziguye mugukomeza ibikorwa bya K binini gusa, kandi amarembo SAE atandukanya icyemezo cyo kurasa nubunini, kugabanya kubogama kuri gahunda L1 itangiza.
Ingaruka z'Ingamba
Igiciro na bije
Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
Ibyemezo bisobanutse
Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
Kugenzura ubuziranenge
Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.
Kazoza ka Autoencoders idasanzwe yo gusobanura
Tegereza SAEs kuva mumatsiko yubushakashatsi yerekeza kubigenzuzi bifatika no gukoresha ibikoresho byumutekano, harimo imbaho zerekana ibimenyetso biranga ibimenyetso byerekana uburiganya cyangwa umutekano muke. Ibibazo byafunguye birimo 'gutandukanya ibice' (igitekerezo kimwe kigabanyamo byinshi), kubura ibintu, hamwe nigiciro cyamahugurwa SAEs kuri buri cyiciro cyurugero rwimipaka. Icyerekezo gishya nka crosscoders, transcoders, na matryoshka SAEs bigamije gufata ibarwa mubice kandi kuri granularies icyarimwe.
Gushyira mu bikorwa Isi
__AIU_PROTECTED_4 __ 'Irembo rya Zahabu Claude' demo, aho kongera imbaraga imwe ya SAE byatumye icyitegererezo cyerekana ikiraro mubisubizo byose
Gukuramo no gushyira akamenyetso hafi miriyoni 34 ziranga Claude 3 Sonnet kugirango ushushanye amakarita nka sycophancy, amakosa ya code, nimyitwarire idakwiye.
Gushakisha ibintu bijyanye n'umutekano nko kubeshya, kubogama, cyangwa ibintu bishobora guteza akaga bishobora gukurikiranwa cyangwa kuyobora mugihe cyoherejwe
Gukemura impanvu icyitegererezo cyerekana nabi inyongeramusaruro mugusuzuma ibintu bisobanurwa byakorewe mugihe runaka
Ingaruka & Kurinda
Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.
Igishushanyo mbonera
Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
Tegura inzira yo gusubiza ibyabaye mbere yo gupima.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Sparse Autoencoders for Interpretability quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Imashini zidasanzwe zo gukuramo ibintu
Ibibazo bikunze kubazwa
What is Sparse Autoencoders for Interpretability?
Imodoka zidasanzwe (SAEs) nigikoresho gikurura ibikorwa byimbere byimbere byurusobe rwimitsi mumurongo munini cyane wibintu bisukuye, bisobanurwa nabantu. Nimwe mubuhanga buyobora mugukingura 'agasanduku kirabura' no kureba icyo icyitegererezo cyerekana.
Niyihe ntego nyamukuru yo guhugura autoencoder idasanzwe kubikorwa bya moderi?
SAEs yubaka ibikorwa ikoresheje ubugari, buke bwihishe kuburyo ibice bitandukanye bikunda guhuza nibitekerezo byumvikana byabantu.
Nigute SAE ishishikariza buri gice cyihishe guhagararira igitekerezo kimwe?
Igihano gike (nk'ijambo L1 cyangwa imbogamizi ya TopK) gihatira ibice byinshi byihishe kuguma hafi ya zeru, bigasunika buri gice gikora kubisobanuro byihariye.
Hafi y'ibintu bingahe Anthropic yakuyemo mugihe yapimye SAE kuri Claude 3 Sonnet muri 2024?
__AIU_PROTECTED_4 __ '2024' Scaling Monosemanticity 'yakuwe ku rutonde rwa miliyoni 34 ziranga icyitegererezo.
Ni iki 'Irembo rya Zahabu Claude' ryerekanaga?
Mugukomeza ibiraro bya Golden Gate Bridge, abashakashatsi bakoze icyitegererezo gikosora ikiraro, berekana ibimenyetso nibitera impamvu, ntabwo ari ibirango gusa.
Ni ukubera iki igicucu cyihishe muri SAE mubisanzwe CYANE kuruta ibikorwa byubaka?
Abanyamideli bapakira ibintu byinshi mubipimo bigarukira (superposition); birenze, bigari SAE itanga buri cyumba cyicyerekezo kugirango gikore igice cyacyo.