Ubuyobozi bwa tekiniki

Imashini zidasanzwe zo gusobanura

Imodoka zidasanzwe (SAEs) nigikoresho gikurura ibikorwa byimbere byimbere byurusobe rwimitsi mumurongo munini cyane wibintu bisukuye, bisobanurwa nabantu.

Umva iki gitaboAI-byavuzwe · ~2 min

Kuriyi page2 min soma
  1. Incamake
  2. Kwibira cyane
  3. Gushyira mu bikorwa Isi
  4. Kazoza ka Autoencoders idasanzwe yo gusobanura
  5. Kubishyira mubikorwa
  6. Komeza Ubushakashatsi
  7. Ibibazo bikunze kubazwa

Incamake

They are one of the leading techniques for opening the 'black box' and seeing what concepts a model actually represents.

Kwibira cyane

Imbere ya transformateur, icyerekezo kimwe cyo gukora ivanga hamwe nibihumbi nibihumbi icyarimwe, bigatuma bigorana gusoma. Autoencoder ni gake ni urusobe ruto rwibice bibiri byatojwe kongera kubaka ibyo bikorwa binyuze murwego rwagutse rwihishe, ariko hamwe nigihano gike gihatira bake gusa muri neuron nyinshi zacyo kurasa icyarimwe. Kubera uwo muvuduko, buri gice cyihishe gikunda kuba umuhanga mubitekerezo bimwe, nka 'kuvuga ikiraro cya Zahabu Zahabu' cyangwa 'code ya Python'. Muri 2024 Anthropic yapimye ibi kuri Claude 3 Sonnet, ikuramo ibintu bigera kuri miriyoni 34, kandi OpenAI na DeepMind byasohoye imirimo ibangikanye SAE. Abashakashatsi barashobora noneho gufunga ikintu hejuru cyangwa hepfo kugirango bagerageze neza icyo ikora.

Gushyira mu bikorwa Isi

__AIU_PROTECTED_4 __ 'Irembo rya Zahabu Claude' demo, aho kongera imbaraga imwe ya SAE byatumye icyitegererezo cyerekana ikiraro mubisubizo byose

Gukuramo no gushyira akamenyetso hafi miriyoni 34 ziranga Claude 3 Sonnet kugirango ushushanye amakarita nka sycophancy, amakosa ya code, nimyitwarire idakwiye.

Gushakisha ibintu bijyanye n'umutekano nko kubeshya, kubogama, cyangwa ibintu bishobora guteza akaga bishobora gukurikiranwa cyangwa kuyobora mugihe cyoherejwe

Gukemura impanvu icyitegererezo cyerekana nabi inyongeramusaruro mugusuzuma ibintu bisobanurwa byakorewe mugihe runaka

Kazoza ka Autoencoders idasanzwe yo gusobanura

Tegereza SAEs kuva mumatsiko yubushakashatsi yerekeza kubigenzuzi bifatika no gukoresha ibikoresho byumutekano, harimo imbaho ​​zerekana ibimenyetso biranga ibimenyetso byerekana uburiganya cyangwa umutekano muke. Ibibazo byafunguye birimo 'gutandukanya ibice' (igitekerezo kimwe kigabanyamo byinshi), kubura ibintu, hamwe nigiciro cyamahugurwa SAEs kuri buri cyiciro cyurugero rwimipaka. Icyerekezo gishya nka crosscoders, transcoders, na matryoshka SAEs bigamije gufata ibarwa mubice kandi kuri granularies icyarimwe.

Kubishyira mubikorwa

Impamvu ari ngombwa

  • Igiciro na bije. Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
  • Ibyemezo bisobanutse. Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
  • Kugenzura ubuziranenge. Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.

Witondere

  • Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
  • Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
  • Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.

Intambwe ikurikira

  1. Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
  2. Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
  3. Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
  4. Tegura inzira yo gusubiza ibyabaye mbere yo gupima.

Komeza Ubushakashatsi

Ibaruwa y'amakuru y'ubuntu

Ibuka amakuru ya buri munsi ya AI

Inkuru eshatu zizewe za AI buri gitondo cy'iminsi y'akazi, zanditse mu rurimi rworoshye. Ubuntu iteka ryose, nta kwamamaza.

Imeri imwe buri munsi y'akazi. Kwiyimura mu buryo bumwe. Ntitwigera tugurisha cyangwa ngo dusangire aderesi yawe.

Isuzume

Take the Sparse Autoencoders for Interpretability quiz

Ibisubizo ako kanya kuri buri kibazo, n'icyemezo gishobora gusangirwa gifite ID yagenzurwa umaze gutsinda isomo.

Tangira ikibazo

Fasha uburezi bwa AI bw'ubuntu. AI Understanding ni umuryango utegamiye kuri leta wa 501(c)(3) — nta kwamamaza, nta kwishyura, iteka ryose. Tanga impano

Ibibazo bikunze kubazwa

What is Sparse Autoencoders for Interpretability?

Imodoka zidasanzwe (SAEs) nigikoresho gikurura ibikorwa byimbere byimbere byurusobe rwimitsi mumurongo munini cyane wibintu bisukuye, bisobanurwa nabantu. Nimwe mubuhanga buyobora mugukingura 'agasanduku kirabura' no kureba icyo icyitegererezo cyerekana.

Niyihe ntego nyamukuru yo guhugura autoencoder idasanzwe kubikorwa bya moderi?

SAEs yubaka ibikorwa ikoresheje ubugari, buke bwihishe kuburyo ibice bitandukanye bikunda guhuza nibitekerezo byumvikana byabantu.

How does an SAE’s sparsity penalty affect hidden-unit activations?

Sparsity encourages feature-selective activity but does not guarantee perfect one-neuron-per-concept representations.

Hafi y'ibintu bingahe Anthropic yakuyemo mugihe yapimye SAE kuri Claude 3 Sonnet muri 2024?

__AIU_PROTECTED_4 __ '2024' Scaling Monosemanticity 'yakuwe ku rutonde rwa miliyoni 34 ziranga icyitegererezo.

Ni iki 'Irembo rya Zahabu Claude' ryerekanaga?

Mugukomeza ibiraro bya Golden Gate Bridge, abashakashatsi bakoze icyitegererezo gikosora ikiraro, berekana ibimenyetso nibitera impamvu, ntabwo ari ibirango gusa.

Ni ukubera iki igicucu cyihishe muri SAE mubisanzwe CYANE kuruta ibikorwa byubaka?

Abanyamideli bapakira ibintu byinshi mubipimo bigarukira (superposition); birenze, bigari SAE itanga buri cyumba cyicyerekezo kugirango gikore igice cyacyo.