Rarraba Autoencoders don Fassara
Sparse autoencoders (SAEs) kayan aiki ne wanda ke raba abubuwan kunnawa na cikin gida da aka ruɗe na cibiyar sadarwar jijiyoyi zuwa mafi girman saiti na tsafta, fasalulluka masu iya fassarawa mutum.
Dubawa
They are one of the leading techniques for opening the 'black box' and seeing what concepts a model actually represents.
Zurfafa nutsewa
A cikin na'ura mai ba da wuta, nau'in kunnawa guda ɗaya yana haɗuwa tare da dubban ra'ayoyi a lokaci ɗaya, wanda ke da wuyar karantawa. Ƙaƙƙarfan autoencoder ƙaramar hanyar sadarwa ce mai Layer biyu wacce aka horar da ita don sake gina waɗannan kunnawa ta hanyar faffadan ɓoyayyiyar ɓangarorin, amma tare da hukumcin ɓacin rai wanda ke tilastawa kaɗan daga cikin jijiya masu yawa yin wuta a lokaci guda. Saboda wannan matsin lamba, kowane ɗayan ɓoyayyun yana ƙoƙarin ƙware a cikin ra'ayi ɗaya, kamar 'ambaton Gadar Golden Gate' ko 'Python code'. A cikin 2024 Anthropic sun daidaita wannan zuwa Claude 3 Sonnet, yana fitar da kusan siffofi miliyan 34, da OpenAI da DeepMind sun buga layi daya na aikin SAE. Masu bincike za su iya matsawa sama ko ƙasa don gwada abin da yake yi.
Fahimtar Fasaha
SAE taswirar kunna girman d-dimensional zuwa cikin ɓoye mai faɗi da yawa (sau da yawa 8x zuwa 100x ya fi girma), sannan ya sake gina ainihin. Horon yana rage kuskuren sake ginawa tare da hukuncin L1 akan abubuwan kunnawa na ɓoye, wanda ke ƙarfafa rashin ƙarfi don haka yawancin raka'a suna kusa da sifili. Bambance-bambancen kamar TopK SAEs suna tilasta rashin ƙarfi kai tsaye ta hanyar kiyaye manyan ayyukan K kawai, kuma gated SAEs sun raba yanke shawarar yin wuta da girma, rage tsarin son rai L1 yana gabatarwa.
Dabarun Tasiri
Kudin da kasafin kuɗi
Hukunce-hukuncen gine-gine suna haifar da aiki da tsadar aiki na shekaru.
Shawarwari masu haske
Ilimin fasaha yana taimaka wa ƙungiyoyi su zaɓi tari mai kyau, ba kawai sabon abu ba.
Kula da inganci
Zaɓuɓɓukan injiniya mafi kyau suna rage abin dogaro a cikin samarwa.
Makomar Sparse Autoencoders don Fassara
Yi tsammanin SAEs don matsawa daga sha'awar bincike zuwa aikin dubawa mai amfani da kayan aikin aminci, gami da dashboards waɗanda ke yiwa alama alama da gano da'irar yaudara ko mara tsaro. Matsalolin buɗewa sun haɗa da 'tsararriyar fasalin' (ra'ayi ɗaya yana ɓarna cikin mutane da yawa), abubuwan da suka ɓace, da tsadar horar da SAEs akan kowane ƙirar iyaka. Sabbin kwatance kamar crosscoders, transcoders, da matryoshka SAEs suna nufin ɗaukar ƙididdigewa a cikin yadudduka kuma a manyan ƙididdiga masu yawa a lokaci ɗaya.
Aiwatar da Gaskiyar Duniya
Anthropic's 'Golden Gate Claude' demo, inda haɓaka fasalin SAE guda ɗaya ya sanya ƙirar ta yi la'akari da gada a kowace amsa.
Ciro da sanyawa kusan siffofi miliyan 34 daga Claude 3 Sonnet zuwa taswirar ra'ayoyi kamar sycophancy, kurakuran lamba, da halayen rashin aminci.
Nemo abubuwan da suka dace da aminci kamar yaudara, son zuciya, ko abun ciki mai haɗari waɗanda za'a iya sa ido ko jagora yayin turawa.
Gyara dalilin da yasa samfurin ke ɓarna bayanai ta hanyar duba waɗanne fasalolin da za a iya fassarawa aka kunna akan abin da aka bayar
Hatsari & Tsare-tsare
Haɓaka ma'auni ɗaya na iya ɓoye manyan raunin tsarin.
Sau da yawa ana raina kayan more rayuwa da kuma kuɗin kulawa.
Tsaro da gibin lura na iya girma yayin da tsarin ke ƙara haɓaka.
Taswirar Hanya
Ƙayyade latency, inganci, da maƙasudin farashi kafin aiwatarwa.
Alamar ma'auni a ƙarƙashin ainihin kaya da yanayin bayanai.
Kula da kayan aiki don kurakurai, ɗigo, da tasirin mai amfani.
Shirya bijirowa da hanyoyin mayar da martani kafin sikeli.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Sparse Autoencoders for Interpretability quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Sparse Autoencoders don Filayen Haɓaka
Tambayoyin da ake yawan yi
What is Sparse Autoencoders for Interpretability?
Sparse autoencoders (SAEs) kayan aiki ne wanda ke raba abubuwan kunnawa na cikin gida da aka ruɗe na cibiyar sadarwar jijiyoyi zuwa mafi girman saiti na tsafta, fasalulluka masu iya fassarawa mutum. Suna ɗaya daga cikin manyan dabaru don buɗe 'baƙin akwatin' da ganin menene ainihin abin ƙira ke wakilta.
Menene babban manufar horar da ƙwaƙƙwaran autoencoder akan abubuwan kunna samfurin?
SAEs suna sake gina abubuwan kunnawa ta hanyar faffadan faffadan buyayyar buyayyar wuri, ta yadda raka'a guda daya sukan yi daidai da ra'ayoyi guda daya da mutum zai iya fahimta.
Ta yaya SAE ke ƙarfafa kowane rukunin ɓoye don wakiltar ra'ayi ɗaya?
Hukuncin raɗaɗi (kamar kalmar L1 ko ƙuntatawa na TopK) yana tilasta yawancin ɓoyayyun raka'a su tsaya kusa da sifili, suna tura kowace naúrar aiki zuwa ma'ana ta musamman.
Kusan fasali nawa ne Anthropic ta fitar lokacin da aka daidaita SAEs zuwa Claude 3 Sonnet a 2024?
Anthropic's 2024 'Scaling Monosemanticity' aikin da aka fitar akan tsari na fasali miliyan 34 daga samfurin samarwa.
Menene zanga-zangar 'Ƙofar Zinariya Claude' ta nuna?
Ta hanyar haɓaka fasalin gadar Golden Gate, masu bincike sun sanya ƙirar ta daidaita akan gadar, suna nuna fasalulluka ne masu fa'ida, ba kawai alamomi ba.
Me yasa rufin ɓoye a cikin SAE yawanci ya fi fadi fiye da kunnawa da yake sake ginawa?
Samfuran suna tattara ra'ayoyi da yawa zuwa ƙayyadaddun ma'auni (superposition); wani cikakku, faffadan SAE yana ba kowane ɗakin ra'ayi ya mamaye nasa naúrar.