Sparse Autoencoders don Filayen Haɓaka
Rarraba autoencoders sun fashe suna buɗe ruɗaɗɗen kunnawa a cikin hanyar sadarwa ta jijiyoyi zuwa dubban fasalulluka waɗanda mutum zai iya karantawa.
Dubawa
They are the leading tool for understanding what concepts a language model has actually learned.
Zurfafa nutsewa
A cikin na'ura mai canzawa, neuron guda ɗaya yakan ƙone don yawancin ra'ayoyin da ba su da alaƙa - al'amari da ake kira superposition, inda samfurin ya ƙunshi ƙarin fasali fiye da yadda yake da girma. An horar da ƙwaƙƙwarar autoencoder (SAE) don sake gina vector na kunna Layer ta hanyar wuce shi ta wani ɓoye mai faɗi mai fa'ida tare da hukumci mai raɗaɗi, don haka kaɗan ne kawai ke kunna raka'a. Waɗannan raka'o'in sun yi daidai da guda ɗaya, ra'ayoyin da za a iya fassarawa. Aikin 'Scaling Monosemanticity' na 2024 Anthropic ya fitar da miliyoyin siffofi daga Claude 3 Sonnet, gami da sanannen fasalin 'Golden Gate Bridge'. Ƙaddamar da shi ya sa ƙirar ta ambaci gadar cikin damuwa - shaida kai tsaye fasalin fasalin ya kasance sanadi, ba daidaituwa ba.
Fahimtar Fasaha
SAE yana da encoder wanda ke yin taswirar kunnawa-d-dimensional zuwa cikin mafi girma (misali, 10-100x) sararin ɓoye, L1 ko babban-k takurawa da ke tilasta mafi yawan latents zuwa sifili, da mai ƙididdigewa wanda ke sake gina ainihin kunnawa. Horarwa yana rage girman kuskuren sake ginawa tare da hukumcin da bai dace ba. Saboda ƙamus ɗin ya cika kuma ba kaɗan ba, latent ɗin mutum ɗaya ya zama 'monosemantic' - harbe-harbe don ra'ayi ɗaya - yana mai da su mafi fa'ida fiye da ɗanyen neurons.
Dabarun Tasiri
Gudu da sikelin
Gudun aikin harshe na iya tafiya da sauri ba tare da sadaukar da daidaito ba.
Shiga ku isa
Yana faɗaɗa damar shiga cikin harsuna da salon sadarwa.
Shawarwari masu haske
Ƙungiyoyi za su iya ciyar da ƙarin lokaci akan hukunci yayin da aiki da kai ke sarrafa maimaitawa.
Makomar Sparse Autoencoders don Haɓaka Siffar
SAEs suna girma cikin kayan aikin aminci masu amfani: gano yaudara, son zuciya, ko ra'ayoyi mara kyau, da halayyar tuƙi ta hanyar matsi. Kalubale sun kasance - rarrabuwar fasali, asarar sake ginawa, da tabbatar da cewa fasalin sun cika. Yi tsammanin hanyoyin horarwa masu rahusa (top-k da gated SAEs), alamar alama ta atomatik, da haɗin kai cikin dashboards na sa ido don masu aiki su iya tantance abin da ƙirar da aka tura ke 'tunanin' a ainihin lokacin.
Aiwatar da Gaskiyar Duniya
Anthropic Ciro fasalin 'Golden Gate Bridge' daga Claude 3 Sonnet da sarrafa samfurin ta haɓaka shi.
Gano abubuwan da suka dace da aminci kamar ha'inci, sycophancy, ko raunin lambar a cikin kunna samfurin
Rarraba jijiyoyi na polysemantic zuwa cikin abubuwan monosemantic da yawa don warware babban matsayi
Siffar tuƙi: ƙulla fasalin ra'ayi a kunne ko kashe don sarrafa abubuwan samfuri ba tare da sake horarwa ba
Hatsari & Tsare-tsare
Abubuwan da aka ruɗe suna iya shigar da rahotanni cikin nutsuwa, kwararar tallafi, ko abubuwan bincike.
Hankali na gaggawa na iya ƙirƙirar sakamako mara daidaituwa a cikin buƙatun iri ɗaya.
Za a iya fallasa bayanan rubutu mai ma'ana idan ikon samun dama yana da rauni.
Taswirar Hanya
Ƙayyade tsarin fitarwa, sautin, da ma'auni masu inganci kafin fitowa.
Amsa a ƙasa tare da amintattun tushe a duk lokacin da daidaito ya shafi mahimmanci.
Ajiye wurin binciken ɗan adam don abubuwan da ake samu masu girma.
Bibiyar tsarin gazawar kuma sake horar da tsokaci ko tafiyar aiki akai-akai.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Sparse Autoencoders for Feature Extraction quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Rarraba Autoencoders don Fassara
Tambayoyin da ake yawan yi
What is Sparse Autoencoders for Feature Extraction?
Rarraba autoencoders sun fashe suna buɗe ruɗaɗɗen kunnawa a cikin hanyar sadarwa ta jijiyoyi zuwa dubban fasalulluka waɗanda mutum zai iya karantawa. Su ne manyan kayan aiki don fahimtar menene ra'ayoyin ƙirar harshe a zahiri ya koya.
Wace matsala ce a cikin cibiyoyin sadarwa na jijiyoyi ke taimaka wa masu amfani da autoencoders da yawa?
Cibiyoyin sadarwa suna ɗaukar ƙarin fasali fiye da girma ta hanyar babban matsayi, suna yin polysemantic neurons guda ɗaya; SAEs suna kwance waɗannan zuwa fasali daban-daban.
Menene fasalin gine-ginen da ke sa latent SAE ke fassarawa?
Hukuncin raɗaɗi (L1 ko top-k) yana tilasta mafi yawan rukunan raka'a zuwa sifili, suna tura raka'a ɗaya zuwa ɗaya, ra'ayoyin monosemantic.
A cikin aikin 'Scaling Monosemanticity' na Anthropic, menene sanannen fasalin misali?
Ƙaddamar da fasalin gadar Golden Gate ya sanya Claude nuna damuwa ga gadar, yana nuna fasalin shine sanadi.
Me yasa rufin ɓoye na SAE ya fi faɗi fiye da kunna shigarwar?
Wurin da bai cika cika ba (misali, 10-100x mai faɗi) sarari mara kyau yana ba da ɗaki ga kowane ra'ayi don mamaye girmansa.
Menene ma'anar 'monosemantic' a cikin wannan mahallin?
Siffar monosemantic tana ba da amsa ga ra'ayi ɗaya, sabanin jijiyoyi na polysemantic waɗanda ke haɗa ra'ayoyi da yawa tare.