Tekki mekanik
Tekkib mekanik mooy jéema soppi xayma yi ci biir reso neuronal yi ci algorithm yu nit ñi mëna xam.
Résumé
Rather than asking 'which input mattered,' it asks 'what is this network actually computing, circuit by circuit?'
Plongeur bu xóot
Fépp fu pexe yu melni SHAP di leeral duggal ak genn, tekki mekanistik ubbi boyet bi ba noppi jàngat poid yi ak aktivasioŋ yi ci seen bopp. Gëstukat yi (espesialeman ci __AIU_PROTECTED_4_, OpenAI, OpenAI, ak ci lekkon yi) dañu jàpp transformatër ni prograam buñu wara dekompile, di xàmmee 'circuit' yi: subgraph yu neuron yi ak boppu attention yiy jëfandikoo benn fonction. Li ñu gis ci gis-gis yi am solo mooy 'boppu induksioŋ,' boppu bàyyi xel yuy koppi motif yi ngir mëna jàng ci biir muy tekki, ak gis-gis ni neuron yu benn yi dañuy faral di nekk 'polysemantic,' di firi konsept yu bari yu amul benn lëkkaloo ndax model bi dafa ëpp man-man yu ëpp dimension (superposition). Leegi dañuy jëfandikoo autoencoder yu néew ngir dindi leen ci 'man-man' yu gëna set, yu am benn maana, lu melni yoon wiy tàmbali ci Golden Gate Bridge.
Gis-gis xarala
Benn ci gaañ-gaañu yi mooy superposition: ab reso bu am d dimension mën na representé lu ëpp d ci denc leen ci yoon yu jege-orthogonal, kon neuron yi benn-benn dañuy tàkk ngir konsept yu amul benn lëkkaloo. Sparse autoencoders yi deñuy saafara jafe-jafe yii ci jàng ab diksoneer bu mat lool buy defaraat ay aktivaasioŋ yu jëfandikoo ay yunit yu néew ci benn yoon, di wane ay man-mani yuñ mëna tekki. Gëstukat yi dañuy saytu sircuit yi ci ay intervention causal, ablating wala 'patching' activations ngir firndeel ni benn composant defna dëgg-dëgg calcul hypothèse bi.
njeextalu pexe
Njëgg ak budget
Dogal yi architecture di jël dañuy indi njariñ ak njëgu liggéey bi ay at ci ginaaw.
dogal yu gëna leer
Njàngalem xarala yi dafay jàppale ekip yi ñu tànn li gën, te baña yam ci li gëna bees daal.
Xool kalite
Tanneef yu gëna baax ci wàllu ingeñër dina wàññi jafe-jafe yi ci wàllu wóor ci liggéey bi.
Ëlëgu Tekki Mekanik
Tekki mekanik lu am solo la ci kaaraange IA: xam biir yi mën nanu may nu saytu ay xeetu njuuj njaaj, gis kàttan yu mëna jur loraange, ak doxal doxalin ci soppali man-man yi ci saasi. Liggéey bi ñuy amal ci diir bu gàtt mingi sëssé ci yokk autoencoder yu bari yi ci model frontiere yi, gis-gis sircuit yi ci otomatik, ak tabax 'diksoneer yu am màndarga' yu wóor. Luñu bëgga mooy 'MRI ngir reso neuronal', anam wu ñuy jàngee xalaatu benn model bala ñu koy dugal, waaye tekki sistem yu am ay miliyaar ciy parametre ci eskaal mingi wéy di nekk jafe-jafe bu mag.
Doxal ci àdduna dëgg
Anthropic génne ay milioŋ ci màndarga yiñ mëna tekki ci Claude ba noppi wane ni yokk benn màndarga 'Golden Gate Bridge' moo tax model bi dafay wax ci pont bi, di wane ni ñuy doxalee ci anam wu jub.
Gëstukat yi gis nañu 'boppu induksioŋ' ci transformatër yiy kopie ak wéyal motif token yu ñuy baamtu, di leeral mekaniism bu am solo bi ci ginaaw jàng ci biir contexte.
Activation patching dañu koy jëfandikoo ngir xam fu model bi denc ab mbir (lu melni, kapitaalu benn réew), di wane ay layer ak ay composant yu ko waral.
Ekipu kaaraange yi dañuy saytu man-mani yi ci biir ngir xam ndax model bi dafay màndargaal ay konsept yu melni njuuj njaaj wala tegtal yu wóorul, loolu mooy tax ñu mëna saytu wala jàppale ci anam wu jaar yoon.
Risk yi ak balustrade yi
Optimize benn benchmark mën na nëbb ñakk kattan yu gëna yaatu ci sistem bi.
Njëg li ñuy fay ci infrastructure yi ak ci toppatoo dañuy faral di suufeel.
Bu sistem yi di gëna xawa jafee xam, jafe-jafe yi am ci wàllu kaaraange ak seetlu mën nañu gëna bari.
Roadmap ngir samp gi
Mandargal latency, kalite, ak njëg yi laata ngay jëfandikoo.
Benchmark ci biir sargal ak done yu dëggu.
Jumtukaay bi di saytu njuumte yi, derive bi ak njeextalu jëfandikukat bi.
Waajal rollback ak yooni tontu ci jafe-jafe yi laata ngay eskale.
Weyal di banneexu
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Mechanistic Interpretability quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gis bi ci topp
Autoencoder yu néew ngir mëna tekki
Laaj yi ñuy faral di laaj
What is Mechanistic Interpretability?
Tekkib mekanik mooy jéema soppi xayma yi ci biir reso neuronal yi ci algorithm yu nit ñi mëna xam. Du laaj 'ban duggal moo am solo', waaye laaj 'lan mooy reso bii di ordinatër, sircuit par sircuit?'
Lan mooy mébetu tekkib mekanik?
Tekkib mekanik dafay fexe xam algorithm yi reso bi di jëfandikoo ci biir, te baña yam ci jëflante duggal ak genn.
Luy 'superposition' ci biir reso neuronal?
Superposisioŋ dafay may reso bi mu kode konsept yu bari yu ëpp limu am ci neuron/dimension yi ci denc leen ci yoon yuy jaxasoo ci anam wu ortogonal, moo waral neuron polysemantik.
Luy 'boppu induksioŋ'?
Boppu induction yi dañuy gis lu njëkk ci token bi fi nekk ba noppi ñu koppi li ci topp, muy jumtukaay bu am solo buy may jàng ci biir contexte.
Nan la gëstukat yi di mëna firndee ni sircuit biñ gis dafay def dëgg-dëgg xayma biñ xalaat?
Pexem sabab yu melni patching wala ablation ngir saytu ndax soppi benn composant dafay soppi doxalin bi war, di firndeel wareefam.
Lan moo waral ñu jàpp ni mën tekki lu mekanik lu am solo la ci kaaraange IA?
Xam man-mani yi ci biir ak sircuit yi mën na tax ñu mëna saytu njuuj njaaj yi wala kàttan yu mëna jur loraange, ba noppi may ñu mëna jàppale ci anam wu wóor, jàppale jëfandikoo gi gëna wóor.