Itondere Kuzunguruka no Gukata Umutwe
Icyitonderwa cyitonderwa nuburyo bwo gukurikirana uburyo amakuru anyura muri Transformer yibitseho ibitekerezo kugirango asobanure ibimenyetso byinjira bigira ingaruka kubihanurwa.
Incamake
Head pruning removes attention heads that contribute little, shrinking models without hurting accuracy. Together they help us interpret and compress Transformers.
Kwibira cyane
Abahinduzi bakwirakwiza ibitekerezo byabo mumitwe myinshi yitabwaho mubice byinshi, bityo ikarita yo kwita kumurongo umwe ntikunze kuvuga inkuru yose. Icyitonderwa cyo gutangiza, cyatangijwe na Abnar na Zuidema muri 2020, gikosora ibi mugwiza matrices yibitekerezo kumurongo (nyuma yo kubara imiyoboro isigaye) kugirango ugereranye umubare buri kimenyetso cyinjiza amaherezo kigira uruhare mubimenyetso byatanzwe. Ku buryo butandukanye, ubushakashatsi nka Michel na bagenzi be '' Ese koko imitwe cumi n'itandatu iruta imwe? ' Yerekanye ko imitwe myinshi irenze urugero: igice kinini gishobora gutemwa mugihe cyagenwe hamwe no gutakaza neza kwukuri. Gutema imitwe itondekanya imitwe kubwingirakamaro, akenshi ukoresheje amanota ashingiye kumyumvire ya sensibilité, hanyuma ugahisha masike yingirakamaro. Ubuhanga bubiri bwuzuzanya: kuzunguruka byerekana ibice bigize urusobe rwibisobanuro kugirango bisobanurwe, no gutema ibikorwa kubirenze kugirango moderi ntoya kandi yihuse.
Ubushishozi
Icyitonderwa cyitonderwa gifata buri cyiciro nkibintu byinzibacyuho, byongeramo indangamuntu kugirango bigereranye ibisigisigi bisimbuka, bihindura umurongo, kandi bigwiza iyi matrices kurwego kugirango ibone ibimenyetso byerekana ibimenyetso. Gutema umutwe bigereranya akamaro ka buri mutwe, mubisanzwe ukoresheje igipimo giteganijwe cyigihombo kubijyanye nimpinduka ya mask yo mumutwe, hanyuma zeru imitwe ifite amanota make. Byombi bishingiye kumiterere yuburyo bwimitwe myinshi.
Ingaruka z'Ingamba
Igiciro na bije
Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
Ibyemezo bisobanutse
Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
Kugenzura ubuziranenge
Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.
Igihe kizaza cyo kwitondera kuzunguruka no gutema umutwe
Mugihe icyitegererezo gikura, umwanzuro unoze hamwe nibisobanuro byizewe byombi byihutirwa. Tegereza gutema umutwe kugirango uhuze no gutema, gutondekanya, no gutobora mu miyoboro yoherejwe kugirango bikorwe kandi bitangwe neza. Ubusobanuro bugenda butera imbere kurenza uburyo bwo kwitabwaho, uburyo buremereye buringaniye, hamwe nisesengura ryumuzunguruko usuzuma imikorere yimitwe ya buri muntu. Umuvuduko wo kugenzura AI isobanurwa uzakomeza ubushakashatsi bwo gutwara ibinyabiziga bihuza imitwe nibintu bibara.
Gushyira mu bikorwa Isi
Kwiyumvisha ayo magambo mu nteruro Transformer classifier yashingiyeho, mugutanga ibitekerezo kugirango ugaragaze ibimenyetso bikomeye
Gucomeka moderi ya BERT yo kohereza mobile mugukata imitwe ititaye kumutwe kugirango ugabanye ubukererwe
Kugenzura icyitegererezo kubogama mugukurikirana ibitekerezo biturutse kubihanuzi bigaruka kubimenyetso byinjiza byoroshye
Kwihutisha umwanzuro muri sisitemu yo guhindura umusaruro ukuraho imitwe-ifite akamaro gake yagaragaye binyuze mu gutanga amanota
Ingaruka & Kurinda
Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.
Igishushanyo mbonera
Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
Tegura inzira yo gusubiza ibyabaye mbere yo gupima.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Attention Rollout and Head Pruning quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Multi-Umutwe Utinze Kwitonda
Ibibazo bikunze kubazwa
What is Attention Rollout and Head Pruning?
Icyitonderwa cyitonderwa nuburyo bwo gukurikirana uburyo amakuru anyura muri Transformer yibitseho ibitekerezo kugirango asobanure ibimenyetso byinjira bigira ingaruka kubihanurwa. Gukata umutwe bikuraho imitwe yitabwaho itanga bike, igabanya moderi itababaje neza. Hamwe na hamwe badufasha gusobanura no guhagarika Transformers.
Niyihe ntego yo kwitabwaho?
Rollout igwiza ibitekerezo-byitondewe (hamwe nibisigara) kugirango ugereranye uburyo buri kimenyetso cyinjiza kigira ingaruka kumusaruro.
Ni ukubera iki ikarita imwe yo kwitondera ikarita idahagije kubisobanuro?
Kuberako ibitekerezo bikwirakwizwa murwego rwumutwe hamwe, imitwe yikarita ntishobora gufata amakuru yuzuye.
Ni ubuhe buryo bwo kwitondera bwiyongera kuri buri matrike yo kwitondera kugirango uhuze ibisigisigi?
Ongeraho ibiranga indangamuntu yerekana ibisigisigi bisimbuka bihuza ibimenyetso bigumana amakuru yabo.
Ni ubuhe bushakashatsi bwakozwe ku mitwe myinshi bwagaragaje iki ku bijyanye no kugabanuka k'umutwe?
Ubushakashatsi nka 'Ese Imitwe cumi n'itandatu iruta imwe?' yerekanye igice kinini cyimitwe irenze urugero.
Nigute akamaro k'umutwe gakunze kugereranywa no gutema?
Akamaro gakunze gutangwa hifashishijwe igipimo cyigihombo kubijyanye na mask ihinduka kuri buri mutwe.