Distillation Ilimi
Distillation na ilimi yana horar da ƙaramin samfurin 'dalibi' don yin koyi da babban, ingantaccen samfurin 'malam'.
Dubawa
It matters because it shrinks powerful models so they run cheaply on phones and servers while keeping much of the accuracy.
Zurfafa nutsewa
Manyan samfura daidai ne amma a hankali da tsada don turawa. Ƙwaƙwalwar ilimi tana canza ƙarfinsu zuwa ƙaramin ƙima ta hanyar sa ɗalibin ya koya daga abubuwan da malamin ya fitar maimakon kawai daga lakabi masu wuya. Mabuɗin fahimta, daga Hinton da abokan aiki, shine cikakken rarraba yiwuwar malami yana ɗaukar 'ilimin duhu': ko da lokacin da yake annabta 'kare', yiwuwar dangi na 'kerkeci' da 'mota' suna bayyana yadda malamin ke ganin kamanceceniya. Tausasa waɗannan yuwuwar tare da zafin jiki yana fallasa wannan tsarin, kuma an horar da ɗalibin don daidaita shi, galibi tare da takalmi na gaskiya. Sakamakon ƙarami ne, samfuri mai sauri wanda ya fi dacewa fiye da wanda aka horar da takalmi shi kaɗai. DistilBERT da TinyBERT sanannu ne nau'ikan yare masu tsantsa.
Fahimtar Fasaha
Asarar al'ada ta haɗu da lokacin distillation (banbancin KL tsakanin yuwuwar tausasa ɗalibi da malami) tare da madaidaicin giciye akan alamomin gaskiya. Tausasawa yana amfani da zafin jiki T a cikin softmax: mafi girma T yana daidaita rarraba don haka ƙananan kamanceceniya tsakanin aji su zama sigina waɗanda za a iya koyo; Distillation gradient yawanci ana auna shi ta T-squared. Bambance-bambancen sun wuce abubuwan da aka fitar: tushen siffa ya yi daidai da tsaka-tsaki na ɓoyayyun yadudduka, da ƙwanƙwasa tushen alaƙa yana daidaita alaƙa tsakanin misalai.
Dabarun Tasiri
Kudin da kasafin kuɗi
Hukunce-hukuncen gine-gine suna haifar da aiki da tsadar aiki na shekaru.
Shawarwari masu haske
Ilimin fasaha yana taimaka wa ƙungiyoyi su zaɓi tari mai kyau, ba kawai sabon abu ba.
Kula da inganci
Zaɓuɓɓukan injiniya mafi kyau suna rage abin dogaro a cikin samarwa.
Makomar Distillation Ilimi
Distillation yanzu shine madaidaicin mataki na jigilar ingantattun samfura kuma shine tsakiyar raƙuman ƙanana, masu iya buɗe samfuri. Halin haɓaka da sauri shine rarrabuwar matakin-jeri daga manyan nau'ikan harshe, inda ƙaƙƙarfan ƙira ke haifar da bayanan horo ko hanyoyin tunani (ciki har da sarkar-tunani) don koyar da ƙananan ɗalibai, bluring layi tare da bayanan roba. Yi tsammanin haɗin kai tare da ƙididdigewa da datse, ƙarin tura kan na'ura, da muhawara mai gudana game da lasisi da inganci lokacin da aka karkata daga samfuran mallakar mallaka waɗanda abubuwan da ke fitowa suka zama siginar horar da fafatawa.
Aiwatar da Gaskiyar Duniya
DistilBERT yana matsawa BERT zuwa kusan 40% ma'auni yayin da yake riƙe mafi yawan fahimtar harshe don saurin fahimta.
Rage babban samfurin hangen nesa don haka mai rarraba hoto zai iya gudana a ainihin lokacin akan app na kyamarar wayar hannu.
Rarraba babban tsarin tunanin tunani cikin ƙaramin tsari don sanya shi amsa math ko yin tambayoyi cikin arha.
Matsa tarin samfura cikin ɗalibi ɗaya don samar da farashi da raguwar latency ba tare da asara mai yawa ba.
Hatsari & Tsare-tsare
Haɓaka ma'auni ɗaya na iya ɓoye manyan raunin tsarin.
Sau da yawa ana raina kayan more rayuwa da kuma kuɗin kulawa.
Tsaro da gibin lura na iya girma yayin da tsarin ke ƙara haɓaka.
Taswirar Hanya
Ƙayyade latency, inganci, da maƙasudin farashi kafin aiwatarwa.
Alamar ma'auni a ƙarƙashin ainihin kaya da yanayin bayanai.
Kula da kayan aiki don kurakurai, ɗigo, da tasirin mai amfani.
Shirya bijirowa da hanyoyin mayar da martani kafin sikeli.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Knowledge Distillation quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Gudanar da Ilimin AI
Tambayoyin da ake yawan yi
What is Knowledge Distillation?
Distillation na ilimi yana horar da ƙaramin samfurin 'dalibi' don yin koyi da babban, ingantaccen samfurin 'malam'. Yana da mahimmanci saboda yana raguwa samfura masu ƙarfi don haka suna gudana cikin arha akan wayoyi da sabobin yayin kiyaye yawancin daidaito.
Menene manufar distillation ilimi?
Distillation yana canja ikon babban malami zuwa ƙaƙƙarfan samfurin ɗalibi mai sauri.
Me ake nufi da 'Dahihin ilimin malami'?
Ilimi mai duhu shine tsari a cikin cikakkiyar rarraba yiwuwar malami, kamar yadda 'kerkeci' yake kama da 'kare', ba kawai babbar amsa ba.
Wace rawa zafin jiki (T) ke takawa wajen distillation?
Maɗaukakin zafin jiki yana daidaita rarraba yuwuwar, yana bayyana kamanceceniya da ɗalibin ya kamata ya koya.
A classic distillation asarar hadawa biyu sassa?
ɗalibin yayi daidai da rarrabuwar laushin malami (KL) yayin da kuma ya dace da alamun gaskiya na ƙasa (cross-entropy).
Wanne misali ne na ƙirar harshe mai tsafta?
DistilBERT sanannen samfuri ne wanda aka goge daga BERT, yana riƙe mafi yawan aiki tare da ƙarancin sigogi.