Faafinta Aqoonta
Faafinta aqoonta waxay tabobaraysaa moodal yar oo 'arday' ah si ay ugu daydaan moodal weyn oo sax ah 'macallin'.
Dulmar
It matters because it shrinks powerful models so they run cheaply on phones and servers while keeping much of the accuracy.
quusid qoto dheer
Moodooyinka waaweyn waa sax laakiin gaabis ah oo qaali ah in la geeyo. Kala saarida aqoontu waxay u beddeshaa awoodooda qaab is haysta iyadoo ardaygu wax ka barto wax soo saarka macalinka halkii uu ka baran lahaa kaliya calaamadaha adag. Aragtida muhiimka ah, ee Hinton iyo asxaabta, waa in qaybinta itimaalka buuxda ee macalinku ay xambaarsan tahay 'aqoon madow': xitaa marka ay saadaaliso 'ey', itimaalka qaraabada ee 'yeey' iyo 'baabuur' waxay muujinayaan sida macalinku u arko waxyaabaha la midka ah. Ku jilcinta itimaalkan heerkulku waxay daaha ka qaadaysaa qaabdhismeedkaas, ardayguna waxaa loo tababbaray inuu la mid noqdo, inta badanna ay la socdaan calaamadaha runta ah. Natiijadu waa nooc ka yar, ka dheereeya oo guud ahaan ka wanaagsan mid lagu tababaray calaamado kaligiis ah. DistilBERT iyo TinyBERT waa moodooyinka luuqadaha si fiican loo yaqaan.
Aragtida Farsamada
Khasaaraha caadiga ah wuxuu isku daraa erey kala-duwan (KL kala duwanaanshaha u dhexeeya ixtimaalka jilicsan ee ardayga iyo macalinka) iyo is-dhaafsiga caadiga ah ee calaamadaha dhabta ah. Jilicsanaantu waxay isticmaashaa heerkulka T ee softmax: T sare waxay fidisaa qaybinta si ay isu ekaanshaha yar ee fasallada u noqdaan calaamado la baran karo; jilitaanka distillation waxaa caadi ahaan lagu qiyaasaa T-squared. Kala duwanaanshiyaha ayaa dhaafsiisan wax soo saarka: muxaadaro sifada ku salaysan waxay la mid tahay lakabyada qarsoon ee dhexdhexaadka ah, iyo xidhiidhka ku salaysan muxaadaraynta waxay la mid tahay xidhiidhada u dhexeeya tusaalooyinka.
Saamaynta Istiraatijiyadeed
Qiimaha iyo miisaaniyada
Go'aamada qaab-dhismeedku waxay horseedaan waxqabadka iyo kharashka hawlgalka sannadaha.
Go'aamo cad
Waxbarashada farsamada waxay ka caawisaa kooxaha inay doortaan xidhmo sax ah, ma aha oo kaliya kan ugu cusub.
Xakamaynta tayada
Doorashooyinka injineernimada ee wanaagsan waxay yareeyaan shilalka la isku halleyn karo ee wax soo saarka.
Mustaqbalka Faafinta Aqoonta
Distillation hadda waa tillaabada caadiga ah ee dhoofinta moodooyinka hufan waxayna udub dhexaad u tahay mowjadaha maanta ee moodooyinka yaryar ee karti leh. Isbeddellada degdega ah u koraya waa isku xigxig-is-daba-marin laga soo bilaabo moodooyinka luqadaha waaweyn, halkaas oo qaabka xooggani uu ka soo saaro xogta tababarka ama raad-raacyada sababaynta (oo ay ku jiraan silsilad-fikirka) si loo baro ardayda yaryar, oo ku kharribisa khadka xogta synthetic. Filo isku-duubnaan adag oo leh qiyaasid iyo manjooyin, qalab geyn badan, iyo dood joogto ah oo ku saabsan shatiga iyo tayada marka aad ka fogaato moodooyinka lahaanshaha kuwaas oo wax soo saarkoodu ay noqdaan calaamada tababarka tartame.
Dhaqangelinta Adduunka-dhabta ah
DistilBERT waxay ku cadaadinaysaa BERT qiyaas ahaan 40% ka yar iyada oo la ilaalinayo fahamka luqadda inteeda badan si loo ogaado deg degga.
Yaraynta qaabka aragga weyn si sawir-kala-soocida sawirku ugu shaqeeyo wakhtiga dhabta ah abka kamarada casriga ah.
U kala qaybinta fikradda silsilad-fikirka ee model weyn qaab ka yar si ay uga jawaabto xisaabta ama su'aalaha kood ka jaban.
Isku-dubbaridka moodooyinka hal arday si wax-soo-saarka u adeego kharashyada iyo hoos u dhaca daahitaanka iyada oo aan la lumin saxnaansho badan.
Khatarta & Dariiqyada Ilaalada
Hagaajinta hal bartilmaameed waxay qarin kartaa daciifnimada nidaamka ballaaran.
Kaabayaasha dhaqaalaha iyo dayactirka inta badan waa la dhayalsadaa.
Nabadgelyada iyo daldaloolada u fiirsashada ayaa kori kara marka nidaamyadu noqdaan kuwo aad u adag.
Qorshe Hawleedka Dhaqangelinta
Qeex daahida, tayada, iyo bartilmaameedyada qiimaha ka hor inta aan la hirgelin.
Benchmark marka la eego culeyska dhabta ah iyo xaaladaha xogta.
La socodka qalabka khaladaadka, leexashada, iyo saamaynta isticmaalaha.
U diyaari dib-u-noqoshada iyo dariiqyada jawaab-celinta dhacdada ka hor inta aanad miisaan.
Sii wad Sahaminta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Knowledge Distillation quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Hagaha xiga
Maareynta Aqoonta AI
Su'aalaha soo noqnoqda
What is Knowledge Distillation?
Faafinta aqoonta waxay tabobaraysaa moodal yar oo 'arday' ah si ay ugu daydaan moodal weyn oo sax ah 'macallin'. Waa arrin sababtoo ah waxay yaraynaysaa moodooyinka awoodda badan si ay si raqiis ah ugu shaqeeyaan taleefannada iyo server-yada iyagoo ilaalinaya inta badan saxnaanta.
In knowledge distillation, what is the student model trained to do?
Distillation waxay u beddeshaa kartida macalinka weyn qaab arday ah oo isku dhafan, dhaqso badan.
Maxaa loola jeedaa 'aqoonta madow' ee macalinka?
Aqoonta mugdiga ahi waa qaab-dhismeedka qaybinta ixtimaalka buuxda ee macalinka, sida sida ay 'yeey' ula mid tahay 'ey', ma aha oo kaliya jawaabta sare.
Doorkee ayuu heerkulku (T) ka ciyaaraa distillation?
Heerkulka sare waxa uu balaadhiyaa qaybinta ixtimaalka, taas oo muujinaysa isku ekaanshaha ardaygu ay tahay in uu barto.
Khasaare ka soo baxa caadiga ah wuxuu isku daraa labada qaybood?
Ardaygu waxa uu la mid yahay qaybinta jilicsan ee macalinka (KL) halka uu sidoo kale ku haboon yahay summadaha runta-hoose (cross-entropy).
Keebaa tusaale u ah qaabka luqadda distilled?
DistilBERT waa moodel caan ah oo laga soocay BERT, isagoo haya inta badan waxqabadka oo leh xaddi yar.