Moodooyinka TF-IDF iyo Boorsada-Erayada
Kiishka-ereyada ayaa qoraalka u beddela tirinta kelmadda iyada oo la iska indho-tiray nidaamka, iyo TF-IDF miisaannadaas ayaa aad dhif u ah, ereyada gaarka ah ayaa ka muhiimsan kuwa caadiga ah.
Dulmar
Together they were the workhorses of search and text classification before deep learning.
quusid qoto dheer
Qaabka bac-of-ereyada (BoW) waxa uu ka dhigan yahay dukumeenti ahaan sida vector ee ereyada loo tiriyo, tuuraya naxwaha iyo habaynta kelmad: 'eyga ayaa qaniinay ninkii' iyo 'ninkii eeyga qaniinay' waxay u egyihiin isku mid. Fududeyntani waxay si la yaab leh ugu shaqeysaa hawlo badan. TF-IDF waxay ku nadiifisaa BoW iyadoo dib u miisaamaysa shuruudaha. Soo noqnoqoshada Erayga (TF) waxa ay cabbirtaa inta jeer ee uu kelmad ka soo baxo dukumeentigu, halka Soo noqnoqoshada Dukumeentigu (IDF) ay hoos u dhigto erayada ka soo muuqda dukumeentiyo badan. Isku dhufashada iyaga ayaa dhibco sare siinaysa ereyada ku badan hal dukumeenti laakiin naadir ku ah ururinta oo dhan, sida erey mawduuc gaar ah, halka ereyada caadiga ah sida 'the' ay ku dhow yihiin miisaan eber ah. TF-IDF vectors awood u leh raadinta ereyga muhiimka ah waxayna quudiyaan kalasaarayaasha qadiimiga ah sida Naive Bayes iyo SVMs.
Aragtida Farsamada
IDF waxaa sida caadiga ah loo xisaabiyaa sida Log(N/df), halkaas oo N ay tahay tirada guud ee dukumeentiyada iyo df waa tirada dukumeentiyada ka kooban ereyga, sidaas darteed kelmad dukumeenti kastaa waxay keentaa IDF meel eber u dhow. Dhibcaha TF-IDF ee ugu dambeeya waa TF lagu dhufto IDF. Caleemaha dukumeentigu inta badan waa L2-caadi oo marka la barbar dhigo isku ekaanshaha cosine, kaas oo cabbira xagasha u dhaxaysa faleebada iskana indhatira farqiga dhererka dukumeentiga.
Saamaynta Istiraatijiyadeed
Xawaaraha iyo miisaanka
Socodka shaqada luqaddu si dhakhso leh ayay u socon kartaa iyada oo aan la hurayn joogteynta.
Helitaanka iyo gaarsiinta
Waxay balaadhisaa gelitaanka luqadaha iyo qaababka isgaarsiinta.
Go'aamo cad
Kooxuhu waxay waqti badan ku qaadan karaan xukunka halka otomaatiggu uu qabanayo ku celcelinta.
Mustaqbalka TF-IDF iyo Moodooyinka Bac-of-Words
Ku-xidhnaanta cufan ee neerfaha iyo moodooyinka beddelka ayaa hadda qabta habka erayga iyo macnaha in BoW iyo TF-IDF aysan awoodin, sidaas darteed moodooyinka qoto dheer ayaa xukuma NLP-gaynta. Hase yeeshee TF-IDF ayaa weli ah mid degdeg ah, la tarjumi karo, saldhig hoose oo kheyraad ah oo adag in lagu garaaco raadinta ereyga muhiimka ah, walina waxay taageertaa hababka dib u soo celinta isku-dhafka ah halkaasoo buundooyinka TF-IDF/BM25 ay ku jiraan cufan cufan si loo hagaajiyo raadinta iyo soo celinta-jiilka la kordhiyay.
Dhaqangelinta Adduunka-dhabta ah
Matoorada raadinta waxay ku qiimeeyaan dukumeentiyada TF-IDF ama BM25 ku xiga ee ka dhanka ah weydiinta
Shaandhaynta spamka iyadoo la isticmaalayo sifooyin bac-ereyada lagu quudiyay kala soocida Naive Bayes
Ka soo saarida ereyada muhiimka ah ama calaamadaynta maqaalka adiga oo dooranaya shuruudaha TF-IDF ee ugu sarreeya
Soo jeedinta maqaallada wararka ee la midka ah iyadoo la is barbardhigayo faleebada TF-IDF ee isku midka ah cosine
Khatarta & Dariiqyada Ilaalada
Xaqiiqooyinka dhalanteed waxay si deggan u geli karaan warbixinnada, taageerada socodka, ama natiijooyinka cilmi-baarista.
Dareenka degdega ahi wuxuu abuuri karaa natiijooyin aan iswaafaqayn codsiyada la midka ah.
Xogta qoraalka xasaasiga ah ayaa laga yaabaa in la kashifo haddii kontaroolada gelitaanka ay daciif yihiin.
Qorshe Hawleedka Dhaqangelinta
Qeex qaabka wax soo saarka, codka, iyo heerarka tayada ka hor inta aan la baahin.
Jawaabaha salka ku haya ilo lagu kalsoon yahay mar kasta oo saxnidu ay muhiim tahay.
Hayso isbaarada dib u eegista bini aadamka ee wax soo saarka sare.
Lasoco qaababka guuldarada oo dib u leyli dardargelinta ama socodka shaqada si joogto ah.
Sii wad Sahaminta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the TF-IDF and Bag-of-Words Models quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Hagaha xiga
Erayada Embending
Su'aalaha soo noqnoqda
What is TF-IDF and Bag-of-Words Models?
Kiishka-ereyada ayaa qoraalka u beddela tirinta kelmadda iyada oo la iska indho-tiray nidaamka, iyo TF-IDF miisaannadaas ayaa aad dhif u ah, ereyada gaarka ah ayaa ka muhiimsan kuwa caadiga ah. Si wada jir ah waxay ahaayeen faras-hawleedkii goobidda iyo kala soocidda qoraalka kahor barashada qoto dheer.
Waa maxay macluumaadka muhiimka ah ee bac-ereyada moodelku tuurayo?
Shandad ereyadu waxay ilaalisaa tirinta erayada laakiin waxay meesha ka saartaa habaynta erayada iyo qaabdhismeedka naxwaha.
Maxay qabtaa qaybta IDF ee TF-IDF?
Soo noqnoqoshada dukumeentiga rogan waxay yaraynaysaa miisaanka ereyada ka soo muuqda dukumeentiyo badan, sidaa darteed ereyada caadiga ah sida 'ka' wax yar bay soo kordhiyaan.
Eray ka soo muuqda dukumeenti kasta oo ku jira ururinta waxa uu helayaa qiimaha IDF ee maxaa ku dhow?
Iyada oo IDF = log (N/df), haddii df la mid yahay N saamiga waa 1 iyo log(1) waa 0, bixinta ereyga ku dhawaad miisaan lahayn.
Sidee buundada TF-IDF loo xisaabiyaa?
Miisaanka TF-IDF waa ereyga soo noqnoqda oo lagu dhufto inta jeer ee dukumeentiga ka soo horjeeda.
Waa kuwee cabbirka isku midka ah ee sida caadiga ah loo isticmaalo in la is barbar dhigo xuubka dukumeentiga TF-ID?
Isku ekaanshaha Cosine waxay cabbirtaa xagasha u dhaxaysa faleebada waana halbeeg lagu barbardhigayo matalaada TF-IDF iyadoon loo eegin dhererka dukumeentigu.