Ụdị TF-IDF na akpa-okwu
Akpa-okwu na-atụgharị ederede ka ọ bụrụ ọnụ ọgụgụ okwu na-eleghara usoro anya, na TF-IDF na-atụle ọnụ ọgụgụ ndị ahụ dị ụkọ, okwu pụrụ iche dị mkpa karịa nke nkịtị.
Nchịkọta
Together they were the workhorses of search and text classification before deep learning.
Ime miri emi
Ụdị akpa-okwu (BoW) na-anọchi anya akwụkwọ dị ka vector nke ọnụ ọgụgụ okwu, na-atụfu ụtọ asụsụ na usoro okwu: 'nkịta tara nwoke ahụ' na 'nwoke ahụ tara nkịta' yiri otu. Nke a dị mfe na-arụ ọrụ dị ịtụnanya nke ọma maka ọtụtụ ọrụ. TF-IDF na-anụcha BoW site n'ịgbanwe usoro. Ugboro ugboro (TF) na-atụ ugboro ole okwu pụtara na akwụkwọ, ebe Inverse Document Frequency (IDF) na-eweda okwu ndị pụtara n'ọtụtụ akwụkwọ. Ịba ụba ha na-enye ọnụ ọgụgụ dị elu maka okwu ndị na-emekarị n'otu akwụkwọ mana ọ na-adịghị ahụkebe n'ofe mkpokọta ahụ, dị ka isiokwu isiokwu pụrụ iche, ebe okwu nkịtị dị ka 'the' na-eru nso-efu efu. TF-IDF vectors na-enye ike nchọ isiokwu dị elu ma na-eri nri klas oge gboo dị ka Naive Bayes na SVM.
Nghọta nka nka
A na-agbakọkarị IDF dị ka log (N/df), ebe N bụ mkpokọta akwụkwọ na df bụ ọnụọgụ akwụkwọ nwere okwu ahụ, yabụ otu okwu n'ime akwụkwọ ọ bụla na-enye IDF nso efu. Akara TF-IDF ikpeazụ bụ TF ji IDF mụbaa. Vectors akwụkwọ na-abụkarị L2-normalized na atụnyere myirịta cosine, nke na-atụ akụkụ dị n'etiti vectors ma na-eleghara ọdịiche dị ogologo akwụkwọ anya.
Mmetụta atụmatụ
Ọsọ na ọnụ ọgụgụ
Usoro ọrụ asụsụ nwere ike ịga ngwa ngwa n'achụghị nkwụsi ike.
Nweta na iru
Ọ na-agbasawanye ohere n'ofe asụsụ na ụdị nzikọrịta ozi.
Mkpebi doro anya
Ndị otu nwere ike itinyekwu oge na ikpe ebe akpaaka na-ejikwa nkwughachi.
Ọdịnihu nke TF-IDF na ụdị akpa-okwu
Ihe mgbakwunye akwara dị oke na ụdị mgbanwe ugbu a na-ejide usoro okwu yana pụtara na BoW na TF-IDF enweghị ike, yabụ ụdị miri emi na-achịkwa NLP dị oke ọnụ. N'agbanyeghị TF-IDF na-anọgide na-adị ngwa ngwa, ntụgharị okwu, usoro ihe onwunwe dị ala nke siri ike ịkụ maka ọchụchọ isiokwu, ma ọ ka na-akwado usoro nchịkọta ngwakọ ebe ọnụ ọgụgụ TF-IDF/BM25 na-ejikọta ya na nnukwu ntinye iji melite ọchụchọ na iweghachite ọgbọ.
Mmejuputa n'ezie n'ụwa
Mpempe akwụkwọ nyocha nke TF-IDF ma ọ bụ BM25 nọchiri ya megide ajụjụ
Ihe nzacha spam na-eji njirimara akpa-okwu etinyere na nhazi ọkwa Naive Bayes
Ịwepụ mkpụrụokwu ma ọ bụ mkpado site na edemede site n'ịhọrọ usoro TF-IDF ya kachasị elu
Na-atụ aro akụkọ akụkọ ndị yiri ya site n'ịtụle vector TF-IDF na myirịta cosine
Ihe ize ndụ & okporo ụzọ nche
Eziokwu ndị e chepụtara echepụta nwere ike jiri nwayọ tinye akụkọ, nkwado nkwado, ma ọ bụ nsonaazụ nyocha.
Mmetụta ngwa ngwa nwere ike ịmepụta nsonaazụ na-ekwekọghị ekwekọ n'ofe arịrịọ ndị yiri ya.
Enwere ike ikpughe data ederede nwere mmetụta ma ọ bụrụ na njikwa ohere adịghị ike.
Map mmejuputa
Kọwaa usoro mmepụta, ụda, na ụkpụrụ ịdịmma tupu ibugharị.
Weghachite nzaghachi site na isi mmalite ntụkwasị obi mgbe ọ bụla izi ezi dị mkpa.
Debe ebe nleba anya mmadụ maka mpụta dị elu.
Sochie ụkpụrụ ọdịda ma na-azụghachi mkpali ma ọ bụ usoro ọrụ mgbe niile.
Nọgide na-eme nchọpụta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the TF-IDF and Bag-of-Words Models quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ntuziaka na-esote
Ntinye Okwu
Ajụjụ a na-ajụkarị
What is TF-IDF and Bag-of-Words Models?
Akpa-okwu na-atụgharị ederede ka ọ bụrụ ọnụ ọgụgụ okwu na-eleghara usoro anya, na TF-IDF na-atụle ọnụ ọgụgụ ndị ahụ dị ụkọ, okwu pụrụ iche dị mkpa karịa nke nkịtị. Ọnụ ha bụ ndị na-arụ ọrụ nke ọchụchọ na nhazi ederede tupu mmụta miri emi.
Kedu ozi isi ihe nlereanya akpa-okwu na-atụfu?
Akpa-okwu na-edobe ọnụ okwu mana ọ na-atụfu usoro okwu na nhazi ụtọasụsụ.
Kedu ihe akụkụ IDF nke TF-IDF na-eme?
Frequency Document Inverse na-ebelata ịdị arọ nke okwu ndị na-apụta n'ọtụtụ akwụkwọ, yabụ okwu nkịtị dị ka 'nke' na-atụnye ntakịrị.
Otu okwu pụtara na akwụkwọ ọ bụla na mkpokọta na-enweta uru IDF nso gịnị?
Site na IDF = log (N/df), ọ bụrụ na df nhata N nha bụ 1 na log (1) bụ 0, na-enye okwu ahụ ọ fọrọ nke nta ka ọ bụrụ enweghị ibu.
Kedu ka esi agbakọ akara TF-IDF maka otu okwu?
Ibu TF-IDF bụ okwu ugboro ugboro nke akwụkwọ na-agbanwe agbanwe.
Kedu ihe myirịta a na-ejikarị atụnyere vector akwụkwọ TF-IDF?
Myirịta nke Cosine na-atụ akụkụ dị n'etiti vectors ma bụrụ ọkọlọtọ maka atụnyere nnochite anya TF-IDF n'agbanyeghị ogologo akwụkwọ.