TF-IDF na Bag-y-Amagambo Model
Umufuka-w-amagambo uhindura inyandiko mubare wijambo wirengagije gahunda, kandi TF-IDF ipima iyo mibare kuburyo budasanzwe, amagambo atandukanye afite akamaro kuruta ayasanzwe.
Incamake
Together they were the workhorses of search and text classification before deep learning.
Kwibira cyane
Umufuka-w-amagambo (BoW) icyitegererezo cyerekana inyandiko nkurwego rwo kubara ijambo, guta ikibonezamvugo no gutondekanya ijambo: 'imbwa yarumye umugabo' na 'umugabo yarumye imbwa' bisa. Ubu bworoherane butangaje neza kubikorwa byinshi. TF-IDF itunganya BoW ukoresheje amagambo aremereye. Igihe Frequency (TF) gipima inshuro nyinshi ijambo rigaragara mu nyandiko, mugihe Inverse Document Frequency (IDF) iremereye amagambo agaragara mubyangombwa byinshi. Kubigwiza bitanga amanota menshi kumagambo akunze kugaragara mumyandiko imwe ariko idasanzwe mugukusanya, nkijambo ryihariye ryijambo ryibanze, mugihe amagambo asanzwe nka 'the' yegereye uburemere bwa zeru. TF-IDF ibice byimbaraga zijambo ryibanze gushakisha no kugaburira ibyiciro bya kera nka Naive Bayes na SVMs.
Ubushishozi
IDF isanzwe ibarwa nkibiti (N / df), aho N numubare winyandiko zose kandi df numubare winyandiko zirimo ijambo, ijambo rero muri buri nyandiko ritanga IDF hafi ya zeru. Amanota ya nyuma ya TF-IDF ni TF yikubye IDF. Ibice byinyandiko mubisanzwe L2-isanzwe kandi ugereranije na cosine isa, ipima inguni hagati ya vectors ikirengagiza itandukaniro ryinyandiko.
Ingaruka z'Ingamba
Umuvuduko n'igipimo
Ururimi rwakazi rushobora kugenda byihuse nta gutamba guhuzagurika.
Kugera no kugera
Yagura uburyo bwindimi nuburyo bwo gutumanaho.
Ibyemezo bisobanutse
Amakipe arashobora kumara umwanya munini murubanza mugihe automatike ikora gusubiramo.
Kazoza ka TF-IDF na Bag-y-Amagambo Model
Ubwinshi bwimyanya ndangagitsina hamwe na moderi ya moderi noneho ifata ijambo gutondekanya kandi bivuze ko BoW na TF-IDF idashobora, kubwibyo byimbitse byiganje kuri NLP. Nyamara TF-IDF ikomeje kwihuta, gusobanurwa, umutungo-fatizo wibanze bigoye gutsinda kugirango ushakishe ijambo ryibanze, kandi iracyashimangira sisitemu yo kugarura imvange aho amanota ya TF-IDF / BM25 adahujwe hamwe nudushusho twinshi kugirango tunonosore ibisekuru byongerewe imbaraga.
Gushyira mu bikorwa Isi
Shakisha moteri ikurikirana ibyanditswe na TF-IDF cyangwa umusimbuye BM25 kurwanya ikibazo
Spam muyunguruzi ukoresheje umufuka-w-amagambo ibiranga bigaburiwe muri Naive Bayes
Gukuramo ijambo ryibanze cyangwa tagi mu ngingo uhitamo amagambo yo hejuru ya TF-IDF
Kwemeza inkuru zisa nugereranya TF-IDF vectors hamwe na cosine isa
Ingaruka & Kurinda
Ibintu bifatika bishobora kwinjiza bucece raporo, gushyigikira imigendekere, cyangwa ibisubizo byubushakashatsi.
Kwihuta byihuse birashobora gukora ibisubizo bidahuye mubisabwa bisa.
Ibyanditswe byumvikana birashobora kugaragara niba kugenzura kugenzura ari ntege.
Igishushanyo mbonera
Sobanura imiterere isohoka, amajwi, hamwe nubuziranenge mbere yo gutangira.
Ibisubizo byibanze hamwe nisoko yizewe igihe cyose ukuri kwingirakamaro.
Komeza kugenzura abantu kugenzura ibisubizo byinshi.
Kurikirana uburyo bwo kunanirwa no kongera imyitozo cyangwa akazi gahoraho.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the TF-IDF and Bag-of-Words Models quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Ijambo
Ibibazo bikunze kubazwa
What is TF-IDF and Bag-of-Words Models?
Umufuka-w-amagambo uhindura inyandiko mubare wijambo wirengagije gahunda, kandi TF-IDF ipima iyo mibare kuburyo budasanzwe, amagambo atandukanye afite akamaro kuruta ayasanzwe. Hamwe na hamwe bari inzu yakazi yo gushakisha no gutondekanya inyandiko mbere yo kwiga byimbitse.
Ni ayahe makuru y'ingenzi umufuka-w'amagambo moderi uta?
Umufuka-w-amagambo ukomeza ijambo kubara ariko uta ijambo gahunda hamwe nimbonezamvugo.
Igice cya IDF cya TF-IDF gikora iki?
Inverse Document Frequency igabanya uburemere bwamagambo agaragara mubyangombwa byinshi, bityo amagambo asanzwe nka 'the' atanga bike.
Ijambo rigaragara muri buri nyandiko mu cyegeranyo ribona IDF agaciro kegereye iki?
Hamwe na IDF = log (N / df), niba df ihwanye N igipimo ni 1 naho log (1) ni 0, itanga ijambo hafi yuburemere.
Nigute amanota ya TF-IDF mugihe kibarwa?
Uburemere bwa TF-IDF nijambo inshuro zigwizwa ninyandiko zinyuranye.
Ni ubuhe buryo busa busanzwe bukoreshwa mu kugereranya inyandiko za TF-IDF?
Guhuza Cosine bipima inguni hagati ya vectors kandi nibisanzwe byo kugereranya TF-IDF ititaye kuburebure bwinyandiko.