Ijambo rya WordPiece
WordPiece nijambo ryibimenyetso byerekana algorithm iha imbaraga BERT hamwe na moderi nyinshi Google, igabanya amagambo mo ibice byongera gukoreshwa kuburyo icyitegererezo gishobora gukoresha inyandiko iyariyo yose hamwe namagambo ahamye.
Incamake
It is why a model that has never seen 'unhappiness' can still understand it by reading 'un', '##happy', and '##ness'.
Kwibira cyane
WordPiece yubaka amagambo yamagambo aho kuba amagambo yose cyangwa inyuguti imwe. Guhera ku nyuguti zitandukanye, umururumba uhuza ibimenyetso bibiri byongera amahirwe menshi yo guhugura corpus, bigasubiramo kugeza bigeze kubunini bw'amagambo (BERT ikoresha ibimenyetso 30.000). Mugusuzuma, byerekana umururumba ibumoso-iburyo, uhuza ijambo rirerire mumagambo, hanyuma ugakomeza kubisigaye. Ibice bikomeza imbere yijambo byerekanwe na '##' prefix, bityo 'gukina' bihinduka 'gukina' + '## ing'. Ibi bikemura ikibazo kitari mu magambo: amagambo adasanzwe cyangwa atagaragara gusa abora mo ibice bizwi, kugeza kumurongo umwe niba bikenewe, mugihe amagambo asanzwe aguma nkibimenyetso bimwe kugirango bikore neza.
Ubushishozi
WordPiece itandukanye na Byte-Pair Encoding muburyo bwo guhuza. BPE ihuza inshuro nyinshi zegeranye; WordPiece ihuza ibice byerekana imyitozo-yamakuru ashobora kuba, guhitamo hafi ya jambo ihuriweho cyane irenze ibicuruzwa byibice byayo. Ikimenyetso cya '##' gitandukanya ijambo-intangiriro ibice bikomeza, kureka tokenizer ikubaka imipaka yamagambo bidasubirwaho mugihe yisubije inyuma yinyandiko.
Ingaruka z'Ingamba
Umuvuduko n'igipimo
Ururimi rwakazi rushobora kugenda byihuse nta gutamba guhuzagurika.
Kugera no kugera
Yagura uburyo bwindimi nuburyo bwo gutumanaho.
Ibyemezo bisobanutse
Amakipe arashobora kumara umwanya munini murubanza mugihe automatike ikora gusubiramo.
Kazoza ka WordPiece Tokenisation
Ururimi rushya rwururimi rugenda rushyigikira byte-urwego BPE (GPT umuryango) cyangwa SentencePiece unigram moderi, birinda ururimi rwihariye-gutunganyiriza hamwe no gukora icya Unicode cyose. WordPiece ikomeza kuba urufatiro muri BERT ikomoka kuri kodegisi iracyakoreshwa cyane mugushakisha no gutondekanya. Witegereze gukomeza gukoreshwa mubikorwa NLP, hamwe nubushakashatsi kuri tokenizer-byte na moderi yimiterere ishobora amaherezo kugabanya gushingira kumagambo asobanutse neza.
Gushyira mu bikorwa Isi
BERT yerekana ibibazo byubushakashatsi muri Google Gushakisha, kumena amagambo utamenyereye muri subwords kugirango moderi irashobora guhuza nimpapuro zijyanye.
Guhobera Isura ya BertTokenizer ikoresha WordPiece kugirango ihindure inyandiko mbisi mu ndangamuntu zerekana zahawe BERT kugirango zisesengure amarangamutima kandi zimenyekanishe.
Indimi nyinshi BERT ikoresha amagambo asanganywe ya WordPiece mu ndimi 100+, kureka ibice bikongera gukoreshwa mu nyandiko zijyanye.
DistilBERT na clinique / biomedical BERT ihinduranya WordPiece, ikoresha imvugo idasanzwe yubuvuzi nka 'pneumonoconiose' uyigabanyamo ibice bizwi.
Ingaruka & Kurinda
Ibintu bifatika bishobora kwinjiza bucece raporo, gushyigikira imigendekere, cyangwa ibisubizo byubushakashatsi.
Kwihuta byihuse birashobora gukora ibisubizo bidahuye mubisabwa bisa.
Ibyanditswe byumvikana birashobora kugaragara niba kugenzura kugenzura ari ntege.
Igishushanyo mbonera
Sobanura imiterere isohoka, amajwi, hamwe nubuziranenge mbere yo gutangira.
Ibisubizo byibanze hamwe nisoko yizewe igihe cyose ukuri kwingirakamaro.
Komeza kugenzura abantu kugenzura ibisubizo byinshi.
Kurikirana uburyo bwo kunanirwa no kongera imyitozo cyangwa akazi gahoraho.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the WordPiece Tokenization quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Subword Tokenisation
Ibibazo bikunze kubazwa
What is WordPiece Tokenization?
WordPiece nijambo ryibimenyetso byerekana algorithm iha imbaraga BERT hamwe na moderi nyinshi Google, igabanya amagambo mo ibice byongera gukoreshwa kuburyo icyitegererezo gishobora gukoresha inyandiko iyariyo yose hamwe namagambo ahamye. Niyo mpamvu icyitegererezo kitigeze kibona 'umunezero' gishobora kubyumva usoma 'un', '## byishimo', na '## ness'.
Niki prefix '##' yerekana mubisohoka muri WordPiece?
WordPiece iranga ijambo rikomeza hamwe na '##', bityo 'gukina' bihinduka 'gukina' + '## ing', kureka decoder ikubaka imipaka yamagambo.
Hafi yamagambo angana gute WordPiece ikoreshwa na BERT yumwimerere?
BERT ikoresha amagambo ya WordPiece yamagambo agera kuri 30.000 yama subword, iringaniza ikwirakwizwa nubunini bwimbonerahamwe.
Nigute amahame ya WordPiece yo guhuza atandukanye na Encoding ya Byte-Pair?
BPE ihuza ibice byinshi byegeranye, mugihe WordPiece ihuza byombi byongerera corpus amahirwe, bikunda kubiri byombi bihuza ibicuruzwa byibice byabo.
Nigute WordPiece ikoresha ijambo ritigeze riboneka mugihe cy'amahugurwa?
Amagambo atagaragara yacitsemo ibice birebire bihuye bizwi cyane, bigaruka ku nyuguti imwe, bikemura ikibazo kitari mu magambo.
Mugihe cyo gufata umwanzuro, ni gute WordPiece ihitamo uburyo bwo gutandukanya ijambo?
WordPiece yerekana umururumba, ihuza amagambo maremare yinjira guhera kumwanya uriho, hanyuma ugasubiramo kubisigaye.