Uhlolojikelele
Ukusampula okungalungile kanye ne-Noise Contrastive Estimation (NCE) kungamaqhinga avumela amamodeli afunde kumagama amakhulu ngaphandle kokubala i-softmax egcwele ebizayo. Esikhundleni sokuthola yonke into engaba khona, bafundisa imodeli ukuthi isho izibonelo zangempela (ezinhle) ezivela kwembalwa yalezo ezingamanga (ezingezinhle).
Isampula Engalungile kanye Noise Contrastive Estimation iyibhulokhi yokwakha yobuchwepheshe ethinta ikhwalithi yemodeli, izindleko zengqalasizinda, ukubambezeleka, nokuthembeka esikalini.
I-Deep Dive
Uma silulumagama sinamakhulu ezinkulungwane zamagama, i-softmax evamile kufanele ijwayele phezu kwegama ngalinye esinyathelweni ngasinye sokuqeqesha - ihamba kancane kakhulu. I-Noise Contrastive Estimation ihlela kabusha inkinga njengokwahlukanisa kanambambili: uma kunikezwe ithagethi kanye namasampuli ambalwa 'omsindo' athathwe ekusabalaliseni okwaziwayo, funda ukuhlukanisa isampula yangempela nomsindo, obuyisa ngokusobala amathuba afunekayo ngaphandle kokujwayelekile okusobala. Isampula elibi, elidume ngemodeli ye-word2vec's skip-gram, ingumzala owenziwe lula: kuqiniso ngalinye (igama, umongo) limatanisa amasampula k ama-negative futhi iqeqeshela imodeli ukuthi inikeze amaphuzu aphezulu kubabhangqwana bangempela namaphuzu aphansi kuma-fake, kusetshenziswa inhloso ye-sigmoid. Kokubili kuguqule inkinga ebizayo yezigaba eziningi ibe ngamanani anambambili ashibhile, okwenza ukuqeqeshwa kokushumeka okukhulu kusebenze. Ukukhethwa kokusatshalaliswa komsindo (ngokuvamile i-unigram ephakanyiswe emandleni angu-3/4) kuthinta kakhulu ikhwalithi.
I-Technical Insight
I-NCE ilinganisela imodeli ngokuhlukanisa idatha nomsindo, futhi njengoba inani lamasampuli omsindo likhula cishe licishe libe nokwenzeka kakhulu nge-softmax efanele evamile. Ukusampula okungalungile kwehlisa ngokuphelele imigomo ye-NCE yokujwayela, kuthuthukisa ilogi σ(amaphuzu amahle) + Σ log σ(−amaphuzu angalungile). Lokho kuyenza isheshe kodwa ingabi isilinganiselo sokuminyana esingaguquki - ihlelelwe ukufunda ukushumeka okuhle kunokuthi kube namathuba alinganiselwe. Amasampula amanegethivu asuka ekusabalaliseni kwe-unigram ebushelelezi (imvamisa^0.75) kulinganisa amagama avamile nangavamile.
Ukuphatha Isampula Elibi kanye Nesilinganiso Sokuqhathanisa Nomsindo
Ukusampula okungalungile kanye ne-Noise Contrastive Estimation (NCE) kungamaqhinga avumela amamodeli afunde kumagama amakhulu ngaphandle kokubala i-softmax egcwele ebizayo. Esikhundleni sokuthola yonke into engaba khona, bafundisa imodeli ukuthi isho izibonelo zangempela (ezinhle) ezivela kwembalwa yalezo ezingamanga (ezingezinhle). Isampula Engalungile kanye Noise Contrastive Estimation iyibhulokhi yokwakha yobuchwepheshe ethinta ikhwalithi yemodeli, izindleko zengqalasizinda, ukubambezeleka, nokuthembeka esikalini. Ukuze wakhe ukuqonda okujulile, phatha i-Negative Sampling and Noise Contrastive Estimation njengemodeli yokusebenza, hhayi isici esisodwa: chaza imiphumela efiselekayo, ucacise ukuqagela, futhi uhlukanise lokho isistimu engakwenza ngokwethembeka kulokho okusadinga ukwahlulela kochwepheshe.
Empeleni, amaqembu aqinile asebenzisa Ukusampula Okubi kanye Nesilinganiso Esiphikisana Nomsindo athuthukisa izakhiwo, idatha, nokukhetha kwengqalasizinda ngokumelene nokuthembeka nezindleko. Babhala imibandela yempumelelo ecacile, ukuhlola okuqhathaniswa nedatha engokoqobo nokugeleza komsebenzi, futhi baphindaphinde ngokusekelwe kumaphethini okuhluleka aqashiwe esikhundleni sokuwina kwebhentshimakhi yesikhathi esisodwa. Yilapho ukuqonda kwethiyori kuguquka kube amandla ahlala njalo kuwo wonke umkhiqizo, inqubomgomo, kanye nokusebenza.
Izinqumo zezakhiwo ziqhuba ukusebenza kanye nezindleko zokusebenza iminyaka. Ngesikhathi esifanayo, Ukuthuthukisa ibhentshimakhi eyodwa kungafihla ubuthakathaka obubanzi besistimu. Indlela eqine kakhulu iwukuhlanganisa isivinini sokuhlola nesiyalo sokuphatha: qhuba abashayeli bezindiza, bamba ubufakazi, ushicilele amalogi ezinqumo, futhi ubuyekeze izivikelo ngokuqhubekayo njengoba imodeli yokuziphatha, okulindelwe ngabasebenzisi, kanye nezimfuneko zokulawula zishintsha.
I-Strategic Impact
Izinqumo zezakhiwo ziqhuba ukusebenza kanye nezindleko zokusebenza iminyaka.
Izinqumo zezakhiwo ziqhuba ukusebenza kanye nezindleko zokusebenza iminyaka. Ekusetshenzisweni kwekhwalithi ephezulu, lokhu kuhunyushwa emithethweni yokusebenza elinganisekayo, imingcele yobunikazi, nemikhuba yokubuyekeza ephindelelayo ukuze amaqembu akwazi ukukala ukuzethemba esikhundleni sokukala ukungaqondakali.
Imfundo yobuchwepheshe isiza amaqembu ukuthi akhethe isitaki esifanele, hhayi nje esisha.
Imfundo yobuchwepheshe isiza amaqembu ukuthi akhethe isitaki esifanele, hhayi nje esisha. Ekusetshenzisweni kwekhwalithi ephezulu, lokhu kuhunyushwa emithethweni yokusebenza elinganisekayo, imingcele yobunikazi, nemikhuba yokubuyekeza ephindelelayo ukuze amaqembu akwazi ukukala ukuzethemba esikhundleni sokukala ukungaqondakali.
Izinketho ezingcono zobunjiniyela zinciphisa izehlakalo ezinokwethenjelwa ekukhiqizeni.
Izinketho ezingcono zobunjiniyela zinciphisa izehlakalo ezinokwethenjelwa ekukhiqizeni. Ekusetshenzisweni kwekhwalithi ephezulu, lokhu kuhunyushwa emithethweni yokusebenza elinganisekayo, imingcele yobunikazi, nemikhuba yokubuyekeza ephindelelayo ukuze amaqembu akwazi ukukala ukuzethemba esikhundleni sokukala ukungaqondakali.
Ukuqaliswa Komhlaba Wangempela
I-word2vec skip-gram enesampula engalungile yokufunda amagama ashumekiwe ezigidini zamathokheni ngaphandle kwe-softmax egcwele.
Amamodeli olimi ngokomlando ayesebenzisa i-NCE ukuqeqesha amagama angamakhulu ezinkulungwane zamagama ngendlela efanele.
Amasistimu wokuncoma kanye nokubuyisa amasampula ezinto 'ezimbi' umsebenzisi angazange ahlanganyele nazo ukuze aqeqeshe amamodeli okushumeka emibhoshongo emibili.
Ukushumeka kwegrafu negrafu yolwazi (isb., ukonakalisa ikhanda noma umsila we-triple) kusetshenziswa amasampula angemuhle ukuze kufundwe ubudlelwano bebhizinisi.
Amaphethini Okusebenzisa
Ukuthatha Isampula Okubi kanye Nezilinganiso Eziphikisana Nomsindo ekusebenzeni
I-word2vec skip-gram enesampula engalungile yokufunda amagama ashumekiwe ezigidini zamathokheni ngaphandle kwe-softmax egcwele.
I-word2vec skip-gram enesampula engalungile yokushumeka kwamagama okufunda ezinkulungwaneni zezigidi zamathokheni ngaphandle kwe-softmax ephelele Amaqembu ngokuvamile athola imiphumela engcono lapho echaza imingcele yekhwalithi ngaphambili, egcina indlela yokukhuphuka yomuntu yamacala asemaphethelweni, futhi alandelele kokubili izinzuzo zokukhiqiza nezindleko zamaphutha ngokuhamba kwesikhathi.
Ukuthatha Isampula Okubi kanye Nezilinganiso Eziphikisana Nomsindo ekusebenzeni
Amamodeli olimi ngokomlando ayesebenzisa i-NCE ukuqeqesha amagama angamakhulu ezinkulungwane zamagama ngendlela efanele.
Amamodeli olimi ngokomlando ayesebenzisa i-NCE ukuze aqeqeshe amagama angamakhulu ezinkulungwane zamagama ngempumelelo Amaqembu ngokuvamile athola imiphumela engcono lapho echaza izinga eliphezulu ngaphambili, egcina indlela yokukhuphuka yabantu yamakesi asemaphethelweni, futhi elandelela kokubili izinzuzo zokukhiqiza nezindleko zamaphutha ngokuhamba kwesikhathi.
Ukuthatha Isampula Okubi kanye Nezilinganiso Eziphikisana Nomsindo ekusebenzeni
Amasistimu wokuncoma kanye nokubuyisa amasampula ezinto 'ezimbi' umsebenzisi angazange ahlanganyele nazo ukuze aqeqeshe amamodeli okushumeka emibhoshongo emibili.
Izincomo nezinhlelo zokubuyisa amasampula ezinto 'ezimbi' umsebenzisi angazange asebenzisane nazo ukuze aqeqeshe amamodeli okushumeka emibhoshongo emibili Amaqembu ngokuvamile athola imiphumela engcono uma echaza imikhawulo yekhwalithi ngaphambili, egcina indlela yokukhuphuka yabantu yamakesi asemaphethelweni, futhi alandelele kokubili izinzuzo zokukhiqiza nezindleko zamaphutha ngokuhamba kwesikhathi.
Ukuthatha Isampula Okubi kanye Nezilinganiso Eziphikisana Nomsindo ekusebenzeni
Ukushumeka kwegrafu negrafu yolwazi (isb., ukonakalisa ikhanda noma umsila we-triple) kusetshenziswa amasampula angemuhle ukuze kufundwe ubudlelwano bebhizinisi.
Ukushumeka kwegrafu negrafu yolwazi (isb., ukonakalisa ikhanda noma umsila we-triple) kusetshenziswa amasampula angalungile ukuze afunde ubudlelwano bebhizinisi Amaqembu ngokuvamile athola imiphumela engcono uma echaza imikhawulo yekhwalithi ngaphambili, egcina indlela yokukhuphuka kwabantu yamakesi asemaphethelweni, futhi elandelela kokubili izinzuzo zokukhiqiza nezindleko zamaphutha ngokuhamba kwesikhathi.
Izingozi & Guardrails
Ukuthuthukisa ibhentshimakhi eyodwa kungafihla ubuthakathaka obubanzi besistimu.
Izindleko zengqalasizinda nezokulungisa zivame ukubukelwa phansi.
Izikhala zokuphepha nokubonakala zingakhula njengoba izinhlelo ziba nzima kakhulu.
Ukuqalisa Umhlahlandlela
Chaza ukubambezeleka, ikhwalithi, nezindleko ezihlosiwe ngaphambi kokuqaliswa.
Chaza ukubambezeleka, ikhwalithi, nezindleko ezihlosiwe ngaphambi kokuqaliswa. Phatha isinyathelo ngasinye njengesango lobufakazi: uma imibandela ingafinyelelwa, misa ukukhishwa, vala igebe, bese unweba ukusetshenziswa.
Ibhentshimakhi ngaphansi komthwalo wangempela nezimo zedatha.
Ibhentshimakhi ngaphansi komthwalo wangempela nezimo zedatha. Phatha isinyathelo ngasinye njengesango lobufakazi: uma imibandela ingafinyelelwa, misa ukukhishwa, vala igebe, bese unweba ukusetshenziswa.
Ukuqapha amathuluzi amaphutha, ukukhukhuleka, nomthelela wabasebenzisi.
Ukuqapha amathuluzi amaphutha, ukukhukhuleka, nomthelela wabasebenzisi. Phatha isinyathelo ngasinye njengesango lobufakazi: uma imibandela ingafinyelelwa, misa ukukhishwa, vala igebe, bese unweba ukusetshenziswa.
Lungiselela izindlela zokuhlehlisa nezigameko ngaphambi kokukala.
Lungiselela izindlela zokuhlehlisa nezigameko ngaphambi kokukala. Phatha isinyathelo ngasinye njengesango lobufakazi: uma imibandela ingafinyelelwa, misa ukukhishwa, vala igebe, bese unweba ukusetshenziswa.