Zuwa gabaJagora na gaba
Zaren Imel da Gano Kusa da Kwafi a cikin Gano E-Gano
Na fasaha
Jagorar Fasaha
Rarraba Saitin Data da Gano Kusa da Kwafi
Ƙaddamarwa tana gano ainihin bayanai ko makamancin haka don ƙungiyoyi su iya rage raguwa da kuma hana haɗuwa tsakanin bayanan horo da ƙima.
A wannan shafi3 min karatu
Dubawa
Daidai hashes, MinHash, da haɗa kamanceceniya suna gano ra'ayoyi daban-daban na iri ɗaya; Ƙofa da manufofin cirewa na iya watsar da misalai masu amfani ko bambance-bambancen tsiraru.
Zurfafa nutsewa
Daidaitaccen rarrabuwa yana gano bayanai tare da bytes iri ɗaya ko daidaitaccen rubutu. Hashes na sirri suna da inganci don ainihin matches, amma ba za su gano gyare-gyare, tsara canje-canje, ko fastoci ba. Hanyoyi na kusa-kwafi suna amfani da hotunan yatsu kamar n-grams da MinHash, hashing-m na yanki, ko haɗa kamanni. Kowace hanya tana bayyana “kamar” daban kuma tana da farashi daban-daban da halayen ƙarya. Kwafi yana da mahimmanci don horo da kimantawa. Misalai masu maimaitawa na iya yin kiba da ƙima, ƙara haddar abubuwa, da sanya abubuwa masu kama da juna a duka horo da ɓangarorin gwaji. Takardar bincike ta 2021 kan horar da ƙirar harshe ta sami rubutu kusa-kwafi a cikin manyan ma'ajin bayanai kuma an ba da rahoton rage yawan fitarwar rubutu da aka haddace bayan ƙaddamarwa; Takardar ta kuma sami babban jigon gwajin jirgin ƙasa a wasu bayanan ingantattun ma'auni. Waɗannan sakamakon sun keɓance ga ƙungiyar da aka yi nazari da hanyoyin. Deduplication ba kawai "cire duk wani abu makamancin haka." Rubuce-rubucen da aka maimaita na iya wakiltar mitoci na gaske, ingantattun bambance-bambancen, maɓuɓɓuka da yawa, ko tambari daban-daban. Matsakaicin kamanni na duniya na iya cire yare, yare marasa rinjaye, ko takamaiman misalan yanki ba daidai ba. Tsare tabbatarwa, ayyana waɗanne filaye da sauye-sauye aka kwatanta, da kuma duba ƙimar cirewa ta tushe da ƙungiyoyi masu dacewa. Don sarrafa yatsan ruwa, gano gungu masu kwafi kafin a raba su don a iya sanya bayanan da ke da alaƙa tare. Ajiye bayanan duba algorithm, daidaitawa, kofa, da wakilcin da aka riƙe. Ƙimar bututun akan nau'i-nau'i da aka yi bita da hannu kuma bincika duka matches da aka rasa da kuma matches na karya. Rarrabawa yana inganta tsaftar saitin bayanai kawai lokacin da manufofin sa suka dace da aikin kuma suna adana bambance-bambance masu ma'ana.
Dabarun Tasiri
Kudin da kasafin kuɗi
Hukunce-hukuncen gine-gine suna haifar da aiki da tsadar aiki na shekaru.
Shawarwari masu haske
Ilimin fasaha yana taimaka wa ƙungiyoyi su zaɓi tari mai kyau, ba kawai sabon abu ba.
Kula da inganci
Zaɓuɓɓukan injiniya mafi kyau suna rage abin dogaro a cikin samarwa.
Makomar Kwafiwar Saitin Bayanai da Gano Kusa da Kwafi
Hanyoyin ƙetare ƙila za su iya haɗa ƙamus, tsari, da siginar ma'ana, amma ƙofofin za su kasance masu dogaro da ɗawainiya. Ma'auni na gaba ya kamata su ba da rahoton cinkoson bayanai a tsakanin horo da ƙungiyoyin kimantawa da fitar da hanyoyin da za a iya jujjuyawa a inda ba da izini izini. Masu binciken ya kamata su gwada ko tacewa ba ta dace ba yana kawar da yare ko yare na musamman na al'umma. Masu kula da bayanai za su buƙaci bayyananniyar tabbatarwa da yanke shawara mai jujjuyawar cirewa kamar yadda ƙira da saitin bayanai ke tasowa. Har ila yau ya kamata a gwada sabbin hanyoyin ma'anar ma'anar don ƙididdige ƙididdiga na ƙididdigewa kafin aikewa da yawa, tare da bitar ɗan adam don ƙararraki.
Aiwatar da Gaskiyar Duniya
Hash yana gano fayilolin hoto iri ɗaya na byte guda biyu kuma yana adana kwafi ɗaya yayin da yake riƙe tabbacin tushe.
MinHash yana ba da tutoci guda biyu tare da madaidaicin n-gram don bitar hannu.
Ƙungiyoyin ƙofofin da aka haɗa sun fayyace misalai kafin aikin gwajin jirgin ƙasa.
Mai bita yana bincika ko cirewa ya cire misalai da yawa daga ƙaramin rukunin harshe.
Hatsari & Tsare-tsare
Haɓaka ma'auni ɗaya na iya ɓoye manyan raunin tsarin.
Sau da yawa ana raina kayan more rayuwa da kuma kuɗin kulawa.
Tsaro da gibin lura na iya girma yayin da tsarin ke ƙara haɓaka.
Taswirar Hanya
Ƙayyade latency, inganci, da maƙasudin farashi kafin aiwatarwa.
Alamar ma'auni a ƙarƙashin ainihin kaya da yanayin bayanai.
Kula da kayan aiki don kurakurai, ɗigo, da tasirin mai amfani.
Shirya bijirowa da hanyoyin mayar da martani kafin sikeli.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Dataset Deduplication and Near-Duplicate Detection quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Tambayoyin da ake yawan yi
Menene Rarraba Dataset da Gano Kusa da Kwafi?
Ƙaddamarwa tana gano ainihin bayanai ko makamancin haka don ƙungiyoyi su iya rage raguwa da kuma hana haɗuwa tsakanin bayanan horo da ƙima. Daidai hashes, MinHash, da haɗa kamanceceniya suna gano ra'ayoyi daban-daban na iri ɗaya; Ƙofa da manufofin cirewa na iya watsar da misalai masu amfani ko bambance-bambancen tsiraru.
Menene hash ɗin sirri ke ganowa kai tsaye?
Hashes suna da amfani don daidaitaccen daidaito, ba kamanceceniya ta tamani ba.
Wane irin kamanni MinHash akan shingles na rubutu zai iya taimakawa ganowa?
Kimanin MinHash yana saita kamance don fasali kamar shingles.
Me yasa gungu kusa-kwafi kafin yin sassan gwajin jirgin ƙasa?
Misalai masu alaƙa da rarrabuwa tsakanin ɓangarori na iya haɓaka ƙima.
Shin babban haɗin kai yana tabbatar da misalan biyu suna musanyawa?
Kamanci na Semantic baya bada garantin lakabi iri ɗaya ko amfani.
Ci gaba da koyo
Jagora masu alaƙa
An zaɓi ƙarin jagora don wannan batu