Dubawa
SmoothQuant wata dabara ce da ke ba da damar damfara manyan nau'ikan harshe har zuwa intigers 8-bit don duka nauyi da kunnawa ba tare da sake horarwa ba. Yana da mahimmanci saboda kunnawa a cikin manyan samfura suna ƙunshe da matsananciyar ƙetare waɗanda galibi suna lalata math ɗin ƙima, kuma SmoothQuant yana horar da su.
SmoothQuant da Kunna ƙididdigewa ƙaƙƙarfan ginin fasaha ne wanda ke shafar ingancin samfuri, farashin kayayyakin more rayuwa, latency, da aminci a sikeli.
Zurfafa nutsewa
Lokacin da ka rage samfurin daga 16-bit yana yawo zuwa lamba 8-bit, ma'aunin nauyi yana daɗaɗa sauƙi amma kunnawa yana da matsala: wasu tashoshi suna ɗaukar ƙimar sau 10 zuwa 100 fiye da sauran, kuma tilasta su cikin grid mai ƙima yana lalata daidaito. SmoothQuant, Xiao et al. a cikin 2022, ya lura cewa ma'aunin nauyi yana da santsi kuma mai sauƙin ƙididdigewa yayin kunnawa yana da kauri. Don haka ta hanyar lissafi yana ƙaura wahalar: yana rarraba tashoshin kunnawa ta hanyar ma'auni na kowane tashoshi kuma yana ninka ma'aunin nauyi daidai da sikelin iri ɗaya. Ayyukan biyu sun soke, suna barin fitowar samfurin baya canzawa, amma yanzu duka tenors suna zaune a cikin jeri na abokantaka. Sakamakon shine W8A8 (ma'auni 8-bit da kunnawa) tare da asarar daidaito kusa-sifiri da kusan saurin 2x da ajiyar ƙwaƙwalwar ajiya.
Fahimtar Fasaha
Babban abin da za'a yi shi ne ma'aunin santsi na kowane tashoshi wanda aka lissafta shi azaman s = max(|X|)^alpha / max(|W|)^(1-alpha). Ana yin ma'auni na kunnawa da 1/s kuma ma'auni ta s, don haka ana adana samfurin matrix XW. Saboda ana shigar da sikelin a layi a cikin ma'aunin ma'aunin Layer na baya ko aiki mai haɗaɗɗiya, yana ƙara farashin lokacin aiki sifili. Alfa hyperparameter (sau da yawa 0.5) yana sarrafa nawa nauyin nauyi ya tashi daga kunnawa zuwa nauyi.
Kwarewar SmoothQuant da Ƙididdiga Kunnawa
SmoothQuant wata dabara ce da ke ba da damar damfara manyan nau'ikan harshe har zuwa intigers 8-bit don duka nauyi da kunnawa ba tare da sake horarwa ba. Yana da mahimmanci saboda kunnawa a cikin manyan samfura suna ƙunshe da matsananciyar ƙetare waɗanda galibi suna lalata math ɗin ƙima, kuma SmoothQuant yana horar da su. SmoothQuant da Kunna ƙididdigewa ƙaƙƙarfan ginin fasaha ne wanda ke shafar ingancin samfuri, farashin kayayyakin more rayuwa, latency, da aminci a sikeli. Don gina zurfin fahimta, bi da SmoothQuant da Ƙididdigar Kunnawa azaman ƙirar aiki, ba fasali ɗaya ba: ayyana sakamakon da ake so, fayyace zato, da raba abin da tsarin zai iya yi da dogaro daga abin da har yanzu yana buƙatar yanke hukunci na ƙwararru.
A aikace, ƙungiyoyi masu ƙarfi da ke amfani da SmoothQuant da Ƙididdigar Kunnawa suna haɓaka gine-gine, bayanai, da zaɓin abubuwan more rayuwa tare da dogaro da farashi. Suna rubuta ƙayyadaddun ƙa'idodin nasara, gwaji akan bayanan gaskiya da gudanawar aiki, da jujjuyawar bisa ga tsarin gazawar da aka lura maimakon cin nasara na lokaci ɗaya. Wannan shine inda fahimtar ka'idar ta juya zuwa iyawa mai dorewa a cikin samfura, manufofi, da ayyuka.
Hukunce-hukuncen gine-gine suna haifar da aiki da tsadar aiki na shekaru. A lokaci guda, Haɓaka ma'auni ɗaya na iya ɓoye manyan raunin tsarin. Hanyar da ta fi dacewa ita ce haɗa saurin gwaji tare da horon gudanarwa: gudanar da matukin jirgi, kama shaida, buga rajistan ayyukan yanke shawara, da ci gaba da sabunta abubuwan tsaro kamar yadda halayen ƙira, tsammanin mai amfani, da buƙatun tsari ke tasowa.
Dabarun Tasiri
Hukunce-hukuncen gine-gine suna haifar da aiki da tsadar aiki na shekaru.
Hukunce-hukuncen gine-gine suna haifar da aiki da tsadar aiki na shekaru. A cikin ƙawance masu inganci, ana fassara wannan zuwa ƙa'idodin aiki waɗanda za a iya aunawa, iyakokin ikon mallaka, da kuma bita-da-kullin bita don ƙungiyoyi su iya haɓaka kwarin gwiwa a maimakon ɓata shakku.
Ilimin fasaha yana taimaka wa ƙungiyoyi su zaɓi tari mai kyau, ba kawai sabon abu ba.
Ilimin fasaha yana taimaka wa ƙungiyoyi su zaɓi tari mai kyau, ba kawai sabon abu ba. A cikin ƙawance masu inganci, ana fassara wannan zuwa ƙa'idodin aiki waɗanda za a iya aunawa, iyakokin ikon mallaka, da kuma bita-da-kullin bita don ƙungiyoyi su iya haɓaka kwarin gwiwa a maimakon ɓata shakku.
Zaɓuɓɓukan injiniya mafi kyau suna rage abin dogaro a cikin samarwa.
Zaɓuɓɓukan injiniya mafi kyau suna rage abin dogaro a cikin samarwa. A cikin ƙawance masu inganci, ana fassara wannan zuwa ƙa'idodin aiki waɗanda za a iya aunawa, iyakokin ikon mallaka, da kuma bita-da-kullin bita don ƙungiyoyi su iya haɓaka kwarin gwiwa a maimakon ɓata shakku.
Aiwatar da Gaskiyar Duniya
Hidimar 70B-parameter LLM a W8A8 akan ƴan GPUs ta hanyar rage duka ƙwaƙwalwar ajiya da matrix-yawan farashin
Ƙaddamar da bayanin INT8 akan NVIDIA Hopper/Blackwell tensor cores wanda ke haɓaka math 8-bit na asali.
Aiwatar da samfuran taɗi akan wuraren ƙarshe na girgije mai ƙayyadaddun farashi inda yawan abubuwan da aka samu sau biyu ke yanke lissafin kowane-alama kai tsaye.
Matsa maɓallan masu canza canji don magana a kan na'urar ko fassarar inda kernels 8-bit ke gudana cikin sauri da sanyi.
Hanyoyin Aiwatarwa
SmoothQuant da Ƙididdiga Kunnawa a aikace
Bauta wa madaidaicin 70B-LLM a W8A8 akan ƴan GPUs ta hanyar rage duka ƙwaƙwalwar ajiya da matrix-yawan farashin.
Hidimar 70B-parameter LLM a W8A8 akan ƙananan GPUs ta hanyar rage ƙwaƙwalwar ajiya da matrix-yawan farashi Ƙungiyoyi yawanci suna samun sakamako mafi kyau lokacin da suka ayyana ma'auni masu inganci a gaba, kiyaye hanyar haɓakar ɗan adam don shari'o'in gefe, da bin duk nasarorin samarwa da ƙimar kuskure akan lokaci.
SmoothQuant da Ƙididdiga Kunnawa a aikace
Ƙaddamar da bayanin INT8 akan NVIDIA Hopper/Blackwell tensor cores waɗanda ke haɓaka lissafin integer 8 a asali.
Ƙaddamar da bayanin INT8 akan NVIDIA Hopper/Blackwell tensor cores waɗanda ke haɓaka Ƙungiyoyin lissafi na 8-bit yawanci suna samun sakamako mafi kyau lokacin da suka ayyana ma'auni masu inganci a gaba, kiyaye hanyar haɓakar ɗan adam don shari'o'in gefe, da kuma bin duk abubuwan da aka samu na samarwa da kuma kashe kuɗi na lokaci.
SmoothQuant da Ƙididdiga Kunnawa a aikace
Aiwatar da samfuran taɗi akan madaidaitan ma'aunin gajimare masu ƙayyadaddun ƙima inda yawan abubuwan da aka samu sau biyu ke yanke lissafin kowane-alama kai tsaye.
Aiwatar da samfuran taɗi a kan ƙarshen ƙarshen girgije mai ƙarancin tsada inda haɓaka kayan aiki sau biyu kai tsaye yana yanke lissafin kowace alama Ƙungiyoyi yawanci suna samun sakamako mafi kyau lokacin da suka ayyana ƙofofin inganci a gaba, kiyaye hanyar haɓakar ɗan adam don ƙararraki, da bin diddigin nasarorin samarwa da ƙimar kuskure akan lokaci.
SmoothQuant da Ƙididdiga Kunnawa a aikace
Matsa maɓallan masu canza canji don magana a kan na'urar ko fassarar inda kernels 8-bit ke gudana cikin sauri da sanyi.
Matsa maɓallan masu canza canji don magana a kan na'urar ko fassarar inda kernels 8-bit ke gudana da sauri kuma ƙungiyoyi masu sanyaya yawanci suna samun sakamako mafi kyau lokacin da suka ayyana ma'auni masu inganci a gaba, kiyaye hanyar haɓakar ɗan adam don ƙararraki, da bin diddigin nasarorin samarwa da ƙimar kuskure akan lokaci.
Hatsari & Tsare-tsare
Haɓaka ma'auni ɗaya na iya ɓoye manyan raunin tsarin.
Sau da yawa ana raina kayan more rayuwa da kuma kuɗin kulawa.
Tsaro da gibin lura na iya girma yayin da tsarin ke ƙara haɓaka.
Taswirar Hanya
Ƙayyade latency, inganci, da maƙasudin farashi kafin aiwatarwa.
Ƙayyade latency, inganci, da maƙasudin farashi kafin aiwatarwa. Ɗauki kowane mataki azaman ƙofar shaida: idan ba a cika sharuɗɗa ba, dakatar da fitar, rufe tazarar, sannan kawai faɗaɗa amfani.
Alamar ma'auni a ƙarƙashin ainihin kaya da yanayin bayanai.
Alamar ma'auni a ƙarƙashin ainihin kaya da yanayin bayanai. Ɗauki kowane mataki azaman ƙofar shaida: idan ba a cika sharuɗɗa ba, dakatar da fitar, rufe tazarar, sannan kawai faɗaɗa amfani.
Kula da kayan aiki don kurakurai, ɗigo, da tasirin mai amfani.
Kula da kayan aiki don kurakurai, ɗigo, da tasirin mai amfani. Ɗauki kowane mataki azaman ƙofar shaida: idan ba a cika sharuɗɗa ba, dakatar da fitar, rufe tazarar, sannan kawai faɗaɗa amfani.
Shirya bijirowa da hanyoyin mayar da martani kafin sikeli.
Shirya bijirowa da hanyoyin mayar da martani kafin sikeli. Ɗauki kowane mataki azaman ƙofar shaida: idan ba a cika sharuɗɗa ba, dakatar da fitar, rufe tazarar, sannan kawai faɗaɗa amfani.