Dubawa
Ƙididdiga ta ƙasa (wanda kuma ake kira maƙasudin kunnawa) dabara ce ta adana ƙwaƙwalwar ajiya wacce ke watsar da mafi yawan kunnawa na tsaka-tsaki yayin wucewar gaba da sake ƙididdige su akan tashi yayin yaɗa baya. Yana ba ku damar horar da zurfafa, manyan cibiyoyin sadarwa ta hanyar yin ciniki da ƙarin ƙididdigewa don ƙarancin amfani da ƙwaƙwalwar ajiya.
Binciken Gradient tubalan ginin fasaha ne wanda ke shafar ingancin samfuri, farashin kayayyakin more rayuwa, jinkiri, da aminci a sikeli.
Zurfafa nutsewa
Horar da hanyoyin sadarwa na jijiyoyi galibi suna adana ayyukan kunnawa kowane Layer yayin wucewar gaba saboda yada baya yana buƙatar su don ƙididdige gradients. Don ƙira mai zurfi waɗannan kunnawa suna mamaye ƙwaƙwalwar ajiya. Ma'aunin bincike a hankali a maimakon haka yana adana kunnawa kawai a cikin ɗimbin saiti na yadudduka 'Checkpoint' kuma yana watsar da sauran. Lokacin da backprop ya isa yankin da aka daina kunnawa, yana sake aiwatar da lissafin gaba don kawai wannan ɓangaren don sake haɓaka abin da yake buƙata, sannan ya ci gaba. Tare da wuraren bincike da aka sanya kusan kowane murabba'i-tushen-na-N, ƙwaƙwalwar ajiya don kunnawa tana raguwa daga oda N don yin odar murabba'in-na-N, yayin da ƙididdigewa ya tashi da kusan ƙarin wucewar gaba ɗaya kawai (kusan 20-30% a hankali). Wannan yana ba da damar dacewa da manyan nau'ikan batch ko mafi zurfi masu canzawa akan GPU iri ɗaya.
Fahimtar Fasaha
Dabarar tana amfani da cinikin lokaci-da-memori. Ajiye duk kunnawa yana da sauri amma ƙwaƙwalwar ƙwaƙwalwa-yunwa; sake lissafta su yana da arha akan na'urori na zamani dangane da tsadar ƙarancin ƙwaƙwalwar ajiya. Tsari kamar PyTorch (torch.utils.checkpoint) nannade wani module don haka ana iya yin lissafin abubuwan da ke cikin sa a baya. Zaɓin abubuwan wurin bincike: ko da tazara na kusan sassan sqrt(N) yana rage girman ƙwaƙwalwar ajiya yayin ƙara ƙarin faci ɗaya kawai na ƙididdige gabaɗaya.
Ƙwararren Ƙwararren Ƙwararru
Ƙididdiga ta ƙasa (wanda kuma ake kira maƙasudin kunnawa) dabara ce ta adana ƙwaƙwalwar ajiya wacce ke watsar da mafi yawan kunnawa na tsaka-tsaki yayin wucewar gaba da sake ƙididdige su akan tashi yayin yaɗa baya. Yana ba ku damar horar da zurfafa, manyan cibiyoyin sadarwa ta hanyar yin ciniki da ƙarin ƙididdigewa don ƙarancin amfani da ƙwaƙwalwar ajiya. Binciken Gradient tubalan ginin fasaha ne wanda ke shafar ingancin samfuri, farashin kayayyakin more rayuwa, jinkiri, da aminci a sikeli. Don gina zurfin fahimta, kula da Binciken Bincike na Gradient azaman ƙirar aiki, ba fasali ɗaya ba: ayyana sakamakon da ake so, fayyace zato, da raba abin da tsarin zai iya yi da dogaro daga abin da har yanzu ke buƙatar yanke hukunci na ƙwararru.
A aikace, ƙungiyoyi masu ƙarfi da ke amfani da Gradient Checkpointing suna haɓaka gine-gine, bayanai, da zaɓin abubuwan more rayuwa tare da dogaro da farashi. Suna rubuta ƙayyadaddun ƙa'idodin nasara, gwaji akan bayanan gaskiya da gudanawar aiki, da jujjuyawar bisa ga tsarin gazawar da aka lura maimakon cin nasara na lokaci ɗaya. Wannan shine inda fahimtar ka'idar ta juya zuwa iyawa mai dorewa a cikin samfura, manufofi, da ayyuka.
Hukunce-hukuncen gine-gine suna haifar da aiki da tsadar aiki na shekaru. A lokaci guda, Haɓaka ma'auni ɗaya na iya ɓoye manyan raunin tsarin. Hanyar da ta fi dacewa ita ce haɗa saurin gwaji tare da horon gudanarwa: gudanar da matukin jirgi, kama shaida, buga rajistan ayyukan yanke shawara, da ci gaba da sabunta abubuwan tsaro kamar yadda halayen ƙira, tsammanin mai amfani, da buƙatun tsari ke tasowa.
Dabarun Tasiri
Hukunce-hukuncen gine-gine suna haifar da aiki da tsadar aiki na shekaru.
Hukunce-hukuncen gine-gine suna haifar da aiki da tsadar aiki na shekaru. A cikin ƙawance masu inganci, ana fassara wannan zuwa ƙa'idodin aiki waɗanda za a iya aunawa, iyakokin ikon mallaka, da kuma bita-da-kullin bita don ƙungiyoyi su iya haɓaka kwarin gwiwa a maimakon ɓata shakku.
Ilimin fasaha yana taimaka wa ƙungiyoyi su zaɓi tari mai kyau, ba kawai sabon abu ba.
Ilimin fasaha yana taimaka wa ƙungiyoyi su zaɓi tari mai kyau, ba kawai sabon abu ba. A cikin ƙawance masu inganci, ana fassara wannan zuwa ƙa'idodin aiki waɗanda za a iya aunawa, iyakokin ikon mallaka, da kuma bita-da-kullin bita don ƙungiyoyi su iya haɓaka kwarin gwiwa a maimakon ɓata shakku.
Zaɓuɓɓukan injiniya mafi kyau suna rage abin dogaro a cikin samarwa.
Zaɓuɓɓukan injiniya mafi kyau suna rage abin dogaro a cikin samarwa. A cikin ƙawance masu inganci, ana fassara wannan zuwa ƙa'idodin aiki waɗanda za a iya aunawa, iyakokin ikon mallaka, da kuma bita-da-kullin bita don ƙungiyoyi su iya haɓaka kwarin gwiwa a maimakon ɓata shakku.
Aiwatar da Gaskiyar Duniya
Horar da na'ura mai zurfi tare da girman tsari mai girma akan GPU guda ta hanyar watsar da sake lissafin kunnawa Layer.
Kyakkyawan samfurin hangen nesa akan hotuna masu tsayi inda taswirorin kunnawa zasu mamaye ƙwaƙwalwar GPU.
Rungumar Face Transformers yana ba da damar gradient_checkpointing=Gaskiya ya dace da ƙirar sigar biliyan-biliyan yayin daidaitawa.
Haɗa wuraren bincike tare da FSDP don haka duka sigogi da kunnawa ana kiyaye su ƙanana, suna ba da damar horar da samfuran manyan harshe.
Hanyoyin Aiwatarwa
Tukwici na Gradient a aikace
Horar da na'ura mai zurfi tare da girman tsari mai girma akan GPU guda ta hanyar watsar da sake lissafin kunnawa Layer.
Horar da mai canji mai zurfi tare da girman tsari mai girma akan GPU guda ɗaya ta hanyar watsar da sake ƙididdige yawan kunna aikin Layer Ƙungiyoyi yawanci suna samun sakamako mafi kyau lokacin da suka ayyana ma'auni masu inganci a gaba, kiyaye hanyar haɓakar ɗan adam don ƙararraki, da bin diddigin nasarorin samarwa da ƙimar kuskure akan lokaci.
Tukwici na Gradient a aikace
Kyakkyawan samfurin hangen nesa akan hotuna masu tsayi inda taswirorin kunnawa zasu mamaye ƙwaƙwalwar GPU.
Kyakkyawan ƙirar hangen nesa akan hotuna masu ƙarfi inda taswirorin kunnawa in ba haka ba za su cika Ƙungiyoyin ƙwaƙwalwar ajiya na GPU yawanci suna samun sakamako mafi kyau lokacin da suka ayyana ƙofofin inganci a gaba, kiyaye hanyar haɓakar ɗan adam don shari'o'in gefe, da bin duk nasarorin samarwa da ƙimar kuskure akan lokaci.
Tukwici na Gradient a aikace
Rungumar Face Transformers yana ba da damar gradient_checkpointing=Gaskiya ya dace da ƙirar sigar biliyan-biliyan yayin daidaitawa.
Rungumar Face Transformers yana ba da damar gradient_checkpointing=Gaskiya don dacewa da nau'ikan sigar biliyan biliyan yayin daidaitawa Ƙungiyoyi yawanci suna samun sakamako mafi kyau lokacin da suka ayyana ma'auni masu inganci a gaba, kiyaye hanyar haɓakar ɗan adam don shari'o'in ƙira, da bin diddigin nasarorin samarwa da tsadar kurakurai a kan lokaci.
Tukwici na Gradient a aikace
Haɗa wuraren bincike tare da FSDP don haka duka sigogi da kunnawa ana kiyaye su ƙanana, suna ba da damar horar da samfuran manyan harshe.
Haɗa wuraren bincike tare da FSDP don haka duka sigogi da kunnawa ana kiyaye su ƙanana, yana ba da damar horar da manyan nau'ikan harshe Ƙungiyoyi yawanci suna samun sakamako mafi kyau lokacin da suka ayyana ma'auni masu inganci a gaba, kiyaye hanyar haɓakar ɗan adam don shari'o'in gefe, da bin duk nasarorin samarwa da farashi na kuskure a kan lokaci.
Hatsari & Tsare-tsare
Haɓaka ma'auni ɗaya na iya ɓoye manyan raunin tsarin.
Sau da yawa ana raina kayan more rayuwa da kuma kuɗin kulawa.
Tsaro da gibin lura na iya girma yayin da tsarin ke ƙara haɓaka.
Taswirar Hanya
Ƙayyade latency, inganci, da maƙasudin farashi kafin aiwatarwa.
Ƙayyade latency, inganci, da maƙasudin farashi kafin aiwatarwa. Ɗauki kowane mataki azaman ƙofar shaida: idan ba a cika sharuɗɗa ba, dakatar da fitar, rufe tazarar, sannan kawai faɗaɗa amfani.
Alamar ma'auni a ƙarƙashin ainihin kaya da yanayin bayanai.
Alamar ma'auni a ƙarƙashin ainihin kaya da yanayin bayanai. Ɗauki kowane mataki azaman ƙofar shaida: idan ba a cika sharuɗɗa ba, dakatar da fitar, rufe tazarar, sannan kawai faɗaɗa amfani.
Kula da kayan aiki don kurakurai, ɗigo, da tasirin mai amfani.
Kula da kayan aiki don kurakurai, ɗigo, da tasirin mai amfani. Ɗauki kowane mataki azaman ƙofar shaida: idan ba a cika sharuɗɗa ba, dakatar da fitar, rufe tazarar, sannan kawai faɗaɗa amfani.
Shirya bijirowa da hanyoyin mayar da martani kafin sikeli.
Shirya bijirowa da hanyoyin mayar da martani kafin sikeli. Ɗauki kowane mataki azaman ƙofar shaida: idan ba a cika sharuɗɗa ba, dakatar da fitar, rufe tazarar, sannan kawai faɗaɗa amfani.