Uhlolojikelele
I-HiFi-GAN iyivowuda yokukhiqiza eguqula i-mel-spectrogram ibe i-waveform yomsindo ongahluziwe cishe ngaso leso sikhathi, ikhiqize inkulumo yekhwalithi yesitudiyo ngokushesha kakhulu kunesikhathi sangempela. Kube yisigaba sokugcina esijwayelekile sokuguqula umbhalo ube inkulumo ngoba kuyashesha, kulula, futhi kunzima ukuhlukanisa nokurekhodwa kwangempela.
I-HiFi-GAN kanye ne-GAN Vocoders ihlezi ku-audio-AI workflows eguqula inkulumo, umculo, nomsindo wokuxhumana, ukufinyeleleka, nokukhiqizwa kwemidiya.
I-Deep Dive
Ivokhoda iyisinyathelo sokugcina kumapayipi amaningi e-TTS: imodeli efana ne-Tacotron noma i-FastSpeech ibikezela i-mel-spectrogram (isithombe esihlangene samafrikhwensi ngokuhamba kwesikhathi), futhi i-vokhoda igcwalisa amasampuli angempela e-waveform. Ama-vocoder asekuqaleni kwe-neural afana ne-WaveNet azwakala emuhle kodwa akhiqizwe amasampula omsindo ngesampula, okuwenza angasheshi kabuhlungu. I-HiFi-GAN, ekhishwe ngu-Kong, Kim, kanye no-Bae ngo-2020, ithathele indawo leyo luphu ye-autoregressive generator eyodwa yokuphakela phambili eqeqeshelwe ukulwa. Iqhinga layo eliyinhloko ukusebenzisa ababandlululi abaningi abahlulela umsindo ngezikali ezihlukene kanye namaphethini ezikhathi ezithile, okuphoqelela ijeneretha ukuthi ithole kokubili ukuthungwa okuhle kanye ne-pitch periodicity efanele. Umphumela uba inkulumo engu-22 kHz ehlanganiswe ngokushesha izikhathi ezingamakhulu kunesikhathi sangempela ku-GPU, ngomsindo weqiniso ophikisanayo wekhwalithi.
I-Technical Insight
Ijeneretha ye-HiFi-GAN iphakamisa isampula ye-mel-spectrogram ngokusebenzisa ama-convolutions aguquguqukayo, namabhulokhi astakiwe e-Multi-Receptive Field ahlanganisa osayizi abahlukahlukene be-kernel nokwenyuswa ukuze athwebule amaphethini amagagasi ahlukahlukene. Imindeni emibili ebandlululayo yenza umsebenzi wobuphoyisa: I-Multi-Period Discriminator iphinda ilolonge isignali ye-1D ibe amagridi e-2D ngesikhathi sokuqala njengo-2, 3, 5, 7, 11 ukuze ibambe i-pitch periodicity, futhi I-Multi-Scale Discriminator ihlola i-waveform ezinqumweni ezimbalwa ezehlisiwe. I-Mel-spectrogram nokulahlekelwa okuhambisana nesici kugcina ukuqeqeshwa kuzinzile.
I-Mastering HiFi-GAN kanye namaVokhoda e-GAN
I-HiFi-GAN iyivowuda yokukhiqiza eguqula i-mel-spectrogram ibe i-waveform yomsindo ongahluziwe cishe ngaso leso sikhathi, ikhiqize inkulumo yekhwalithi yesitudiyo ngokushesha kakhulu kunesikhathi sangempela. Kube yisigaba sokugcina esijwayelekile sokuguqula umbhalo ube inkulumo ngoba kuyashesha, kulula, futhi kunzima ukuhlukanisa nokurekhodwa kwangempela. I-HiFi-GAN kanye ne-GAN Vocoders ihlezi ku-audio-AI workflows eguqula inkulumo, umculo, nomsindo wokuxhumana, ukufinyeleleka, nokukhiqizwa kwemidiya. Ukuze wakhe ukuqonda okujulile, phatha i-HiFi-GAN ne-GAN Vocoder njengemodeli yokusebenza, hhayi isici esisodwa: chaza imiphumela efiselekayo, ucacise ukucabanga, futhi uhlukanise lokho uhlelo olungakwenza ngokwethembeka kulokho okusadinga ukwahlulela kochwepheshe.
Empeleni, amaqembu aqinile asebenzisa i-HiFi-GAN ne-GAN Vocoder aphatha ikhwalithi, ukubambezeleka, kanye nemvume njengezingxenye ezibaluleke ngokulinganayo zesu lokuthumela. Babhala imibandela yempumelelo ecacile, ukuhlola okuqhathaniswa nedatha engokoqobo nokugeleza komsebenzi, futhi baphindaphinde ngokusekelwe kumaphethini okuhluleka aqashiwe esikhundleni sokuwina kwebhentshimakhi yesikhathi esisodwa. Yilapho ukuqonda kwethiyori kuguquka kube amandla ahlala njalo kuwo wonke umkhiqizo, inqubomgomo, kanye nokusebenza.
Ithuthukisa ukufinyeleleka ngokuloba, ukulandisa, nezixhumi ezibonakalayo zezwi. Ngesikhathi esifanayo, ukusetshenziswa kabi kwezwi kanye nezingozi zokuzenza ongeyena ziyakhuphuka uma imvume ingekho. Indlela eqine kakhulu iwukuhlanganisa isivinini sokuhlola nesiyalo sokuphatha: qhuba abashayeli bezindiza, bamba ubufakazi, ushicilele amalogi ezinqumo, futhi ubuyekeze izivikelo ngokuqhubekayo njengoba imodeli yokuziphatha, okulindelwe ngabasebenzisi, kanye nezimfuneko zokulawula zishintsha.
I-Strategic Impact
Ithuthukisa ukufinyeleleka ngokuloba, ukulandisa, nezixhumi ezibonakalayo zezwi.
Ithuthukisa ukufinyeleleka ngokuloba, ukulandisa, nezixhumi ezibonakalayo zezwi. Ekusetshenzisweni kwekhwalithi ephezulu, lokhu kuhunyushwa emithethweni yokusebenza elinganisekayo, imingcele yobunikazi, nemikhuba yokubuyekeza ephindelelayo ukuze amaqembu akwazi ukukala ukuzethemba esikhundleni sokukala ukungaqondakali.
Amaqembu emidiya angathumela umsindo opholishiwe ngokushesha ngamabhajethi amancane.
Amaqembu emidiya angathumela umsindo opholishiwe ngokushesha ngamabhajethi amancane. Ekusetshenzisweni kwekhwalithi ephezulu, lokhu kuhunyushwa emithethweni yokusebenza elinganisekayo, imingcele yobunikazi, nemikhuba yokubuyekeza ephindelelayo ukuze amaqembu akwazi ukukala ukuzethemba esikhundleni sokukala ukungaqondakali.
Amasistimu abhekene nekhasimende angacubungula ukusebenzelana okukhulunyiwe ngesilinganiso esikhulu.
Amasistimu abhekene nekhasimende angacubungula ukusebenzelana okukhulunyiwe ngesilinganiso esikhulu. Ekusetshenzisweni kwekhwalithi ephezulu, lokhu kuhunyushwa emithethweni yokusebenza elinganisekayo, imingcele yobunikazi, nemikhuba yokubuyekeza ephindelelayo ukuze amaqembu akwazi ukukala ukuzethemba esikhundleni sokukala ukungaqondakali.
Ukuqaliswa Komhlaba Wangempela
Ikhiqiza okukhiphayo okukhulunyiwe kwabasizi ababonakalayo nezinhlelo zokusebenza zokuzulazula ezidinga izimpendulo ngaphandle kokulibaziseka okuzwakalayo.
Inika amandla amathuluzi e-cloning wezwi lesikhathi sangempela kanye namathuluzi wokukopisha lapho i-mel-spectrogram ehlanganisiwe inikezwa umsindo ozwakalayo wemvelo.
Ukushayela i-audiobook kanye nezinkundla zokulandisa ze-podcast ezihlanganisa amahora okukhuluma ngokushesha naphansi.
Isebenza njengesiteji se-waveform ngaphakathi kwama-synthesizers okucula-izwi namademo omculo ngamavokhoda asemhlabeni wonke esitayela se-BigVGAN.
Amaphethini Okusebenzisa
I-HiFi-GAN ne-GAN Vocoder iyasebenza
Ikhiqiza okukhiphayo okukhulunyiwe kwabasizi ababonakalayo nezinhlelo zokusebenza zokuzulazula ezidinga izimpendulo ngaphandle kokulibaziseka okuzwakalayo.
Ukukhiqiza okukhiphayo okukhulumayo kwabasizi ababonakalayo nezinhlelo zokusebenza zokuzulazula ezidinga izimpendulo ngaphandle kokulibaziseka okuzwakalayo Amaqembu ngokuvamile athola imiphumela engcono uma echaza imikhawulo yekhwalithi ngaphambili, egcina indlela yokukhuphuka yabantu yamakesi asemaphethelweni, futhi alandelela kokubili izinzuzo zokukhiqiza nezindleko zamaphutha ngokuhamba kwesikhathi.
I-HiFi-GAN ne-GAN Vocoder iyasebenza
Inika amandla amathuluzi e-cloning wezwi lesikhathi sangempela kanye namathuluzi wokukopisha lapho i-mel-spectrogram ehlanganisiwe inikezwa umsindo ozwakalayo wemvelo.
Ukunika amandla amathuluzi wokuhlanganisa izwi wesikhathi sangempela namathuluzi wokukopisha lapho i-mel-spectrogram ekhiqiziwe inikezwa Amathimba omsindo wemvelo ngokuvamile athola imiphumela engcono lapho echaza imingcele yekhwalithi ngaphambili, egcina indlela yokukhuphuka yomuntu yamakesi asemaphethelweni, futhi alandelele kokubili izinzuzo zokukhiqiza nezindleko zamaphutha ngokuhamba kwesikhathi.
I-HiFi-GAN ne-GAN Vocoder iyasebenza
Ukushayela i-audiobook kanye nezinkundla zokulandisa ze-podcast ezihlanganisa amahora okukhuluma ngokushesha naphansi.
Ukushayela i-audiobook kanye nenkundla yokulandisa ye-podcast ehlanganisa amahora okukhuluma ngokushesha nangeshibhile Amathimba ngokuvamile athola imiphumela engcono uma echaza ikhwalithi ephezulu ngaphambili, egcina indlela yokukhuphuka yabantu yamakesi asemaphethelweni, futhi alandelele kokubili izinzuzo zokukhiqiza nezindleko zamaphutha ngokuhamba kwesikhathi.
I-HiFi-GAN ne-GAN Vocoder iyasebenza
Isebenza njengesiteji se-waveform ngaphakathi kwama-synthesizers okucula-izwi namademo omculo ngamavokhoda asemhlabeni wonke esitayela se-BigVGAN.
Isebenza njengesiteji se-waveform ngaphakathi kwama-synthesizers okucula-izwi namademo omculo ngesitayela se-BigVGAN sama-vocoder Amaqembu ngokuvamile athola imiphumela engcono uma echaza imingcele yekhwalithi ngaphambili, agcina indlela yokukhuphuka komuntu yamakesi asemaphethelweni, futhi alandelele kokubili izinzuzo zokukhiqiza nezindleko zamaphutha ngokuhamba kwesikhathi.
Izingozi & Guardrails
Ukusetshenziswa kabi kwezwi kanye nezingozi zokuzenza ongeyena ziyanda uma imvume ingekho.
Ukunemba kungase kwehle kuzo zonke izinhlobo zokuphimisela, izilimi zesigodi, noma izindawo ezinomsindo.
Umsindo wokwenziwa ungenziwa iphutha njengenkulumo eyiqiniso ngaphandle kokulebula okucacile.
Ukuqalisa Umhlahlandlela
Thola imvume esobala yokuthwebula izwi, ukuhlanganisa, nokusebenzisa kabusha.
Thola imvume esobala yokuthwebula izwi, ukuhlanganisa, nokusebenzisa kabusha. Phatha isinyathelo ngasinye njengesango lobufakazi: uma imibandela ingafinyelelwa, misa ukukhishwa, vala igebe, bese unweba ukusetshenziswa.
Ikhwalithi yokuhlola kuzo zonke izipikha nezimo zangemuva.
Ikhwalithi yokuhlola kuzo zonke izipikha nezimo zangemuva. Phatha isinyathelo ngasinye njengesango lobufakazi: uma imibandela ingafinyelelwa, misa ukukhishwa, vala igebe, bese unweba ukusetshenziswa.
Chaza ukuthi kunini lapho umuntu kufanele abuyekeze noma agunyaze okuphumayo.
Chaza ukuthi kunini lapho umuntu kufanele abuyekeze noma agunyaze okuphumayo. Phatha isinyathelo ngasinye njengesango lobufakazi: uma imibandela ingafinyelelwa, misa ukukhishwa, vala igebe, bese unweba ukusetshenziswa.
Lebula umsindo wokwenziwa futhi ugcine amarekhodi atholakalayo ukuze aziphendulele.
Lebula umsindo wokwenziwa futhi ugcine amarekhodi atholakalayo ukuze aziphendulele. Phatha isinyathelo ngasinye njengesango lobufakazi: uma imibandela ingafinyelelwa, misa ukukhishwa, vala igebe, bese unweba ukusetshenziswa.