Overview
HiFi-GAN inogadzira-adversarial vocoder inoshandura mel-spectrogram kuita yakasvibira odhiyo waveform kanenge ipapo, ichigadzira studio-yemhando yekutaura nekukurumidza kupfuura nguva chaiyo. Yakave nhanho yekupedzisira yeazvino zvinyorwa-ku-kutaura nekuti inokurumidza, isingaremi, uye yakaoma kusiyanisa kubva kune chaiyo yakarekodhwa.
HiFi-GAN neGAN Vocoders inogara muodhiyo-AI workflows inoshandura kutaura, mimhanzi, uye ruzha rwekutaurirana, kuwanikwa, uye kugadzirwa kwenhau.
Deep Dive
Vocoder inhanho yekupedzisira mumapaipi mazhinji eTTS: modhi yakaita seTacotron kana FastSpeech inofanotaura mel-spectrogram (yakabatana mufananidzo wefrequency nekufamba kwenguva), uye vocoder inozadza iwo chaiwo waveform samples. Ekutanga neural vocoder seWaveNet yakanzwika yakakura asi yakagadzira odhiyo sampuli-ne-sample, ichiita kuti inonoke zvinorwadza. HiFi-GAN, yakaburitswa naKong, Kim, naBae muna 2020, yakatsiva iyo autoregressive loop nejenareta rimwe chete rekupa-mberi rakadzidziswa kurwisa. Hunyengeri hwayo hwekushandisa kushandisa rusarura rwakati wandei rwunotonga odhiyo pazvikero zvakasiyana uye pamusoro peakasiyana periodic mapatani, achimanikidza jenareta kuti iwane zvese zviri zviviri manyorerwo akanaka uye pitch periodicity kurudyi. Mhedzisiro yacho ndeye 22 kHz kutaura kwakagadzirwa mazana enguva nekukurumidza pane chaiyo nguva paGPU, ine mhando inokwikwidza pasi-chokwadi odhiyo.
Technical Insight
HiFi-GAN's jenareta inosimudzira iyo mel-spectrogram kuburikidza neyakachinjika convolutions, ine akaturikidzana Multi-Receptive Field mabhuroko anosanganisa akasiyana kernel saizi uye dilations kutora akasiyana mafungu maitiro. Mhuri mbiri dzerusarura dzinoita zvechipurisa: Multi-Period Discriminator inogadzirisa chiratidzo che1D kuita 2D grids pamaprimes se2, 3, 5, 7, 11 kubata pitch periodicity, uye Multi-Scale Discriminator inoongorora waveform pane akati wandei akaderedzwa sarudzo. Mel-spectrogram uye kurasikirwa-kufananidza kurasikirwa kunochengeta kudzidziswa kwakadzikama.
Kugona HiFi-GAN uye GAN Vocoders
HiFi-GAN inogadzira-adversarial vocoder inoshandura mel-spectrogram kuita yakasvibira odhiyo waveform kanenge ipapo, ichigadzira studio-yemhando yekutaura nekukurumidza kupfuura nguva chaiyo. Yakave nhanho yekupedzisira yeazvino zvinyorwa-ku-kutaura nekuti inokurumidza, isingaremi, uye yakaoma kusiyanisa kubva kune chaiyo yakarekodhwa. HiFi-GAN neGAN Vocoders inogara muodhiyo-AI workflows inoshandura kutaura, mimhanzi, uye ruzha rwekutaurirana, kuwanikwa, uye kugadzirwa kwenhau. Kuvaka kunzwisisa kwakadzama, tora HiFi-GAN neGAN Vocoder semuenzaniso wekushandisa, kwete chinhu chimwe chete: tsanangura zvaunoda mhedzisiro, kujekesa fungidziro, uye patsanura izvo zvinogona kuitwa nehurongwa hwakavimbika kubva kune zvichiri kuda kutonga kwenyanzvi.
Mukuita, zvikwata zvakasimba zvinoshandisa HiFi-GAN uye GAN Vocoders zvinobata mhando, latency, uye mvumo sezvikamu zvakakosha zvakaenzana zvehurongwa hwekuendesa. Ivo vanonyora zvakajeka maitiro ebudiriro, bvunzo vachipokana ne data rechokwadi uye mafambiro ebasa, uye iterate zvichibva pane zvakacherechedzwa maitiro ekutadza kwete kuhwina-nguva imwe chete yebhenji. Apa ndipo apo kunzwisisa kwe theoretical kunoshanduka kuve kugona kwakasimba pane chigadzirwa, mutemo, uye mashandiro.
Inonatsiridza kusvikika kuburikidza nekunyora, kurondedzera, uye mazwi ekubatanidza. Panguva imwecheteyo, kusashandiswa kweIzwi zvisizvo uye njodzi dzekuedzesera dzinowedzera kana chibvumirano chisipo. Nzira yakatsiga ndeyekubatanidza kukurumidza kuyedza nekutonga: mhanyisa vatyairi vendege, tora humbowo, buritsa matanda esarudzo, uye urambe uchivandudza chengetedzo semaitiro emuenzaniso, zvinotarisirwa nemushandisi, uye zvinodikanwa zvekutonga.
Strategic Impact
Inonatsiridza kusvikika kuburikidza nekunyora, kurondedzera, uye mazwi ekubatanidza.
Inonatsiridza kusvikika kuburikidza nekunyora, kurondedzera, uye mazwi ekubatanidza. Mukutumirwa kwemhando yepamusoro, izvi zvinoshandurirwa kuita mitemo inoyerwa yekushanda, miganhu yevaridzi, uye tsika dzekudzokorora dzinodzokororwa kuitira kuti zvikwata zvikwire kuvimba pane kukwidza kusajeka.
Zvikwata zveMedia zvinogona kutumira odhiyo yakakwenenzverwa nekukurumidza nemabhajeti madiki.
Zvikwata zveMedia zvinogona kutumira odhiyo yakakwenenzverwa nekukurumidza nemabhajeti madiki. Mukutumirwa kwemhando yepamusoro, izvi zvinoshandurirwa kuita mitemo inoyerwa yekushanda, miganhu yevaridzi, uye tsika dzekudzokorora dzinodzokororwa kuitira kuti zvikwata zvikwire kuvimba pane kukwidza kusajeka.
Masisitimu anotarisana nevatengi anogona kugadzirisa kutaurirana kwekutaura pamwero mukuru.
Masisitimu anotarisana nevatengi anogona kugadzirisa kutaurirana kwekutaura pamwero mukuru. Mukutumirwa kwemhando yepamusoro, izvi zvinoshandurirwa kuita mitemo inoyerwa yekushanda, miganhu yevaridzi, uye tsika dzekudzokorora dzinodzokororwa kuitira kuti zvikwata zvikwire kuvimba pane kukwidza kusajeka.
Real-World Implementation
Kugadzira kuburitswa kwekutaura kwevabatsiri uye maapplication ekufambisa ayo anoda mhinduro pasina kunonoka kunonzwika.
Kupa simba-chaiyo-nguva izwi cloning uye dubbing maturusi uko cloned mel-spectrogram inopihwa mune yakasikwa-inonzwika odhiyo.
Kutyaira audiobook uye podcast kurondedzera mapuratifomu anogadzira maawa ekutaura nekukurumidza uye zvakachipa.
Kushanda senge waveform nhanho mukati mekuimba-izwi synthesizer uye mimhanzi demos kuburikidza neBigVGAN-maitiro epasirese vocoder.
Maitiro Ekuita
HiFi-GAN uye GAN Vocoders mukuita
Kugadzira kuburitswa kwekutaura kwevabatsiri uye maapplication ekufambisa ayo anoda mhinduro pasina kunonoka kunonzwika.
Kugadzira kuburitswa kwekutaura kwevabatsiri chaiwo uye maapplication ekufambisa ayo anoda mhinduro pasina kunonoka kunzwika Matimu anowanzo kuwana mhedzisiro iri nani kana achinge atsanangura mhando yepamusoro kumberi, chengetedza nzira yekukwira kwevanhu yemakesi ekumucheto, uye kuteedzera zvese zvakawanikwa zvechigadzirwa nemitengo yekukanganisa nekufamba kwenguva.
HiFi-GAN uye GAN Vocoders mukuita
Kupa simba-chaiyo-nguva izwi cloning uye dubbing maturusi uko cloned mel-spectrogram inopihwa mune yakasikwa-inonzwika odhiyo.
Kupa simba-chaiyo-nguva izwi cloning uye dubbing maturusi uko yakaumbwa mel-spectrogram inopihwa mune yakasikwa-inonzwika odhiyo Matimu anowanzo kuwana mhedzisiro iri nani kana achinge atsanangura emhando yepamusoro kumberi, chengetedza nzira yekukwira kwevanhu yemakesi emupendero, uye kuteedzera zvese zvakawanikwa zvechigadzirwa uye kukanganisa mutengo nekufamba kwenguva.
HiFi-GAN uye GAN Vocoders mukuita
Kutyaira audiobook uye podcast kurondedzera mapuratifomu anogadzira maawa ekutaura nekukurumidza uye zvakachipa.
Kutyaira mabhuku ekuteerera uye podcast kurondedzera mapuratifomu anogadzira maawa ekutaura nekukurumidza uye zvakachipa Matimu anowanzo kuwana mhedzisiro iri nani kana achinge atsanangura emhando yepamusoro kumberi, chengetedza nzira yekukwira kwevanhu yemakesi ekumucheto, uye kuteedzera zvese zvakawanikwa zvechigadzirwa uye mutengo wekukanganisa nekufamba kwenguva.
HiFi-GAN uye GAN Vocoders mukuita
Kushanda senge waveform nhanho mukati mekuimba-izwi synthesizer uye mimhanzi demos kuburikidza neBigVGAN-maitiro epasirese vocoder.
Kushanda senge waveform nhanho mukati mekuimba-izwi synthesizer uye mimhanzi demos kuburikidza neBigVGAN-chimiro chepasirese vocoder Matimu anowanzo kuwana mhedzisiro iri nani kana achinge atsanangura emhando yepamusoro kumberi, chengetedza nzira yekukwira kwevanhu yemakesi emupendero, uye kuteedzera zvese zvakawanikwa zvekubereka uye kukanganisa mutengo nekufamba kwenguva.
Njodzi & Guardrails
Kusashandisa izwi zvisizvo uye njodzi dzekuedzesera dzinowedzera kana chibvumirano chisipo.
Kururama kunogona kudonha mumitauro, mataurirwo, kana nharaunda dzine ruzha.
Synthetic audio inogona kukanganisa kutaura kwechokwadi isina mavara akajeka.
Implementation Roadmap
Wana mvumo yakajeka yekutora inzwi, kugadzira, uye kushandisa zvakare.
Wana mvumo yakajeka yekutora inzwi, kugadzira, uye kushandisa zvakare. Bata nhanho yega yega segedhi rehumbowo: kana maitiro asina kusangana, imbomira kuburitsa, vhara gaka, uye wobva wawedzera kushandiswa.
Yedza mhando pavatauri vakasiyana uye mamiriro ekumashure.
Yedza mhando pavatauri vakasiyana uye mamiriro ekumashure. Bata nhanho yega yega segedhi rehumbowo: kana maitiro asina kusangana, imbomira kuburitsa, vhara gaka, uye wobva wawedzera kushandiswa.
Tsanangura apo munhu anofanira kuongorora kana kubvumidza zvabuda.
Tsanangura apo munhu anofanira kuongorora kana kubvumidza zvabuda. Bata nhanho yega yega segedhi rehumbowo: kana maitiro asina kusangana, imbomira kuburitsa, vhara gaka, uye wobva wawedzera kushandiswa.
Label synthetic odhiyo uye chengetedza marekodhi ekuzvidavirira.
Label synthetic odhiyo uye chengetedza marekodhi ekuzvidavirira. Bata nhanho yega yega segedhi rehumbowo: kana maitiro asina kusangana, imbomira kuburitsa, vhara gaka, uye wobva wawedzera kushandiswa.