Overview
NaturalSpeech mutsara we Microsoft TTS tsvakiridzo yakanangana nemhando yekutaura kwemhando yevanhu, ine shanduro dzepashure dzinoshandisa kudzokororwa kusinganzwisisike kugadzira manzwi akapfuma. Inoratidza kuti mamodheru ekuparadzira, anozivikanwa nemifananidzo, anogona sei kuburitsa inonzwika, inodzoreka odhiyo.
NaturalSpeech uye Latent Diffusion TTS inogara muodhiyo-AI workflows inoshandura kutaura, mimhanzi, uye ruzha rwekutaurirana, kuwanikwa, uye kugadzirwa kwenhau.
Deep Dive
Yekutanga NaturalSpeech (2022) ndiyo yekutanga sisitimu yakashumwa kuti inosvika pamhando yemunhu paiyo LJSpeech benchmark, inotongwa nevateereri vasingakwanise kuitaurira nekuvimbika kubva kurekodhwa chaiko. Yakashandisa yakasarudzika autoencoder ine yakanyatsoenderana pres kuvhara mukaha pakati pekudzidziswa uye inference. NaturalSpeech 2 yakabva yatora nzira yakasarudzika yekuparadzira: kutaura kunovharirwa neural audio codec mune inoenderera yakadzika mavheji, uye modhi yekuparadzira inodzidza kugadzira iwo malatenti kubva muzvinyorwa, ichigonesa rakasimba zero-kupfura izwi cloning kubva pfupi kukurumidza. NaturalSpeech 3 yakaunza factorized diffusion, kupatsanura kutaura kuita hunhu hwakapatsanuka senge zvirimo, prosody, timbre, uye acoustic tsanangudzo, saka imwe neimwe inogona kuenzanisirwa uye kudzorwa yakazvimiririra kune yakakwirira kutendeka uye kuchinjika.
Technical Insight
Latent diffusion inoshanda nekuwedzera ruzha kune compact latent inomiririra yekutaura uye kudzidzisa network kudzosera iyo ruzha nhanho nhanho. Panzvimbo pekuita denoising waveforms mbishi kana yakazara spectrogram, NaturalSpeech 2 denoises codec latent, iyo yakaderera-dimensional uye iri nyore kutevedzera. Mamiriro ezvinyorwa uye izwi rereferenzi rinotungamira kuchinjika, saka iyo yekupedzisira sampled latent inosarudza mukutaura kunoenderana nezvakumbirwa uye kuzivikanwa kwemutauri.
Kudzidzira NaturalSpeech uye Latent Diffusion TTS
NaturalSpeech mutsara we Microsoft TTS tsvakiridzo yakanangana nemhando yekutaura kwemhando yevanhu, ine shanduro dzepashure dzinoshandisa kudzokororwa kusinganzwisisike kugadzira manzwi akapfuma. Inoratidza kuti mamodheru ekuparadzira, anozivikanwa nemifananidzo, anogona sei kuburitsa inonzwika, inodzoreka odhiyo. NaturalSpeech uye Latent Diffusion TTS inogara muodhiyo-AI workflows inoshandura kutaura, mimhanzi, uye ruzha rwekutaurirana, kuwanikwa, uye kugadzirwa kwenhau. Kuvaka kunzwisisa kwakadzama, tora NaturalSpeech uye Latent Diffusion TTS semuenzaniso wekushandisa, kwete chinhu chimwe chete: tsanangura zvinodiwa, kujekesa fungidziro, uye patsanura zvinogona kuitwa nehurongwa hwakavimbika kubva kune zvichiri kuda kutonga kwenyanzvi.
Mukuita, zvikwata zvakasimba zvinoshandisa NaturalSpeech uye Latent Diffusion TTS zvinobata mhando, latency, uye mvumo sezvikamu zvakakosha zvakaenzana zvehurongwa hwekuendesa. Ivo vanonyora zvakajeka maitiro ebudiriro, bvunzo vachipokana ne data rechokwadi uye mafambiro ebasa, uye iterate zvichibva pane zvakacherechedzwa maitiro ekutadza kwete kuhwina-nguva imwe chete yebhenji. Apa ndipo apo kunzwisisa kwe theoretical kunoshanduka kuve kugona kwakasimba pane chigadzirwa, mutemo, uye mashandiro.
Inonatsiridza kusvikika kuburikidza nekunyora, kurondedzera, uye mazwi ekubatanidza. Panguva imwecheteyo, kusashandiswa kweIzwi zvisizvo uye njodzi dzekuedzesera dzinowedzera kana chibvumirano chisipo. Nzira yakatsiga ndeyekubatanidza kukurumidza kuyedza nekutonga: mhanyisa vatyairi vendege, tora humbowo, buritsa matanda esarudzo, uye urambe uchivandudza chengetedzo semaitiro emuenzaniso, zvinotarisirwa nemushandisi, uye zvinodikanwa zvekutonga.
Strategic Impact
Inonatsiridza kusvikika kuburikidza nekunyora, kurondedzera, uye mazwi ekubatanidza.
Inonatsiridza kusvikika kuburikidza nekunyora, kurondedzera, uye mazwi ekubatanidza. Mukutumirwa kwemhando yepamusoro, izvi zvinoshandurirwa kuita mitemo inoyerwa yekushanda, miganhu yevaridzi, uye tsika dzekudzokorora dzinodzokororwa kuitira kuti zvikwata zvikwire kuvimba pane kukwidza kusajeka.
Zvikwata zveMedia zvinogona kutumira odhiyo yakakwenenzverwa nekukurumidza nemabhajeti madiki.
Zvikwata zveMedia zvinogona kutumira odhiyo yakakwenenzverwa nekukurumidza nemabhajeti madiki. Mukutumirwa kwemhando yepamusoro, izvi zvinoshandurirwa kuita mitemo inoyerwa yekushanda, miganhu yevaridzi, uye tsika dzekudzokorora dzinodzokororwa kuitira kuti zvikwata zvikwire kuvimba pane kukwidza kusajeka.
Masisitimu anotarisana nevatengi anogona kugadzirisa kutaurirana kwekutaura pamwero mukuru.
Masisitimu anotarisana nevatengi anogona kugadzirisa kutaurirana kwekutaura pamwero mukuru. Mukutumirwa kwemhando yepamusoro, izvi zvinoshandurirwa kuita mitemo inoyerwa yekushanda, miganhu yevaridzi, uye tsika dzekudzokorora dzinodzokororwa kuitira kuti zvikwata zvikwire kuvimba pane kukwidza kusajeka.
Real-World Implementation
Dubbing masitudiyo anotevedzera izwi remutambi kubva kumuenzaniso mupfupi kuenda kunzvimbo yemafirimu, vachishandisa NaturalSpeech 2-style zero-shot cloning.
Audiobook mapuratifomu anogadzira nhanho-yemunhu iyo vateereri vanonetseka kusiyanisa kubva kune chaiyo tarenda rezwi.
Maturusi ekugonesa anogadzirisazve izwi remunhu kubva kune akarekodhwa ekare kune avo vakarasikirwa nekutaura.
Zvemukati zvekugadzira masutu regai vapepeti vazvigadzirise vakazvimirira timbre uye prosody, inosimudzira NaturalSpeech 3's factorized hunhu.
Maitiro Ekuita
NaturalSpeech uye Latent Diffusion TTS mukuita
Dubbing masitudiyo anotevedzera izwi remutambi kubva kumuenzaniso mupfupi kuenda kunzvimbo yemafirimu, vachishandisa NaturalSpeech 2-style zero-shot cloning.
Dubbing studios inotevedzera izwi remutambi kubva kupfupi sampuli kuenda kune mamwe mafirimu, vachishandisa NaturalSpeech 2-style zero-shot cloning Matimu anowanzo kuwana mhedzisiro iri nani kana achinge atsanangura emhando yepamusoro kumberi, chengetedza nzira yekukwira kwevanhu yemakesi ekumucheto, uye kuteedzera zvese zvakawanikwa zvechigadzirwa uye mutengo wekukanganisa nekufamba kwenguva.
NaturalSpeech uye Latent Diffusion TTS mukuita
Audiobook mapuratifomu anogadzira nhanho-yemunhu iyo vateereri vanonetseka kusiyanisa kubva kune chaiyo tarenda rezwi.
Audiobook mapuratifomu anoburitsa rondedzero yevanhu iyo vateereri vanonetsekana kusiyanisa kubva kune chaiyo tarenda rezwi Matimu anowanzo kuwana mhedzisiro iri nani kana achinge atsanangura emhando yepamusoro kumberi, chengetedza nzira yekukwira kwevanhu yemakesi emupendero, uye kuteedzera zvese zvakawanikwa zvechigadzirwa nemitengo yekukanganisa nekufamba kwenguva.
NaturalSpeech uye Latent Diffusion TTS mukuita
Maturusi ekugonesa anogadzirisazve izwi remunhu kubva kune akarekodhwa ekare kune avo vakarasikirwa nekutaura.
Zvishandiso zvekuwanikwa zvinogadzirazve izwi remunhu kubva kune akarekodhwa ekare kune avo vakarasikirwa nekutaura Matimu anowanzo kuwana mhedzisiro iri nani kana achinge atsanangura emhando yepamusoro kumberi, chengetedza nzira yekukwira kwevanhu yemakesi emupendero, uye kuteedzera zvese zvakawanikwa zvechigadzirwa uye mutengo wekukanganisa nekufamba kwenguva.
NaturalSpeech uye Latent Diffusion TTS mukuita
Zvemukati zvekugadzira masutu regai vapepeti vazvigadzirise vakazvimirira timbre uye prosody, inosimudzira NaturalSpeech 3's factorized hunhu.
Zvemukati zvekugadzira masutu regai vapepeti vagadzirise vakazvimirira timbre uye prosody, inosimudzira NaturalSpeech 3's factorized hunhu Matimu anowanzo kuwana mhedzisiro iri nani kana achinge atsanangura emhando yepamusoro kumberi, chengetedza nzira yekukwira kwevanhu yemakesi emupendero, uye kuteedzera zvese zvakawanikwa zvechigadzirwa uye kukanganisa mutengo nekufamba kwenguva.
Njodzi & Guardrails
Kusashandisa izwi zvisizvo uye njodzi dzekuedzesera dzinowedzera kana chibvumirano chisipo.
Kururama kunogona kudonha mumitauro, mataurirwo, kana nharaunda dzine ruzha.
Synthetic audio inogona kukanganisa kutaura kwechokwadi isina mavara akajeka.
Implementation Roadmap
Wana mvumo yakajeka yekutora inzwi, kugadzira, uye kushandisa zvakare.
Wana mvumo yakajeka yekutora inzwi, kugadzira, uye kushandisa zvakare. Bata nhanho yega yega segedhi rehumbowo: kana maitiro asina kusangana, imbomira kuburitsa, vhara gaka, uye wobva wawedzera kushandiswa.
Yedza mhando pavatauri vakasiyana uye mamiriro ekumashure.
Yedza mhando pavatauri vakasiyana uye mamiriro ekumashure. Bata nhanho yega yega segedhi rehumbowo: kana maitiro asina kusangana, imbomira kuburitsa, vhara gaka, uye wobva wawedzera kushandiswa.
Tsanangura apo munhu anofanira kuongorora kana kubvumidza zvabuda.
Tsanangura apo munhu anofanira kuongorora kana kubvumidza zvabuda. Bata nhanho yega yega segedhi rehumbowo: kana maitiro asina kusangana, imbomira kuburitsa, vhara gaka, uye wobva wawedzera kushandiswa.
Label synthetic odhiyo uye chengetedza marekodhi ekuzvidavirira.
Label synthetic odhiyo uye chengetedza marekodhi ekuzvidavirira. Bata nhanho yega yega segedhi rehumbowo: kana maitiro asina kusangana, imbomira kuburitsa, vhara gaka, uye wobva wawedzera kushandiswa.