I-NaturalSpeech kanye ne-Latent Diffusion TTS
I-NaturalSpeech iwumugqa Microsoft wocwaningo lwe-TTS oluhloselwe ikhwalithi yenkulumo yezinga lomuntu, enezinguqulo zakamuva ezisebenzisa ukusabalalisa okucashile ukuze kukhiqizwe amazwi acebile, emvelo.
Uhlolojikelele
It shows how diffusion models, famous for images, can produce expressive, controllable audio.
I-Deep Dive
I-NaturalSpeech yoqobo (2022) bekuyisistimu yokuqala ebikwe ukuthi ifinyelele ikhwalithi yezinga lomuntu kubhentshimakhi ye-LJSpeech, ehlulelwa abalaleli abangakwazi ukukusho ngokuthembekile ekurekhodweni kwangempela. Isebenzise i-autoencoder eguquguqukayo enezangaphambili ezifaniswe ngokucophelela ukuze kuvalwe igebe phakathi kokuqeqeshwa nokusho okuthile. I-NaturalSpeech 2 yabe isithatha indlela yokusabalalisa okucashile: inkulumo ifakwe ikhodi ye-neural audio codec ibe amavekhtha afihlekile aqhubekayo, futhi imodeli yokusabalalisa ifunda ukwenza lezo zinto ezifihlekile ngombhalo, okuvumela ukuhlanganisa kwezwi okungasho lutho okuqinile kusuka ekwazisweni okufushane. I-NaturalSpeech 3 yethule ukusabalalisa okuhlukile, okuhlukanisa inkulumo ibe kuzibaluli ezihlukene njengokuqukethwe, i-prosody, i-timbre, nemininingwane ye-acoustic, ukuze ngayinye imodelwe futhi ilawulwe ngokuzimela ngokwethembeka okuphezulu nokuguquguquka.
I-Technical Insight
Ukusabalalisa okucashile kusebenza ngokwengeza umsindo ekumeleleni okucashile okuhlangene kwenkulumo nokuqeqesha inethiwekhi ukuze ihlehlise lowo msindo isinyathelo ngesinyathelo. Kunokuba ikhiphe umsindo wamagagasi angahluziwe noma ama-spectrogram agcwele, i-NaturalSpeech 2 idenoise ama-codec latent, ane-dimensional ephansi futhi kulula ukuyimodelisa. Isimo sombhalo kanye nezwi lereferensi liqondisa ukuhlehla kokusabalalisa, ngakho okokugcina okuyisampula okuyisampula kunquma kube inkulumo efana nokuqukethwe okuceliwe kanye nobunikazi besipikha.
I-Strategic Impact
Finyelela futhi ufinyelele
Ithuthukisa ukufinyeleleka ngokuloba, ukulandisa, nezixhumi ezibonakalayo zezwi.
Izindleko kanye nesabelomali
Amaqembu emidiya angathumela umsindo opholishiwe ngokushesha ngamabhajethi amancane.
Isivinini nesikali
Amasistimu abhekene nekhasimende angacubungula ukusebenzelana okukhulunyiwe ngesilinganiso esikhulu.
Ikusasa le-NaturalSpeech kanye ne-Latent Diffusion TTS
I-TTS esekelwe ekusakazweni kanye ne-factorized ikhomba kumazwi angewona nje awemvelo kodwa alawuleka kahle, avumela abasebenzisi ukuthi balungise i-timbre, imizwa, ne-prosody njengokudayela okuzimele. Lindela amasampula asheshayo nge-distillation kanye nokusabalalisa kwezinyathelo ezimbalwa, i-zero-shot cloning eqinile kusukela kumasekhondi omsindo, nokuhlanganiswa okuqinile okunamamodeli amakhulu olimi ukuze kuthunyelwe okuqaphela umongo. Le ntuthuko iphinde iqinise isidingo se-watermarking kanye nezivikelo zemvume, njengoba ukwenziwa kwe-high-fidelity cloning kuphakamisa izingozi ezicacile zokusebenzisa kabi.
Ukuqaliswa Komhlaba Wangempela
Izitudiyo ezikopishwayo zifanisa izwi lomlingisi ukusuka kusampula emfushane ukuze enze amafilimu abe okwasendaweni, kusetshenziswa i-NaturalSpeech 2-style zero-shot cloning.
Izinkundla ze-Audiobook zikhiqiza ukulandisa okusezingeni lomuntu okuzabalaza abalaleli ukukuhlukanisa kuthalente lezwi langempela.
Amathuluzi okufinyelela akha kabusha izwi lomuntu siqu kusuka kokurekhodiwe okudala kulabo abalahlekelwe ukukhuluma.
Amasudi okudala okuqukethwe avumela abahleli ukuthi balungise ngokuzimela i-timbre ne-prosody, basebenzise izici ezifactorywe ze-NaturalSpeech 3.
Izingozi & Guardrails
Ukusetshenziswa kabi kwezwi kanye nezingozi zokuzenza ongeyena ziyanda uma imvume ingekho.
Ukunemba kungase kwehle kuzo zonke izinhlobo zokuphimisela, izilimi zesigodi, noma izindawo ezinomsindo.
Umsindo wokwenziwa ungenziwa iphutha njengenkulumo eyiqiniso ngaphandle kokulebula okucacile.
Ukuqalisa Umhlahlandlela
Thola imvume esobala yokuthwebula izwi, ukuhlanganisa, nokusebenzisa kabusha.
Ikhwalithi yokuhlola kuzo zonke izipikha nezimo zangemuva.
Chaza ukuthi kunini lapho umuntu kufanele abuyekeze noma agunyaze okuphumayo.
Lebula umsindo wokwenziwa futhi ugcine amarekhodi atholakalayo ukuze aziphendulele.
Qhubeka Uhlole
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the NaturalSpeech and Latent Diffusion TTS quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Umhlahlandlela olandelayo
Ukusabalalisa Okufihlekile Komsindo Okuzinzile
Imibuzo evame ukubuzwa
What is NaturalSpeech and Latent Diffusion TTS?
I-NaturalSpeech iwumugqa Microsoft wocwaningo lwe-TTS oluhloselwe ikhwalithi yenkulumo yezinga lomuntu, enezinguqulo zakamuva ezisebenzisa ukusabalalisa okucashile ukuze kukhiqizwe amazwi acebile, emvelo. Ibonisa ukuthi amamodeli okusabalalisa, adume ngezithombe, angakhiqiza kanjani umsindo ozwakalayo, olawulekayo.
Iyiphi ingqopha-mlando i-NaturalSpeech yasekuqaleni (2022) okubikwe ukuthi iyifezile?
I-NaturalSpeech ibikwe njengesistimu yokuqala yokufinyelela ikhwalithi yezinga lomuntu ku-LJSpeech, enabalaleli abangakwazi ukuyehlukanisa ngokuthembekile enkulumeni yangempela.
Ikhiqiza ini ngempela imodeli yokusabalalisa okucashile ye-NaturalSpeech 2?
I-NaturalSpeech 2 isakazeka phezu kwama-codec latent, ahlangene futhi kulula ukuwenza imodeli kunamagagasi aluhlaza, bese iwakhipha amakhodi ukuze abe umsindo.
Imodeli yokusabalalisa ikhiqiza kanjani idatha ezingeni eliphezulu?
I-Diffusion ingeza umsindo ngesikhathi sokuqeqeshwa futhi ifunde ukuhlehlisa, ikhiqize amasampuli ngokukhipha umsindo oqhubekayo emsindweni ongahleliwe.
Yiliphi ikhono elibalulekile ukusabalalisa okufihlekile okulinikeza i-NaturalSpeech 2?
Ngokumisa ukwaziswa kwezwi okufushane, i-NaturalSpeech 2 ingakwazi ukuhlanganisa izwi lesipikha elingakaze liqeqeshwe ngalo ngokusobala.
Uyini umbono omaphakathi 'we-NaturalSpeech 3's 'factorized diffusion'?
Ukusabalaliswa okuhleliwe kuhlukanisa okuqukethwe, i-prosody, i-timbre, nemininingwane ye-acoustic ukuze isibaluli ngasinye simodelwe futhi silawulwe ngokwehlukana.