VITS Synthese Kaddu bu Jeexal ba Jeexal
VITS xeetu bind-ci-kaddu la buy soppi bind ci forme onde audio bu ñor ci benn sistem buñ tàggat, te baña jëfandikoo pipeline bi am ñaari etap.
Résumé
By combining variational inference with adversarial training, it produces remarkably natural, expressive speech.
Plongeur bu xóot
VITS (Njàngat bu wuute ak jàngat buy xeex ngir Text-to-Speech), Kim, Kong, ak Son dugal ko ci 2021, dafa boole ñetti xalaat yu sistem yu yàgg yi tàqalewoon. Benn autoencoder variational conditionnel (VAE) dafay jàng misaalu kàddu bu nëbbu, normalise flow yi dañuy tax distribution bu nëbbu bi nekk lu yomb ngir jàpp ay detay akustik yu rafet, ba noppi benn discriminateur bu nuroo ak GAN dafay puus forme vague biñ defar mu jëm ci realisme. Lu gëna am solo mooy VITS dafay tàggat model akustik bi ak vocoder bi ñoom ñaar, duñu nekk ñaari etap, di dindi ñàkka méngoo giy yàq kalite bi suñu tàggatee module yi ñu wuute. Dafay dugal itam ab prediktor buy wax luy yàgg, suko defee ñu mëna wax benn frase ak ritm yu wuute, yuy sone saa yu nekk.
Gis-gis xarala
VITS dafay saafara jafe-jafe njubluwaay ak Seetug njubluwaay Monotonic (MAS), biy gis njubluwaay bi gëna baax ci diggante token bind ak kadre audio ci diiru tàggat yaram te amul njubluwaay yu biti. VAE posterior bi ñu ngi koy xayma ci audio bi, ci noonu lañu koy xayma ci bind bi, ñu koy soppi ci normalisee flow yi ngir méngoo ak moom. Ci inference, dangay jël misaal ci mbind mi njëkk ba noppi nga dekode ko ci forme onde, kon soxlawul mel-spectrogram bu wuute wala vocoder bu wuute.
njeextalu pexe
Dugg ak yegg
Dafay gëna yombal jëfandikoo gi jaaraleko ci transkripsioŋ, nettali ak interfaasu baat.
Njëgg ak budget
Ekipu mejaa yi mën nañu yónnee audio bu leer ci anam wu gëna gaaw te seen xaalis gëna néew.
Gaawaay ak yaatuwaay
Sistem yiy jàkkarloo ak kiliyaan bi mën nañu def waxtaan ci anam wu gëna yaatu.
Ëlëgu VITS Synthese Kaddu bu Jeexal ba Jeexal
VITS moo jur njaboot guy wuutu TTS open-source. VITS2 yombal na architecture bi ba noppi gëna yombal nature bi, ci noonu la YourTTS ak Coqui XTTS bi ñuy jëfandikoo bu bari yokk ci klonaasu baat bu amul benn shot ak làkk yu bari. Xaarandi liggéey bu wéy ci anam yu gëna woyof, ci jamono dëgg ci aparey yi, gëna mëna jëfandikoo làkk yu bari ngir làkk yu néew doole yi, ak gëna dëgër di saytu yëg-yëg ak ni ñuy waxee, ndax jëmmal ba ci njeexte gi dafay xëcc nit ñi, fondaasioŋ bu ñu xam bu baax ngir tabax ci kaw.
Doxal ci àdduna dëgg
Coqui TTS dafay yónnee ay model yu lalu ci VITS yu defarkat yi defar ngir klone baat nettalikat buñ tànn ngir téere audio.
Jàppalekatu baat yu ubbeeku yi ci aparey Raspberry Pi-class dañuy jëfandikoo model VITS yu kompact ngir génne kàddu bu amul jokkolante.
Aplikaasioŋ yiy jàng làkk yi dañuy defar misaali waxin bu baax ci jëfandikoo xeeti VITS yu bari làkk yu melni YourTTS.
Istuwaaru jeu indie yi dañuy boole ay lignéeru waxtaan NPC yu bari, lalu ci diir bi ñuy wax ci stochastic ngir ritm bu dul robot.
Risk yi ak balustrade yi
Jëfandikoo baat ci anam wu jaarul yoon ak niru ak nit dafay gëna yokk sudee nanguwul.
Jaar-jaar mën na wàññeeku ci aksan yi, dialect yi wala barab yu bari xumbaay.
Audio synthetik mën nañu ko jaawale ak wax ju dëggu sudee amul etiket bu leer.
Roadmap ngir samp gi
Wutal ndigal bu leer ngir jàpp baat bi, klone ko ak jëfandikoowaat ko.
Saytu kalite ci kàddukat yu bari ak anam yu bari ci ginaaw.
Mandargal kañ la nit wara xoolaat wala nangu ay génne.
Etiketu audio synthetik te nga denc dokimaa ci fimu bawoo ngir mëna lim.
Weyal di banneexu
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the VITS End-to-End Speech Synthesis quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gis bi ci topp
Kaddu yu am yëg-yëg
Laaj yi ñuy faral di laaj
What is VITS End-to-End Speech Synthesis?
VITS xeetu bind-ci-kaddu la buy soppi bind ci forme onde audio bu ñor ci benn sistem buñ tàggat, te baña jëfandikoo pipeline bi am ñaari etap. Suñu boole inference variational ak tàggat adversarial, dafay defar kàddu yu natureel yu yéeme te fësal seen xalaat.
Luy tekki akronim VITS?
VITS mooy Inference Variational ak jàng buy xeex Text-to-Speech, di wane ñatti mbir yu am solo yi ci nekk.
Lan mooy wuutale architecture bu mag bi am ci digganté VITS ak gasoduc TTS yi fi yàgg a nekk?
VITS dafay mujjee ci njeexte: dafay jàng forme text-to-onde ci benn model, moytu ñu baña méngoo ci model akustik bu wuute ak vocoder.
naka la VITS di jàngee njubte gi am ci digganté kadre bind ak audio?
VITS dafay jëfandikoo Seetug Liggéeyukaay bu Monotonik ngir gis lignéer bi gëna baax ci biir mbind ak kadre, te soxlawul lignéeru biti.
Lu tax VITS boole ci luy wax luy yàgg ci stochastic?
Duration stochastic bi dafay may ñu wax benn frase ci ritm yu wuute, moytu cadence bu plat, robot.
Ban composant mooy puus VITS ci audio bu am son bu dëggu?
VITS dafay jëfandikoo tàggat yaram (GAN), muy tàggat yaram buy xool ndax forme vague yi dañu dëggu, gëna baaxal ni ñuy gisee mbir yi.