Imisusire ya TTS 2 Imiterere itandukanye
StyleTTS 2 ni inyandiko-y-imvugo yerekana uburyo 'ijwi' - amajwi, amarangamutima, hamwe na disikuru timbre - nk'impinduka idasanzwe yatanzwe hamwe na moderi yo gukwirakwiza, hanyuma igahuza amajwi hamwe n'amahugurwa yo guhangana n’ururimi runini rwo kuvuga.
Incamake
It matters because it reached human-level naturalness on single-speaker benchmarks without needing a reference clip at inference time.
Kwibira cyane
StyleTTS 2, yasohowe mu 2023 n’abashakashatsi bo muri kaminuza ya Columbiya, itanga ijambo ubanza gutoranya 'style vector' yihishe ukoresheje uburyo bwo gukwirakwiza ibintu byateganijwe gusa ku nyandiko yinjiza, hanyuma ugashiraho ubwo buryo wongeyeho fonema mu muhengeri. Imiterere ya vector igenzura ibintu byose bitanditswe mumyandiko: igipimo cyo kuvuga, intonasiyo, intangiriro, hamwe namabara. Icy'ingenzi, yongeraho imyitozo yo guhangana hamwe n’ururimi runini rwateguwe mbere yo kuvuga imvugo (WavLM) nk'ivangura, bigatuma ibisohoka bigana ku majwi yumvikana n'abantu. Ku gipimo cya LJSpeech cyarenze ibyanditswe n'abantu mu rutonde rw'abumva, naho ku bavuga indimi nyinshi LibriTTS yashyizeho bihuza n'ukuri gushingiye - intambwe yerekana ubwiza bwa TTS iherezo-iherezo.
Ubushishozi
Amayeri yingenzi nuburyo bwo gukwirakwiza: aho guhanura inzira imwe ihamye, StyleTTS 2 yerekana imiterere nkibishoboka byo gukwirakwiza hamwe nicyitegererezo kivuye muri yo ikwirakwizwa na moderi ikwirakwizwa mu mwanya muto wihishe, bityo interuro imwe irashobora kuvugwa muburyo busanzwe. Kurangiza-kurangira, guhanura igihe, imiterere ya kodegisi, decoder, hamwe na WavLM ishingiye ku ivangura rirwanya imyitozo ihuriweho hamwe, ikareka gradients zikagenda ziva kumiterere yumurongo ugasubira mumiyoboro yose.
Ingaruka z'Ingamba
Kugera no kugera
Itezimbere kugerwaho binyuze mu kwandukura, kuvuga, no guhuza amajwi.
Igiciro na bije
Amatsinda yibitangazamakuru arashobora kohereza amajwi yihuse hamwe na bije nto.
Umuvuduko n'igipimo
Sisitemu ireba abakiriya irashobora gutunganya imikoranire ivugwa murwego runini.
Kazoza ka StyleTTS 2 Imiterere itandukanye
Tegereza uburyo bwo gukwirakwiza kugirango uhuze hamwe na zeru-shusho yijwi rya clone kuburyo amasegonda make yerekana amajwi ayobora uburyo bwatanzwe, hamwe nuburyo bugenzurwa butuma abarema bahamagara amarangamutima, bashimangira, cyangwa umuvuduko ugaragara. Impinduramatwara yoroheje igamije kugabanya intambwe-ntambwe yo gukwirakwiza icyitegererezo cyo gukoresha igihe-nyacyo ku bikoresho. Mugihe izo moderi zigera kumurongo wogutangaza amakuru, kumenyekanisha amazi no kugenzura ibyemezo bizaba bisanzwe kugirango bikemure ibibazo byangiza amajwi hamwe nibibazo byimbitse.
Gushyira mu bikorwa Isi
Kubyara amajwi ya audiobook aho umuvugizi umwe asanzwe ahinduranya prosode mubice aho kumvikana monotone
Gukora amajwi yerekana amajwi yimikino ya indie na animasiyo udakoresheje amajwi menshi
Guha imbaraga abasomyi ba ecran basoma byumvikana nkabantu bihagije kugirango bumve igihe kirekire
Gukora amajwi ya e-yiga amajwi yibanze hamwe nibisanzwe hamwe no kwihuta uhereye kumyandiko isanzwe
Ingaruka & Kurinda
Gukoresha nabi amajwi no kwigira ibyago byiyongera mugihe uruhushya rubuze.
Ukuri kurashobora kugabanuka hejuru yimvugo, imvugo, cyangwa urusaku rwibidukikije.
Amajwi yubukorikori arashobora kwibeshya kumvugo yukuri nta kirango gisobanutse.
Igishushanyo mbonera
Shaka uruhushya rusobanutse rwo gufata amajwi, gukoroniza, no gukoresha.
Ikizamini cyiza mubiganiro bitandukanye hamwe nuburyo bwimbere.
Sobanura igihe umuntu agomba gusuzuma cyangwa kwemeza ibisubizo.
Andika amajwi yubukorikori kandi ugumane inyandiko zerekana kubazwa.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the StyleTTS 2 Style Diffusion quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Riffusion Spectrogram Diffusion
Ibibazo bikunze kubazwa
What is StyleTTS 2 Style Diffusion?
StyleTTS 2 ni inyandiko-y-imvugo yerekana uburyo 'ijwi' - amajwi, amarangamutima, hamwe na disikuru timbre - nk'impinduka idasanzwe yatanzwe hamwe na moderi yo gukwirakwiza, hanyuma igahuza amajwi hamwe n'amahugurwa yo guhangana n’ururimi runini rwo kuvuga. Nibyingenzi kuko byageze kurwego rwumuntu kurwego rwumuvugizi umwe udakeneye clip yerekana mugihe cyagenwe.
Ni ubuhe buryo bwa StyleTTS 2 ukoresheje inzira yo gukwirakwiza?
StyleTTS 2 ntangarugero yuburyo buciriritse bwa vector hamwe na moderi yo gukwirakwiza, ifata prosody, amarangamutima, nibiranga imvugo bitagaragajwe ninyandiko.
Ni ubuhe bwoko bw'imvugo yabanje gutozwa bukoreshwa nk'ivangura rirwanya muri StyleTTS 2?
StyleTTS 2 ikoresha imvugo nini yerekana imvugo nka WavLM nk'ivangura mu mahugurwa yo guhangana kugirango itere ibisubizo amajwi yumvikana n'abantu.
Kuki StyleTTS 2 ishobora kuvuga interuro imwe muburyo bwinshi?
Kuberako imiterere ifatwa nkibihinduka bidasanzwe kandi bigakorwa hifashishijwe ikwirakwizwa, buri gisekuru gishobora kubyara ibintu bitandukanye ariko bisanzwe mubyanditswe bisa.
Ni ikihe gipimo cy’umuvugizi umwe StyleTTS 2 bivugwa ko yarenze amajwi y'abantu mu rutonde rw'abumva?
Ku gipimo cya LJSpeech, StyleTTS 2 yageze ku bipimo byabateze amatwi birenze amajwi-yukuri yabantu.
Niki imyitozo 'iherezo-iherezo' itanga StyleTTS 2?
Amahugurwa ahuriweho nurangiza arareka igihombo kumajwi yanyuma yanyuma kuvugurura igihe cyateganijwe, imiterere ya kodegisi, hamwe na decoder hamwe aho kuba mubyiciro byihariye.