StyleTTS 2 Fassarar Salon
StyleTTS 2 samfuri ne na rubutu-zuwa-magana wanda ke kula da 'salo' murya - prosody, motsin rai, da timbre mai magana - azaman mai canzawa bazuwar samfuri tare da ƙirar yaduwa, sannan ya haɗa sauti tare da horo na gaba akan babban samfurin harshen magana.
Dubawa
It matters because it reached human-level naturalness on single-speaker benchmarks without needing a reference clip at inference time.
Zurfafa nutsewa
StyleTTS 2, wanda masu bincike a Jami'ar Columbia suka fitar a cikin 2023, yana haifar da magana ta hanyar fara samar da samfurin 'style vector' ta hanyar amfani da tsarin watsawa wanda aka tsara kawai akan rubutun shigar da shi, sannan yanke wannan salon tare da wayoyin hannu a cikin tsarin motsi. Salon salon salon yana sarrafa duk abin da ba a rubuta a cikin rubutu ba: ƙimar magana, kwatancen innation, tsayawa, da canza launi. Mahimmanci, yana ƙara horarwar gaba tare da manyan samfuran harshen magana da aka riga aka horar (WavLM) azaman masu wariya, suna tura fitarwa zuwa sautin sauti na gaske na ɗan adam. A kan ma'auni na LJSpeech ya zarce rikodin ɗan adam a cikin ƙimar masu sauraro, kuma a kan LibriTTS masu magana da yawa ya saita shi daidai da gaskiyar ƙasa - wani ci gaba na ƙimar TTS na ƙarshen-zuwa-ƙarshen.
Fahimtar Fasaha
Babban abin zamba shine yaduwar salon: maimakon tsinkaya tsayayyen tsari guda ɗaya, salon ƙirar StyleTTS 2 azaman rarraba yiwuwar rarrabawa da samfurori daga gare ta ta hanyar ƙirar yaduwa da ke gudana a cikin ƙaramin sarari latent, don haka ana iya faɗi jumla ɗaya ta hanyoyi da yawa. Ƙarshe-zuwa-ƙarshe, mai tsinkayar tsawon lokaci, mai rikodin salo, mai ƙididdigewa, da WavLM na tushen adawar wariya an horar da su tare, barin gradients suna gudana daga ingancin igiyar igiyar ruwa ta cikin bututun gabaɗaya.
Dabarun Tasiri
Shiga ku isa
Yana inganta samun dama ta hanyar rubutu, ba da labari, da mu'amalar murya.
Kudin da kasafin kuɗi
Ƙungiyoyin kafofin watsa labaru na iya jigilar sauti mai gogewa cikin sauri tare da ƙaramin kasafin kuɗi.
Gudu da sikelin
Tsarin fuskantar abokin ciniki na iya aiwatar da hulɗar magana a mafi girman ma'auni.
Makomar StyleTTS 2 Watsawa Salo
Yi tsammanin yaduwar salo don haɗawa tare da rufewar murya mai harbin sifili don haka 'yan daƙiƙa kaɗan na tunani mai jiwuwa ya jagoranci salon da aka ƙirƙira, kuma tare da hannaye masu sarrafawa waɗanda ke barin masu ƙirƙira su buga motsin rai, girmamawa, ko taki a sarari. Ƙaƙƙarfan juzu'i masu sauƙi suna nufin yanke samfurin watsa matakai masu yawa don amfani da ainihin lokacin akan na'urori. Yayin da waɗannan samfuran suka kai ingancin watsa shirye-shirye, alamar ruwa da tabbatar da yarda za su zama daidaitattun don magance matsalolin muryoyin murya da zurfafa yin amfani da su.
Aiwatar da Gaskiyar Duniya
Ƙirƙirar labari na littafin mai jiwuwa inda mai magana ɗaya ya bambanta a zahiri a cikin surori maimakon sautin monotone
Samar da muryoyin bayyananniyar halaye don wasannin indie da rayarwa ba tare da hayar ƴan wasan murya da yawa ba
Ƙaddamar da masu karanta allo masu isa ga ɗan adam isa don sauraron dogon tsari
Ƙirƙirar sautin e-leon na cikin gida tare da ƙarfafawa na halitta da taki daga rubutun rubutun bayyananne
Hatsari & Tsare-tsare
Rashin amfani da murya da haɗarin kwaikwaya yana ƙaruwa lokacin da aka rasa izini.
Daidaituwa na iya faɗuwa cikin lafuzza, yaruka, ko mahalli masu hayaniya.
Ana iya kuskuren sauti na roba don ingantacciyar magana ba tare da bayyananniyar lakabi ba.
Taswirar Hanya
Sami tabbataccen izini don ɗaukar murya, cloning, da sake amfani.
Gwajin ingantattun masu magana daban-daban da yanayin baya.
Ƙayyade lokacin da dole ne ɗan adam ya duba ko ya amince da abubuwan da aka fitar.
Yi lakabin sauti na roba da kuma adana bayanan da aka tabbatar don yin lissafi.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the StyleTTS 2 Style Diffusion quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Riffusion Spectrogram Diffusion
Tambayoyin da ake yawan yi
What is StyleTTS 2 Style Diffusion?
StyleTTS 2 samfuri ne na rubutu-zuwa-magana wanda ke kula da 'salo' murya - prosody, motsin rai, da timbre mai magana - azaman mai canzawa bazuwar samfuri tare da ƙirar yaduwa, sannan ya haɗa sauti tare da horo na gaba akan babban samfurin harshen magana. Yana da mahimmanci saboda ya kai ga dabi'ar mutum-mutumi akan ma'auni mai magana guda ɗaya ba tare da buƙatar faifan tunani a lokacin ƙididdigewa ba.
Menene samfurin StyleTTS 2 ta amfani da tsarin watsawa?
StyleTTS 2 yana ba da samfura mai ƙarancin ƙima mai ƙima tare da ƙirar yaduwa, ɗaukar hoto, motsin rai, da halayen lasifikar da rubutun ba a kayyade ba.
Wane samfurin magana da aka riga aka horar aka yi amfani da shi azaman mai nuna wariya a cikin StyleTTS 2?
StyleTTS 2 yana amfani da manyan nau'ikan yaren magana kamar WavLM azaman masu nuna wariya a cikin horon adawa don tura abubuwan da aka fitar zuwa sautin sauti na ɗan adam.
Me yasa StyleTTS 2 zai iya faɗi jumla ɗaya ta hanyoyi da yawa na halitta?
Saboda ana ɗaukar salo azaman mai canzawa bazuwar kuma ana yin samfuri ta hanyar watsawa, kowane tsara na iya samar da wani nau'i na daban amma na halitta don rubutu iri ɗaya.
A wanne ma'auni mai magana guda ɗaya aka bayar da rahoton StyleTTS 2 ya zarce rikodin ɗan adam a ƙimar masu sauraro?
A kan ma'auni na LJSpeech, StyleTTS 2 sun sami ƙimar dabi'ar mai sauraro wanda ya wuce rikodin gaskiyar ƙasa na ɗan adam.
Menene horo 'karshe-zuwa-ƙarshe' ke ba wa StyleTTS 2?
Horon haɗin gwiwa daga ƙarshe zuwa ƙarshe yana ba da damar hasara akan ingancin sauti na ƙarshe sabunta mai hasashen tsawon lokaci, mai rikodin salo, da mai ƙididdigewa tare maimakon a keɓance matakai.