VALL-E da Samfuran Harshen Codec
VALL-E ya sake tsara rubutu-zuwa-magana azaman matsalar ƙirar harshe akan alamomin codec mai jiwuwa, yana ba da damar ƙarar murya daga daƙiƙa uku kacal na samfurin.
Dubawa
It showed that the same next-token prediction powering text LLMs can generate remarkably natural, expressive speech.
Zurfafa nutsewa
Microsoft ne ya sanar da shi a farkon 2023, VALL-E tana ɗaukar haɗin magana kamar ƙirar harshe. Maimakon yin tsinkayar siffa, yana yin tsinkaya madaidaitan alamun sauti na codec na jijiya (EnCodec), don haka tsara ya zama tsinkayar alama ta gaba akan ƙamus na sauti. Idan aka ba da rikodin na daƙiƙa 3 na lasifikar da ba a gani da kuma rubutun manufa, VALL-E yana ci gaba a cikin muryar mai magana, yana adana katako har ma da yanayin sauti. An horar da shi a kan kusan sa'o'i 60,000 na magana, wanda ya fi yawan bayanan TTS na yau da kullun, wanda ya ba shi cloning mai ƙarfi. Saboda alamun codec suna layi (ta hanyar RVQ), VALL-E yana amfani da matakai guda biyu: samfurin autoregressive yana annabta na farko, rafi mai ƙaƙƙarfan rafi mai sharadi akan saƙon, kuma ƙirar mara-autoregressive ta cika sauran alamun dalla-dalla. Wannan girke-girke na codec-LM ya yi wahayi zuwa ga magada kamar VALL-E 2 da yawancin ƙirar tushe na magana.
Fahimtar Fasaha
Dabarar ita ce zazzage nau'ikan nau'ikan alamomin codec masu matsayi. Matakin autoregressive yana annabta mafi mahimmancin alamomin littafin farko na farko ɗaya bayan ɗaya, yana ɗaukar fa'ida da abun ciki. Sauran littattafan kade-kade, waɗanda ke ƙara dalla-dalla dalla-dalla, ana annabta su a layi daya ta hanyar ƙirar da ba ta da iko wacce aka sharadi akan rafi na farko da faɗakarwar lasifikar. Wannan tsaga yana kiyaye inganci yayin da yake guje wa farashin samar da kowane alamar bi-da-bi, kuma yin amfani da codec yana nufin za a iya ƙirƙira magana da rubutu tare da injinan taswira iri ɗaya.
Dabarun Tasiri
Shiga ku isa
Yana inganta samun dama ta hanyar rubutu, ba da labari, da mu'amalar murya.
Kudin da kasafin kuɗi
Ƙungiyoyin kafofin watsa labaru na iya jigilar sauti mai gogewa cikin sauri tare da ƙaramin kasafin kuɗi.
Gudu da sikelin
Tsarin fuskantar abokin ciniki na iya aiwatar da hulɗar magana a mafi girman ma'auni.
Makomar VALL-E da Samfuran Harshen Codec
Samfuran yaren Codec suna haɗa magana tare da manyan nau'ikan harshe, suna nuni zuwa ga tsarin haɗin kai waɗanda ke saurara, tunani, da magana cikin ƙira ɗaya. Yi tsammanin ingantacciyar kwanciyar hankali da ƴan kayan tarihi, tsararrun raye-raye na ainihin lokaci, da ƙarin iko akan motsin rai da salo. Iri ɗaya mai ƙarfi cloning wanda ke sa VALL-E mai amfani don samun dama da yin faifai shima yana haifar da zurfin karya da damuwa damuwa, don haka alamar ruwa, tabbatar da muryar murya, da tsare-tsaren tsare-tsare suna zama babban ɓangare na yadda ake tura waɗannan tsarin.
Aiwatar da Gaskiyar Duniya
Rufe murya daga ƴan daƙiƙa na audio don keɓaɓɓen mataimaka ko kayan aikin samun dama waɗanda ke dawo da muryar da ta ɓace
Ganewa da buga bidiyo zuwa wasu yarukan yayin da ake ajiye timbre na ainihin mai magana
Samar da bayyananniyar magana, wacce ta dace da mahallin da ke kiyaye yanayin sautin rikodi
Yin hidima azaman ƙashin bayan magana a cikin mataimakan multimodal waɗanda duka fahimta da samar da sautin magana
Hatsari & Tsare-tsare
Rashin amfani da murya da haɗarin kwaikwaya yana ƙaruwa lokacin da aka rasa izini.
Daidaituwa na iya faɗuwa cikin lafuzza, yaruka, ko mahalli masu hayaniya.
Ana iya kuskuren sauti na roba don ingantacciyar magana ba tare da bayyananniyar lakabi ba.
Taswirar Hanya
Sami tabbataccen izini don ɗaukar murya, cloning, da sake amfani.
Gwajin ingantattun masu magana daban-daban da yanayin baya.
Ƙayyade lokacin da dole ne ɗan adam ya duba ko ya amince da abubuwan da aka fitar.
Yi lakabin sauti na roba da kuma adana bayanan da aka tabbatar don yin lissafi.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the VALL-E and Codec Language Models quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Ƙarfin Gaggawa na Manyan Samfuran Harshe
Tambayoyin da ake yawan yi
What is VALL-E and Codec Language Models?
VALL-E ya sake tsara rubutu-zuwa-magana azaman matsalar ƙirar harshe akan alamomin codec mai jiwuwa, yana ba da damar ƙarar murya daga daƙiƙa uku kacal na samfurin. Ya nuna cewa rubutu mai ƙarfi na gaba iri ɗaya na LLM na iya haifar da yanayi mai ban mamaki, magana mai bayyanawa.
Menene ainihin ra'ayi ya bambanta VALL-E daga tsarin rubutu-zuwa-magana na farko?
VALL-E yana sake tsara TTS azaman tsinkayar alama ta gaba akan fitattun lambobin codec na odiyo, yana ɗaukar tsarar magana kamar ƙirar harshe.
Nawa nawa sautin magana VALL-E yake buƙata don rufe sabon muryar mai magana?
VALL-E na iya yin cloning na murya mai sifili daga kusan samfurin daƙiƙa 3 na lasifikar da ba a gani.
Wane neural codec ne VALL-E ke amfani da shi don samar da alamun sautinsa?
VALL-E yana ginawa akan Meta's EnCodec, yana tsinkayar alamun sautin sautin sa don haɗa magana.
Me yasa VALL-E ke amfani da duka autoregressive da mara-autoregressive mataki?
Alamu na Codec suna da matsayi ta hanyar RVQ; VALL-E yana annabta rafi mai ƙaƙƙarfan rafi na farko kai tsaye da sauran alamun dalla-dalla a layi daya don inganci.
Kusan nawa ne aka horar da bayanan magana akan VALL-E, yana ba da damar cloning mai ƙarfi mai ƙarfi?
An horar da VALL-E akan kusan sa'o'i 60,000 na magana, fiye da na yau da kullun na bayanan TTS, wanda ya haɓaka haɓakar sifilin sa.