I-VALL-E ne-Codec Language Models
I-VALL-E ifake uzimele kabusha umbhalo-kuya-inkulumo njengenkinga yemodeli yolimi phezu kwamathokheni ekhodekhi yomsindo, okuvumela ukwenziwa kwezwi kusuka kumasekhondi amathathu nje esampula.
Uhlolojikelele
It showed that the same next-token prediction powering text LLMs can generate remarkably natural, expressive speech.
I-Deep Dive
Imenyezelwe ngu-Microsoft ekuqaleni kuka-2023, i-VALL-E iphatha ukuhlanganiswa kwenkulumo njengokumodela kolimi. Esikhundleni sokubikezela i-spectrogram, ibikezela amathokheni e-acoustic acacile e-neural codec (EnCodec), ukuze isizukulwane sibe isibikezelo sethokheni elandelayo phezu kwesilulumagama somsindo. Njengoba kunikezwe ukurekhodwa kwamasekhondi angu-3 kwesipikha esingabonakali kanye nombhalo oqondiwe, i-VALL-E iyaqhubeka ngezwi laleso sipikha, igcina i-timbre kanye nendawo yokukhala. Yaqeqeshwa cishe emahoreni angu-60,000 okukhuluma, okungaphezulu kakhulu kwamasethi edatha e-TTS avamile, okunikeze i-zero-shot cloning eqinile. Ngenxa yokuthi amathokheni ekhodekhi anengqimba (nge-RVQ), i-VALL-E isebenzisa izigaba ezimbili: imodeli ezenzakalelayo ibikezela eyokuqala, ukusakaza kwethokheni emahhadla okufakwe kusimo sokwaziswa, futhi imodeli engeyona i-autoregressive igcwalisa amathokheni emininingwane asele. Le recipe ye-codec-LM igqugquzele abalandela njenge-VALL-E 2 namamodeli amaningi esisekelo senkulumo.
I-Technical Insight
Iqhinga ukuhunyushwa kwe-hybrid ngaphezulu kwamathokheni e-codec e-hierarchical. Isigaba se-autoregressive sibikezela amathokheni abaluleke kakhulu e-codebook eyodwa ngesikhathi, athwebula i-prosody nokuqukethwe. Ama-codebook asele, anezela imininingwane emihle ye-acoustic, abikezelwa ngokuhambisana imodeli engeyona ye-autoregressive ebekwe ekusakazeni kokuqala kanye nokwaziswa kwesipika. Lokhu kuhlukaniswa kugcina izinga eliphezulu ngenkathi kugwema izindleko zokukhiqiza wonke amathokheni ngokulandelana, futhi ukusebenzisa i-codec kusho inkulumo nombhalo kungenziwa imodeli ngemishini efanayo ye-transformer.
I-Strategic Impact
Finyelela futhi ufinyelele
Ithuthukisa ukufinyeleleka ngokuloba, ukulandisa, nezixhumi ezibonakalayo zezwi.
Izindleko kanye nesabelomali
Amaqembu emidiya angathumela umsindo opholishiwe ngokushesha ngamabhajethi amancane.
Isivinini nesikali
Amasistimu abhekene nekhasimende angacubungula ukusebenzelana okukhulunyiwe ngesilinganiso esikhulu.
Ikusasa le-VALL-E kanye namamodeli olimi e-Codec
Amamodeli olimi e-codec ahlanganisa inkulumo namamodeli amakhulu olimi, akhomba kumasistimu ahlangene alalelayo, acabange, futhi akhulume ngemodeli eyodwa. Lindela ukuzinza okungcono nama-artifact ambalwa, ukukhiqizwa kokusakaza-bukhoma kwesikhathi sangempela, nokulawula okuqinile phezu kwemizwelo nesitayela. I-cloning efanayo enamandla eyenza i-VALL-E isebenziseke ekufinyeleleni nasekubhalweni iphinde iphakamise ukukhathazeka okujulile nemvume, ngakho ukubeka i-watermarking, izivikelo zokuqinisekisa izwi, kanye nemithetho yokuqapha inqubomgomo kuba yingxenye eyinhloko yokuthi lezi zinhlelo zisetshenziswa kanjani.
Ukuqaliswa Komhlaba Wangempela
Ukuhlanganisa izwi kusuka kumasekhondi ambalwa omsindo kubasizi abaqondene nawe noma amathuluzi okufinyelela abuyisela izwi elilahlekile
Ukwenza kube okwasendaweni nokubizela ividiyo kwezinye izilimi kuyilapho ugcina i-timbre yesikhulumi sokuqala
Ikhiqiza ukulandisa okuvezayo, okufaniswa nomxholo okugcina indawo ye-acoustic yokurekhodwa
Isebenza njengomgogodla wenkulumo kuma-multimodal abasizi abaqondayo futhi bakhiqize umsindo okhulunyiwe
Izingozi & Guardrails
Ukusetshenziswa kabi kwezwi kanye nezingozi zokuzenza ongeyena ziyanda uma imvume ingekho.
Ukunemba kungase kwehle kuzo zonke izinhlobo zokuphimisela, izilimi zesigodi, noma izindawo ezinomsindo.
Umsindo wokwenziwa ungenziwa iphutha njengenkulumo eyiqiniso ngaphandle kokulebula okucacile.
Ukuqalisa Umhlahlandlela
Thola imvume esobala yokuthwebula izwi, ukuhlanganisa, nokusebenzisa kabusha.
Ikhwalithi yokuhlola kuzo zonke izipikha nezimo zangemuva.
Chaza ukuthi kunini lapho umuntu kufanele abuyekeze noma agunyaze okuphumayo.
Lebula umsindo wokwenziwa futhi ugcine amarekhodi atholakalayo ukuze aziphendulele.
Qhubeka Uhlole
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the VALL-E and Codec Language Models quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Umhlahlandlela olandelayo
Amakhono Athuthukayo Amamodeli Olimi Olukhulu
Imibuzo evame ukubuzwa
What is VALL-E and Codec Language Models?
I-VALL-E ifake uzimele kabusha umbhalo-kuya-inkulumo njengenkinga yemodeli yolimi phezu kwamathokheni ekhodekhi yomsindo, okuvumela ukwenziwa kwezwi kusuka kumasekhondi amathathu nje esampula. Kubonise ukuthi umbhalo ofanayo wokubikezela ithokheni elandelayo wokunika amandla umbhalo we-LLM ungakhiqiza inkulumo engokwemvelo ngokuphawulekayo, evezayo.
Imuphi umbono obalulekile ohlukanisa i-VALL-E ezinhlelweni zangaphambilini zombhalo uye-enkulumweni?
I-VALL-E yenza kabusha i-TTS njengesibikezelo sethokheni elandelayo phezu kwamathokheni ekhodekhi yomsindo ahlukahlukene, iphatha ukukhiqizwa kwenkulumo njengemodeli yolimi.
I-VALL-E idinga umsindo ongakanani oyisithenjwa ukuze uhlanganise izwi lesipika esisha?
I-VALL-E ingakwazi ukwenza i-zero-shot voice cloning kusukela kusampula cishe yamasekhondi angu-3 esipikha esingabonakali.
Iyiphi i-neural codec esetshenziswa i-VALL-E ukuze ikhiqize amathokheni ayo omsindo?
I-VALL-E yakhela phezu kwe-Meta's EnCodec, ibikezela amathokheni ayo ahlukene e-acoustic ukuze ahlanganise inkulumo.
Kungani i-VALL-E isebenzisa kokubili i-autoregressive kanye ne-non-autoregressive stage?
Amathokheni e-Codec anomlando nge-RVQ; I-VALL-E ibikezela ukusakaza kokuqala okumaholoholo ngokuzenzakalelayo kanye namathokheni emininingwane asele ngokuhambisana ukuze asebenze kahle.
Ilinganiselwa ukuthi ingakanani idatha yenkulumo eyaqeqeshwa i-VALL-E, evumela i-zero-shot cloning eqinile?
I-VALL-E yaqeqeshwa cishe emahoreni angu-60,000 okukhuluma, okungaphezulu kakhulu kwamasethi edatha e-TTS avamile, okukhuphule ukwenziwa kwayo okungasho lutho.