Uhlolojikelele
I-VALL-E ifake uzimele kabusha umbhalo-kuya-inkulumo njengenkinga yemodeli yolimi phezu kwamathokheni ekhodekhi yomsindo, okuvumela ukwenziwa kwezwi kusuka kumasekhondi amathathu nje esampula. Kubonise ukuthi umbhalo ofanayo wokubikezela ithokheni elandelayo wokunika amandla umbhalo we-LLM ungakhiqiza inkulumo engokwemvelo ngokuphawulekayo, evezayo.
I-VALL-E kanye ne-Codec Language Models ihlezi ku-audio-AI workflows eguqula inkulumo, umculo, nomsindo wokuxhumana, ukufinyeleleka, nokukhiqizwa kwemidiya.
I-Deep Dive
Imenyezelwe ngu-Microsoft ekuqaleni kuka-2023, i-VALL-E iphatha ukuhlanganiswa kwenkulumo njengokumodela kolimi. Esikhundleni sokubikezela i-spectrogram, ibikezela amathokheni e-acoustic acacile e-neural codec (EnCodec), ukuze isizukulwane sibe isibikezelo sethokheni elandelayo phezu kwesilulumagama somsindo. Njengoba kunikezwe ukurekhodwa kwamasekhondi angu-3 kwesipikha esingabonakali kanye nombhalo oqondiwe, i-VALL-E iyaqhubeka ngezwi laleso sipikha, igcina i-timbre kanye nendawo yokukhala. Yaqeqeshwa cishe emahoreni angu-60,000 okukhuluma, okungaphezulu kakhulu kwamasethi edatha e-TTS avamile, okunikeze i-zero-shot cloning eqinile. Ngenxa yokuthi amathokheni ekhodekhi anengqimba (nge-RVQ), i-VALL-E isebenzisa izigaba ezimbili: imodeli ezenzakalelayo ibikezela eyokuqala, ukusakaza kwethokheni emahhadla okufakwe kusimo sokwaziswa, futhi imodeli engeyona i-autoregressive igcwalisa amathokheni emininingwane asele. Le recipe ye-codec-LM igqugquzele abalandela njenge-VALL-E 2 namamodeli amaningi esisekelo senkulumo.
I-Technical Insight
Iqhinga ukuhunyushwa kwe-hybrid ngaphezulu kwamathokheni e-codec e-hierarchical. Isigaba se-autoregressive sibikezela amathokheni abaluleke kakhulu e-codebook eyodwa ngesikhathi, athwebula i-prosody nokuqukethwe. Ama-codebook asele, anezela imininingwane emihle ye-acoustic, abikezelwa ngokuhambisana imodeli engeyona ye-autoregressive ebekwe ekusakazeni kokuqala kanye nokwaziswa kwesipika. Lokhu kuhlukaniswa kugcina izinga eliphezulu ngenkathi kugwema izindleko zokukhiqiza wonke amathokheni ngokulandelana, futhi ukusebenzisa i-codec kusho inkulumo nombhalo kungenziwa imodeli ngemishini efanayo ye-transformer.
I-Mastering VALL-E kanye ne-Codec Language Models
I-VALL-E ifake uzimele kabusha umbhalo-kuya-inkulumo njengenkinga yemodeli yolimi phezu kwamathokheni ekhodekhi yomsindo, okuvumela ukwenziwa kwezwi kusuka kumasekhondi amathathu nje esampula. Kubonise ukuthi umbhalo ofanayo wokubikezela ithokheni elandelayo wokunika amandla umbhalo we-LLM ungakhiqiza inkulumo engokwemvelo ngokuphawulekayo, evezayo. I-VALL-E kanye ne-Codec Language Models ihlezi ku-audio-AI workflows eguqula inkulumo, umculo, nomsindo wokuxhumana, ukufinyeleleka, nokukhiqizwa kwemidiya. Ukuze wakhe ukuqonda okujulile, phatha i-VALL-E ne-Codec Language Models njengemodeli yokusebenza, hhayi isici esisodwa: chaza imiphumela efiselekayo, ucacise ukucabanga, futhi uhlukanise lokho uhlelo olungakwenza ngokwethembeka kulokho okusadinga ukwahlulela kochwepheshe.
Empeleni, amaqembu aqinile asebenzisa i-VALL-E kanye ne-Codec Language Models aphatha ikhwalithi, ukubambezeleka, kanye nemvume njengezingxenye ezibalulekile ngokulinganayo zesu lokuthunyelwa. Babhala imibandela yempumelelo ecacile, ukuhlola okuqhathaniswa nedatha engokoqobo nokugeleza komsebenzi, futhi baphindaphinde ngokusekelwe kumaphethini okuhluleka aqashiwe esikhundleni sokuwina kwebhentshimakhi yesikhathi esisodwa. Yilapho ukuqonda kwethiyori kuguquka kube amandla ahlala njalo kuwo wonke umkhiqizo, inqubomgomo, kanye nokusebenza.
Ithuthukisa ukufinyeleleka ngokuloba, ukulandisa, nezixhumi ezibonakalayo zezwi. Ngesikhathi esifanayo, ukusetshenziswa kabi kwezwi kanye nezingozi zokuzenza ongeyena ziyakhuphuka uma imvume ingekho. Indlela eqine kakhulu iwukuhlanganisa isivinini sokuhlola nesiyalo sokuphatha: qhuba abashayeli bezindiza, bamba ubufakazi, ushicilele amalogi ezinqumo, futhi ubuyekeze izivikelo ngokuqhubekayo njengoba imodeli yokuziphatha, okulindelwe ngabasebenzisi, kanye nezimfuneko zokulawula zishintsha.
I-Strategic Impact
Ithuthukisa ukufinyeleleka ngokuloba, ukulandisa, nezixhumi ezibonakalayo zezwi.
Ithuthukisa ukufinyeleleka ngokuloba, ukulandisa, nezixhumi ezibonakalayo zezwi. Ekusetshenzisweni kwekhwalithi ephezulu, lokhu kuhunyushwa emithethweni yokusebenza elinganisekayo, imingcele yobunikazi, nemikhuba yokubuyekeza ephindelelayo ukuze amaqembu akwazi ukukala ukuzethemba esikhundleni sokukala ukungaqondakali.
Amaqembu emidiya angathumela umsindo opholishiwe ngokushesha ngamabhajethi amancane.
Amaqembu emidiya angathumela umsindo opholishiwe ngokushesha ngamabhajethi amancane. Ekusetshenzisweni kwekhwalithi ephezulu, lokhu kuhunyushwa emithethweni yokusebenza elinganisekayo, imingcele yobunikazi, nemikhuba yokubuyekeza ephindelelayo ukuze amaqembu akwazi ukukala ukuzethemba esikhundleni sokukala ukungaqondakali.
Amasistimu abhekene nekhasimende angacubungula ukusebenzelana okukhulunyiwe ngesilinganiso esikhulu.
Amasistimu abhekene nekhasimende angacubungula ukusebenzelana okukhulunyiwe ngesilinganiso esikhulu. Ekusetshenzisweni kwekhwalithi ephezulu, lokhu kuhunyushwa emithethweni yokusebenza elinganisekayo, imingcele yobunikazi, nemikhuba yokubuyekeza ephindelelayo ukuze amaqembu akwazi ukukala ukuzethemba esikhundleni sokukala ukungaqondakali.
Ukuqaliswa Komhlaba Wangempela
Ukuhlanganisa izwi kusuka kumasekhondi ambalwa omsindo kubasizi abaqondene nawe noma amathuluzi okufinyelela abuyisela izwi elilahlekile
Ukwenza kube okwasendaweni nokubizela ividiyo kwezinye izilimi kuyilapho ugcina i-timbre yesikhulumi sokuqala
Ikhiqiza ukulandisa okuvezayo, okufaniswa nomxholo okugcina indawo ye-acoustic yokurekhodwa
Isebenza njengomgogodla wenkulumo kuma-multimodal abasizi abaqondayo futhi bakhiqize umsindo okhulunyiwe
Amaphethini Okusebenzisa
I-VALL-E ne-Codec Language Models iyasebenza
Ukuhlanganisa izwi kusuka kumasekhondi ambalwa omsindo kubasizi abaqondene nawe noma amathuluzi okufinyelela abuyisela izwi elilahlekile.
Ukuhlanganisa izwi elisuka kumasekhondi ambalwa omsindo kubasizi abaqondene nawe noma amathuluzi okufinyelela abuyisela izwi elilahlekile Amathimba ngokuvamile athola imiphumela engcono uma echaza ikhwalithi ephezulu ngaphambili, egcina indlela yokukhuphuka yomuntu yamakesi asemaphethelweni, futhi alandelele kokubili izinzuzo zokukhiqiza nezindleko zamaphutha ngokuhamba kwesikhathi.
I-VALL-E ne-Codec Language Models iyasebenza
Ukwenza kube okwasendaweni nokubizela ividiyo kwezinye izilimi kuyilapho ugcina i-timbre yesikhulumi sokuqala.
Ukwenza kube okwasendaweni kanye nokudubisela ividiyo kwezinye izilimi kuyilapho ugcina i-timbre yesipika sokuqala Amathimba ngokuvamile athola imiphumela engcono uma echaza izilinganiso zekhwalithi ngaphambili, egcina indlela yokukhuphuka yabantu yamakesi asemaphethelweni, futhi elandelela kokubili izinzuzo zokukhiqiza nezindleko zamaphutha ngokuhamba kwesikhathi.
I-VALL-E ne-Codec Language Models iyasebenza
Ikhiqiza ukulandisa okuvezayo, okufaniswa nomxholo okugcina indawo ye-acoustic yokurekhodwa.
Ukukhiqiza ukulandisa okuvezayo, okuhambisana nomxholo okugcina indawo yokulalela okurekhodiwe Amathimba ngokuvamile athola imiphumela engcono uma echaza izinga eliphezulu ngaphambili, egcina indlela yokukhuphuka yabantu yamakesi asemaphethelweni, futhi alandelele kokubili izinzuzo zokukhiqiza nezindleko zamaphutha ngokuhamba kwesikhathi.
I-VALL-E ne-Codec Language Models iyasebenza
Isebenza njengomgogodla wenkulumo kuma-multimodal abasizi abaqondayo futhi bakhiqize umsindo okhulunyiwe.
Ukukhonza njengomgogodla wenkulumo kubasizi be-multimodal abaqondayo bobabili futhi bakhiqize Amathimba omsindo okhulunywayo ngokuvamile bathola imiphumela engcono uma bechaza ikhwalithi ephezulu ngaphambili, begcina indlela yokukhuphuka yomuntu yamakesi asemaphethelweni, futhi balandelele kokubili izinzuzo zokukhiqiza nezindleko zamaphutha ngokuhamba kwesikhathi.
Izingozi & Guardrails
Ukusetshenziswa kabi kwezwi kanye nezingozi zokuzenza ongeyena ziyanda uma imvume ingekho.
Ukunemba kungase kwehle kuzo zonke izinhlobo zokuphimisela, izilimi zesigodi, noma izindawo ezinomsindo.
Umsindo wokwenziwa ungenziwa iphutha njengenkulumo eyiqiniso ngaphandle kokulebula okucacile.
Ukuqalisa Umhlahlandlela
Thola imvume esobala yokuthwebula izwi, ukuhlanganisa, nokusebenzisa kabusha.
Thola imvume esobala yokuthwebula izwi, ukuhlanganisa, nokusebenzisa kabusha. Phatha isinyathelo ngasinye njengesango lobufakazi: uma imibandela ingafinyelelwa, misa ukukhishwa, vala igebe, bese unweba ukusetshenziswa.
Ikhwalithi yokuhlola kuzo zonke izipikha nezimo zangemuva.
Ikhwalithi yokuhlola kuzo zonke izipikha nezimo zangemuva. Phatha isinyathelo ngasinye njengesango lobufakazi: uma imibandela ingafinyelelwa, misa ukukhishwa, vala igebe, bese unweba ukusetshenziswa.
Chaza ukuthi kunini lapho umuntu kufanele abuyekeze noma agunyaze okuphumayo.
Chaza ukuthi kunini lapho umuntu kufanele abuyekeze noma agunyaze okuphumayo. Phatha isinyathelo ngasinye njengesango lobufakazi: uma imibandela ingafinyelelwa, misa ukukhishwa, vala igebe, bese unweba ukusetshenziswa.
Lebula umsindo wokwenziwa futhi ugcine amarekhodi atholakalayo ukuze aziphendulele.
Lebula umsindo wokwenziwa futhi ugcine amarekhodi atholakalayo ukuze aziphendulele. Phatha isinyathelo ngasinye njengesango lobufakazi: uma imibandela ingafinyelelwa, misa ukukhishwa, vala igebe, bese unweba ukusetshenziswa.