VALL-E uye Codec Mutauro Models
VALL-E yakagadziridza zvakare mavara-kune-kutaura sedambudziko rekuenzanisira mutauro pamusoro pemaodhiyo ecodec tokens, ichigonesa kuumba izwi kubva pamasekonzi matatu chete emuenzaniso.
Pfupiso
It showed that the same next-token prediction powering text LLMs can generate remarkably natural, expressive speech.
Kudzika Kwakadzika
Yakaziviswa ne Microsoft kutanga kwa 2023, VALL-E inobata mataurirwo ekutaura semhando yemutauro. Panzvimbo pekufanotaura spectrogram, inofanotaura discrete acoustic tokens yeneural codec (EnCodec), saka chizvarwa chinova chinotevera-chiratidzo kufanotaura pamusoro pezwi rekuteerera. Tichipihwa kurekodha kwemasekondi matatu emutauri asingaonekwe pamwe nemavara anonongedzwa, VALL-E inoenderera nezwi remutauri, ichichengetedza timbre uye kunyangwe nharaunda yeacoustic. Yakadzidziswa paanenge 60,000 maawa ekutaura, zvakanyanya kudarika akajairwa TTS datasets, iyo yakapa iyo yakasimba zero-shot cloning. Nekuti macodec tokens akaturikidzana (kuburikidza neRVQ), VALL-E inoshandisa matanho maviri: autoregressive modhi inofanotaura yekutanga, yakaomarara tokeni rukova rwakamisikidzwa nekukasira, uye isiri-autoregressive modhi inozadza mune asara ma tokeni. Iyi codec-LM resipi yakafuridzira vatsivi vakaita seVALL-E 2 uye akawanda ekutaura nheyo modhi.
Technical Insight
Iwo manomano ndeye hybrid decoding pamusoro pehierarchical codec tokens. Iyo autoregressive nhanho inofanotaura iyo yakakosha yekutanga-codebook tokens imwe panguva, ichitora prosody uye zvirimo. Iwo asara macodebook, ayo anowedzera yakanaka acoustic tsanangudzo, anofanotaurwa mukufambirana neasiri-autoregressive modhi akaiswa parukova rwekutanga uye nekukasira kwemukurukuri. Kupatsanura uku kunochengeta hunhu hwakakwirira uchidzivirira mutengo wekugadzira chiratidzo chese zvakatevedzana, uye kushandisa codec zvinoreva kutaura uye zvinyorwa zvinogona kuenzanisirwa nemuchina weshanduko imwe chete.
Strategic Impact
Svika uye svika
Inonatsiridza kusvikika kuburikidza nekunyora, kurondedzera, uye mazwi ekubatanidza.
Mutengo uye bhajeti
Zvikwata zveMedia zvinogona kutumira odhiyo yakakwenenzverwa nekukurumidza nemabhajeti madiki.
Kumhanya uye chiyero
Masisitimu anotarisana nevatengi anogona kugadzirisa kutaurirana kwekutaura pamwero mukuru.
Ramangwana reVALL-E uye Codec Mutauro Models
Mitauro yeCodec iri kubatanidza kutaura nemhando dzemitauro mikuru, ichinongedza kumasisitimu akabatana anoteerera, kufunga, uye kutaura mune imwe modhi. Tarisira kugadzikana kuri nani uye zvishoma zvigadzirwa, chaiyo-nguva yekutepfenyura chizvarwa, uye kutonga kwakasimba pamusoro pemanzwiro uye maitiro. Iyo imwecheteyo ine simba cloning inoita kuti VALL-E ibatsire kuwanikwa uye kudhibhiwa zvakare inosimudza zvakadzama fake uye kushushikana kwemvumo, saka watermarking, izwi-yekusimbisa chengetedzo, uye chengetedzo yemitemo yave kuita chikamu chepakati pamashandisirwo aya masisitimu.
Real-World Implementation
Kuvhara izwi kubva pamasekonzi mashoma ekuteerera kune vanhu vabatsiri kana maturusi ekuwanika anodzoreredza izwi rakarasika
Kuunza uye kuisa vhidhiyo mune mimwe mitauro uchichengeta timb remutauri wekutanga
Kugadzira rondedzero inonakidza, inofananidzwa nemamiriro ezvinhu inochengetedza kurekodha mamiriro eacoustic
Kushanda semusana wekutaura mune multimodal vabatsiri vanonzwisisa uye kuburitsa odhiyo inotaurwa
Njodzi & Guardrails
Kushandisa izwi zvisizvo uye njodzi dzekuedzesera dzinowedzera kana chibvumirano chisipo.
Kururama kunogona kudonha mumitauro, mataurirwo, kana nharaunda dzine ruzha.
Synthetic audio inogona kukanganisa kutaura kwechokwadi isina mavara akajeka.
Implementation Roadmap
Wana mvumo yakajeka yekutora inzwi, kugadzira, uye kushandisa zvakare.
Yedza mhando pavatauri vakasiyana uye mamiriro ekumashure.
Tsanangura apo munhu anofanira kuongorora kana kubvumidza zvabuda.
Label synthetic odhiyo uye chengetedza marekodhi ekuzvidavirira.
Ramba Uchiongorora
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the VALL-E and Codec Language Models quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gaidhi rinotevera
Emergent Mano eMienzaniso Yakakura yeMutauro
Mibvunzo inowanzo bvunzwa
What is VALL-E and Codec Language Models?
VALL-E yakagadziridza zvakare mavara-kune-kutaura sedambudziko rekuenzanisira mutauro pamusoro pemaodhiyo ecodec tokens, ichigonesa kuumba izwi kubva pamasekonzi matatu chete emuenzaniso. Yakaratidza kuti iwowo anotevera-chiratidzo fungidziro yekusimbisa mameseji maLLM anogona kuburitsa zvinoshamisa, kutaura kunonzwika.
Ndeipi pfungwa yepakati inosiyanisa VALL-E kubva kumavambo mavara-kune-kutaura masisitimu?
VALL-E inogadziridza TTS sekufanotaura kunotevera pamusoro pe discrete audio codec tokens, inobata chizvarwa chekutaura senge modhi yemutauro.
VALL-E inoda mareferensi akawanda sei kuti igadzire izwi remutauri mutsva?
VALL-E inogona kuita zero-shot voice cloning kubva pamuenzaniso we3-sekondi yemutauri asingaonekwe.
Ndeipi neural codec inoshandiswa neVALL-E kugadzira maodhiyo tokeni?
VALL-E inovaka pa Meta's EnCodec, vachifanotaura ma tokens ayo akasimba ekubatanidza kutaura.
Sei VALL-E ichishandisa zvese zviri zviviri autoregressive uye isiri-autoregressive nhanho?
Codec tokens ndeye hierarchical kuburikidza neRVQ; VALL-E inofanotaura rwizi rwekutanga rwakakasharara autoregressively uye mamwe ma tokens asara anoenderana nekuita basa.
Zvichida kuti ingani data rekutaura rakadzidziswa VALL-E, richigonesa kusimba zero-shot cloning?
VALL-E yakadzidziswa paanenge 60,000 maawa ekutaura, zvakanyanya kupfuura akajairwa TTS datasets, iyo yakasimudzira yayo zero-pfuti generalization.