Overview
EnCodec ndeye Meta's high-fidelity neural audio codec inodzvanya kutaura uye mimhanzi payakaderera bitrate ine mhando inokwikwidza inorema mafomati. Izvo zvine basa nekuti inosimbisa yemazuva ano inobereka maodhiyo masisitimu uye ngarava mune yakavhurika-sosi fomu kuti chero munhu ashandise.
EnCodec Audio Compression inogara muodhiyo-AI workflows inoshandura kutaura, mimhanzi, uye ruzha rwekutaurirana, kuwanikwa, uye kugadzirwa kwenhau.
Deep Dive
Yakaburitswa ne Meta AI muna 2022, EnCodec inotevera SoundStream blueprint ye encoder, yasara vector quantizer (RVQ), uye decoder yakadzidziswa kupera, asi inowedzera zvakati wandei. Iyo inoshandisa kutenderera-inokwanisa convolutional encoder, yakawanda-scale spectrogram uye nguva-domain kuvakazve kurasikirwa, uye vanopikisa vanosarura kune yemhando yekunzwisisa. Mupiro unozivikanwa idiki Transformer-based entropy modhi iyo inowedzera kudzvanya macode akaverengerwa zvisina kurasikirwa, kudzvanya mabheti kunze pasina kurasikirwa kwemhando. EnCodec inosvitsawo chiyero chinoyera otomatiki akawanda anokwikwidza kurasikirwa kwekudzidziswa kuti vagare vakagadzikana. Inobata 24 kHz monophonic uye 48 kHz stereo odhiyo, inoshanda pamabitrates akadai se1.5, 3, 6, uye 12 kbps, uye pa6 kbps inosvika pamhando inofananidzwa neMP3 pa64 kbps. Zviratidzo zvayo zvine simba Meta's MusicGen uye AudioGen.
Technical Insight
EnCodec's encoder inodzikisira iyo waveform ine strided convolutions kuita yakatevedzana yakateedzana, iyo RVQ inoshandura kuita akaturikidzana codebook indices. Chimiro chemutauro chakareruka cheTransformer chinofanotaura zvingangoitika zvetokens uye arithmetic-makodhi, kudzoreredza kumwe kudzvanywa mahara. Iyo yekudzidzira balancer inodzoreredza mipiro ye gradient kubva pakuvaka patsva, spectral, uye kurasikirwa kweanopikisa kuitira kuti pasave netemu imwe chete inotonga, izvo zvinoita kuti kudzidziswa kwezvinangwa zvakawanda kugadzikana pazere bitrate renji.
Mastering EnCodec Audio Compression
EnCodec ndeye Meta's high-fidelity neural audio codec inodzvanya kutaura uye mimhanzi payakaderera bitrate ine mhando inokwikwidza inorema mafomati. Izvo zvine basa nekuti inosimbisa yemazuva ano inobereka maodhiyo masisitimu uye ngarava mune yakavhurika-sosi fomu kuti chero munhu ashandise. EnCodec Audio Compression inogara muodhiyo-AI workflows inoshandura kutaura, mimhanzi, uye ruzha rwekutaurirana, kuwanikwa, uye kugadzirwa kwenhau. Kuvaka kunzwisisa kwakadzama, bata EnCodec Audio Compression semuenzaniso wekushandisa, kwete chinhu chimwe chete: tsanangura zvinodikanwa, kujekesa fungidziro, uye patsanura izvo zvinogona kuitwa nehurongwa hwakavimbika kubva kune zvichiri kuda kutonga kwenyanzvi.
Mukuita, zvikwata zvakasimba zvinoshandisa EnCodec Audio Compression zvinobata mhando, latency, uye mvumo sezvikamu zvakakosha zvakaenzana zvehurongwa hwekuendesa. Ivo vanonyora zvakajeka maitiro ebudiriro, bvunzo vachipokana ne data rechokwadi uye mafambiro ebasa, uye iterate zvichibva pane zvakacherechedzwa maitiro ekutadza kwete kuhwina-nguva imwe chete yebhenji. Apa ndipo apo kunzwisisa kwe theoretical kunoshanduka kuve kugona kwakasimba pane chigadzirwa, mutemo, uye mashandiro.
Inonatsiridza kusvikika kuburikidza nekunyora, kurondedzera, uye mazwi ekubatanidza. Panguva imwecheteyo, kusashandiswa kweIzwi zvisizvo uye njodzi dzekuedzesera dzinowedzera kana chibvumirano chisipo. Nzira yakatsiga ndeyekubatanidza kukurumidza kuyedza nekutonga: mhanyisa vatyairi vendege, tora humbowo, buritsa matanda esarudzo, uye urambe uchivandudza chengetedzo semaitiro emuenzaniso, zvinotarisirwa nemushandisi, uye zvinodikanwa zvekutonga.
Strategic Impact
Inonatsiridza kusvikika kuburikidza nekunyora, kurondedzera, uye mazwi ekubatanidza.
Inonatsiridza kusvikika kuburikidza nekunyora, kurondedzera, uye mazwi ekubatanidza. Mukutumirwa kwemhando yepamusoro, izvi zvinoshandurirwa kuita mitemo inoyerwa yekushanda, miganhu yevaridzi, uye tsika dzekudzokorora dzinodzokororwa kuitira kuti zvikwata zvikwire kuvimba pane kukwidza kusajeka.
Zvikwata zveMedia zvinogona kutumira odhiyo yakakwenenzverwa nekukurumidza nemabhajeti madiki.
Zvikwata zveMedia zvinogona kutumira odhiyo yakakwenenzverwa nekukurumidza nemabhajeti madiki. Mukutumirwa kwemhando yepamusoro, izvi zvinoshandurirwa kuita mitemo inoyerwa yekushanda, miganhu yevaridzi, uye tsika dzekudzokorora dzinodzokororwa kuitira kuti zvikwata zvikwire kuvimba pane kukwidza kusajeka.
Masisitimu anotarisana nevatengi anogona kugadzirisa kutaurirana kwekutaura pamwero mukuru.
Masisitimu anotarisana nevatengi anogona kugadzirisa kutaurirana kwekutaura pamwero mukuru. Mukutumirwa kwemhando yepamusoro, izvi zvinoshandurirwa kuita mitemo inoyerwa yekushanda, miganhu yevaridzi, uye tsika dzekudzokorora dzinodzokororwa kuitira kuti zvikwata zvikwire kuvimba pane kukwidza kusajeka.
Real-World Implementation
Tokening audio ye Meta's MusicGen uye AudioGen mavara-kune-audhiyo jenareta
Kudzvanya 24 kHz kutaura kune 1.5-6 kbps kune bandwidth-shoma kutapurirana
Encoding 48 kHz stereo mimhanzi ine mhando padyo neMP3 pakakwirira zvakanyanya bitrate
Kushanda seyakavhurika-sosi yekudonha-in codec yekutsvagisa uye odhiyo ML mapaipi kuburikidza neakaburitswa macheki
Maitiro Ekuita
EnCodec Audio Compression mukuita
Tokening audio ye Meta's MusicGen uye AudioGen zvinyorwa-kune-audhiyo jenareta.
Tokenizing odhiyo yeMeta's MusicGen uye AudioGen mavara-kune-audhiyo majenareta Matimu anowanzo kuwana mibairo iri nani kana achinge atsanangura mhando yepamusoro kumberi, chengetedza nzira yekukwira kwevanhu yemakesi emupendero, uye kuteedzera zvese zvakawanikwa pakubereka uye mutengo wekukanganisa nekufamba kwenguva.
EnCodec Audio Compression mukuita
Kudzvanya 24 kHz kutaura kune 1.5-6 kbps kune bandwidth-shoma kutapurirana.
Kudzvanya 24 kHz kutaura kune 1.5-6 kbps kune bandwidth-yakaganhurirwa kutapurirana Matimu anowanzo kuwana mhedzisiro iri nani kana achinge atsanangura emhando yepamusoro kumberi, chengetedza nzira yekukwira kwevanhu yemakesi emupendero, uye kuteedzera zvese zvakawanikwa zvechigadzirwa nemitengo yekukanganisa nekufamba kwenguva.
EnCodec Audio Compression mukuita
Encoding 48 kHz stereo mimhanzi ine mhando padyo neMP3 pakakwirira zvakanyanya bitrate.
Encoding 48 kHz stereo mimhanzi ine mhando padhuze neMP3 payakanyanya kukwirisa bitrate Matimu anowanzo kuwana mibairo iri nani kana achinge atsanangura mhando yepamusoro kumberi, chengetedza nzira yekukwira kwevanhu yemakesi ekumucheto, uye kuteedzera zvese zvakawanikwa zvechigadzirwa uye mutengo wekukanganisa nekufamba kwenguva.
EnCodec Audio Compression mukuita
Kushanda seyakavhurika-sosi yekudonha-in codec yekutsvagisa uye odhiyo ML mapaipi kuburikidza neakaburitswa macheki.
Kushanda seyakavhurika-sosi yekudonhedza-in codec yekutsvagisa uye odhiyo ML mapaipi kuburikidza neakaburitswa macheki Matimu anowanzo kuwana zvirinani kana achinge atsanangura emhando yepamusoro kumberi, chengetedza nzira yekukwira kwevanhu yemakesi ekumucheto, uye kuteedzera zvese zvakawanikwa zvechigadzirwa uye mutengo wekukanganisa nekufamba kwenguva.
Njodzi & Guardrails
Kusashandisa izwi zvisizvo uye njodzi dzekuedzesera dzinowedzera kana chibvumirano chisipo.
Kururama kunogona kudonha mumitauro, mataurirwo, kana nharaunda dzine ruzha.
Synthetic audio inogona kukanganisa kutaura kwechokwadi isina mavara akajeka.
Implementation Roadmap
Wana mvumo yakajeka yekutora inzwi, kugadzira, uye kushandisa zvakare.
Wana mvumo yakajeka yekutora inzwi, kugadzira, uye kushandisa zvakare. Bata nhanho yega yega segedhi rehumbowo: kana maitiro asina kusangana, imbomira kuburitsa, vhara gaka, uye wobva wawedzera kushandiswa.
Yedza mhando pavatauri vakasiyana uye mamiriro ekumashure.
Yedza mhando pavatauri vakasiyana uye mamiriro ekumashure. Bata nhanho yega yega segedhi rehumbowo: kana maitiro asina kusangana, imbomira kuburitsa, vhara gaka, uye wobva wawedzera kushandiswa.
Tsanangura apo munhu anofanira kuongorora kana kubvumidza zvabuda.
Tsanangura apo munhu anofanira kuongorora kana kubvumidza zvabuda. Bata nhanho yega yega segedhi rehumbowo: kana maitiro asina kusangana, imbomira kuburitsa, vhara gaka, uye wobva wawedzera kushandiswa.
Label synthetic odhiyo uye chengetedza marekodhi ekuzvidavirira.
Label synthetic odhiyo uye chengetedza marekodhi ekuzvidavirira. Bata nhanho yega yega segedhi rehumbowo: kana maitiro asina kusangana, imbomira kuburitsa, vhara gaka, uye wobva wawedzera kushandiswa.