Ubuyobozi bwa Audio AI

Demucs na HT-Demucs Umuziki Inkomoko Gutandukana

Demucs ni __AIU_PROTECTED_13 __ 'ifunguye-isoko yumuziki isoko yo gutandukanya; Hybrid Transformer Demucs (HT-Demucs) igabanya indirimbo mumajwi, ingoma, bass, nibindi biti ukoresheje flake ya flake na sprogramu.

Umva iki gitaboAI-byavuzwe · ~8 min

Kuriyi page9 min soma
  1. Ibyingenzi byingenzi
  2. Kwibira cyane
  3. Separating a song into four stems with Demucs
  4. Gushyira mu bikorwa Isi
  5. Kazoza ka Demucs na HT-Demucs Inkomoko Yumuziki Gutandukana
  6. Kubishyira mubikorwa
  7. Inkomoko no gusoma
  8. Komeza Ubushakashatsi
  9. Ibibazo bikunze kubazwa

Ibyingenzi byingenzi

  1. Demucs is an open-source model family that separates a mixed song into vocals, drums, bass and other stems.
  2. The 2019 Demucs paper worked directly on waveforms and averaged 6.3 dB SDR on MUSDB18, beating the spectrogram methods it was compared against.
  3. Hybrid Demucs combined waveform and spectrogram branches and won Sony's Music Demixing Challenge 2021.
  4. Hybrid Transformer Demucs reached 9.20 dB SDR on MUSDB in 2022 using a cross-domain Transformer and 800 extra training songs.
  5. Separated stems still contain bleed between instruments, and the six-source model's piano output is described by its own documentation as weak.
  6. The official Meta repository is archived, and running the default model needs about 7 GB of GPU memory or runs slowly on a CPU.

Kwibira cyane

Music source separation is the task of taking a mixed recording, the single stereo file that ends up on a streaming service, and recovering the individual parts that were added together to make it. The standard research setting asks for four stems: vocals, drums, bass and other, where other holds everything else. Demucs is a family of deep learning models for this task created by Alexandre Défossez and colleagues at Meta AI (then Facebook AI Research). The code and pretrained models are free on GitHub, and they became a common default for musicians, remixers, educators and researchers.

Most earlier systems worked on a spectrogram, a picture of sound that shows how much energy each frequency has at each moment, computed with the short-time Fourier transform (STFT). A network predicts a mask for each source, a set of values between 0 and 1 that says what fraction of each time-frequency cell belongs to, say, the vocals, and the masked spectrogram is turned back into audio. The weakness is phase, the timing information in each frequency component, which masks usually copy from the mixture rather than estimate. The original Demucs paper, posted in November 2019 by Défossez, Nicolas Usunier, Léon Bottou and Francis Bach, instead worked waveform to waveform: raw audio samples in, raw audio samples out, so the network learns phase as well as loudness.

That first model was a U-Net, an encoder-decoder network with skip connections that pass detail from each encoder layer straight to the matching decoder layer. The encoder is a stack of strided one-dimensional convolutions that shrink the time axis while adding channels, so deep layers summarize longer stretches of sound. A bidirectional LSTM, a recurrent layer that reads the compressed sequence forwards and backwards, sits in the middle to capture longer context. The decoder mirrors the encoder with transposed convolutions and outputs one waveform per source. The paper reported an average of 6.3 dB signal-to-distortion ratio (SDR) on the MUSDB18 test set with data augmentation, beating every architecture it compared against, and 6.8 dB with 150 extra training songs. For bass, that extra-data result surpassed the ideal ratio mask, an oracle that knows the true sources.

The second milestone was Hybrid Demucs, described by Défossez in November 2021 for the ISMIR 2021 Music Demixing (MDX) Workshop. It runs two U-Net branches in parallel, one on the waveform and one on the spectrogram, which share layers in the middle, and lets training decide which representation suits each source. Hybrid Demucs won the Music Demixing Challenge 2021 organized by Sony. It improved SDR by 1.4 dB across sources on MUSDB HQ, and in human listening tests scored 2.83 out of 5 for overall quality against 2.36 for the waveform-only version, with fewer contamination artifacts than the second-place challenge entry.

The third milestone, Hybrid Transformer Demucs (HT Demucs), was posted to arXiv in November 2022 by Simon Rouard, Francisco Massa and Défossez. It keeps the two-branch hybrid design but replaces the innermost layers with a cross-domain Transformer encoder, a stack of attention layers that let every time step look at every other and let the waveform and spectrogram branches exchange information. The paper found the Transformer performed poorly when trained only on the 100 MUSDB training songs but beat Hybrid Demucs by 0.45 dB when both used 800 extra songs. With sparse attention, which skips most pairwise comparisons so longer clips fit in memory, and per-source fine-tuning, it reached 9.20 dB SDR, then state of the art on MUSDB with extra data. HT Demucs is the v4 release and the default model in the package.

Compared with alternatives, Demucs sits between pure spectrogram maskers such as Open-Unmix, which scored 5.33 dB in the 2019 paper's comparison table, and pure waveform models such as Conv-TasNet, a speech separation network the 2019 paper adapted to music. The paper found Conv-TasNet scored well on numbers but human listeners heard significant artifacts, while Demucs sounded more natural. The hybrid design was a direct answer to the finding that some sources, such as sustained vocals, are easier in the frequency domain while transient drums are easier in the time domain.

Limits are audible. The 2019 paper's listeners noticed bleeding, where one instrument leaks into another stem, especially between vocals and other. The six-source model's piano stem is described in the README as weak. Scores come from MUSDB18, which holds 150 tracks and about 10 hours of mostly Western popular music, so results on orchestral, heavily processed or non-Western material may differ. The models also cannot recover what was never recorded separately: reverb shared by all parts, or two instruments playing the same note in the same register, will blur. Training data rights matter too, since MUSDB18 is restricted to academic use.

In practice, Demucs installs as a Python package and runs from the command line. The README says it needs at least 3 GB of GPU memory, about 7 GB with default settings, and a segment option lowers memory use at some cost in quality. On a CPU it takes roughly 1.5 times the length of the track. The original Meta repository is now archived and read-only because the author no longer works at Meta, and the README points to a personal fork maintained for important bug fixes. Separated stems are estimates, not the original multitracks, so anyone releasing a remix still needs the rights to the source recording.

03Urugero rwakazi

Separating a song into four stems with Demucs

  1. Install the package with pip install demucs on a computer with Python, ideally with a GPU that has at least 3 GB of memory.

  2. Run demucs on a song file you have the rights to use; the default htdemucs model writes four WAV files: drums, bass, vocals and other.

  3. If memory runs out, add a smaller segment value, or add -d cpu and expect processing to take about 1.5 times the song's length.

  4. Listen to each stem on headphones and note bleed, for example cymbals faintly audible in the vocal stem.

  5. Re-run with -n htdemucs_ft and compare the vocal stems; the fine-tuned model is slower but may be slightly cleaner.

  6. Load all four stems into an audio editor at zero gain and play them together to check they add back up to the original mix.

Ibyo byerekana

Four stem files and a written note on audible artifacts per stem. The sum check should sound almost identical to the original song; any obvious difference means a stem is missing content or a file was misaligned.

Gushyira mu bikorwa Isi

Making a karaoke track by running demucs --two-stems=vocals on a song, which the official README documents as a way to separate the vocals from everything else in one pass.

Isolating a drum or bass part for practice or transcription with the default htdemucs model, then checking the stem by ear for bleed from other instruments.

Using Demucs inside Ultimate Vocal Remover, an MIT-licensed desktop app whose credits list Demucs alongside MDX-Net and VR architectures as bundled separation models.

Benchmarking a new separation research model against Demucs on MUSDB18-HQ, the 150-track dataset of uncompressed stereo stems at 44.1 kHz used throughout the Demucs papers.

Splitting guitar and piano as extra stems with the experimental htdemucs_6s model, while noting the README's warning that its piano output has notable bleeding and artifacts.

Kazoza ka Demucs na HT-Demucs Inkomoko Yumuziki Gutandukana

Open problems in music source separation are now less about average benchmark scores and more about artifacts, data and scope. MUSDB18 is small and genre-limited, and the 2022 HT Demucs paper showed that its Transformer only beat older designs once 800 extra training songs were added, which points to data rather than architecture as the main constraint. Licensed multitrack data is scarce, and the benchmark itself is restricted to academic use. A second direction is more and finer stems, such as guitar, piano, strings and individual voices; the experimental six-source Demucs model shows the difficulty, since its own README calls the piano output weak. Evaluation is a third issue: SDR rewards low error energy but the 2021 Hybrid Demucs paper still relied on human listening tests to judge contamination and naturalness. Finally, the original Meta repository has been archived and is no longer maintained, so continued development depends on forks and on newer research systems that build on Demucs ideas.

Kubishyira mubikorwa

Impamvu ari ngombwa

  • Kugera no kugera. Itezimbere kugerwaho binyuze mu kwandukura, kuvuga, no guhuza amajwi.
  • Igiciro na bije. Amatsinda yibitangazamakuru arashobora kohereza amajwi yihuse hamwe na bije nto.
  • Umuvuduko n'igipimo. Sisitemu ireba abakiriya irashobora gutunganya imikoranire ivugwa murwego runini.

Witondere

  • Gukoresha nabi amajwi no kwigira ibyago byiyongera mugihe uruhushya rubuze.
  • Ukuri kurashobora kugabanuka hejuru yimvugo, imvugo, cyangwa urusaku rwibidukikije.
  • Amajwi yubukorikori arashobora kwibeshya kumvugo yukuri nta kirango gisobanutse.

Intambwe ikurikira

  1. Shaka uruhushya rusobanutse rwo gufata amajwi, gukoroniza, no gukoresha.
  2. Ikizamini cyiza mubiganiro bitandukanye hamwe nuburyo bwimbere.
  3. Sobanura igihe umuntu agomba gusuzuma cyangwa kwemeza ibisubizo.
  4. Andika amajwi yubukorikori kandi ugumane inyandiko zerekana kubazwa.

Inkomoko no gusoma

  1. arXivMusic Source Separation in the Waveform Domain
  2. arXiv (ISMIR 2021 MDX Workshop)Hybrid Spectrogram and Waveform Source Separation
  3. arXiv (ICASSP 2023)Hybrid Transformers for Music Source Separation
  4. GitHub (Meta Research)facebookresearch/demucs: Code for the paper Hybrid Spectrogram and Waveform Source Separation
  5. SigSepMUSDB18 dataset
  6. GitHubUltimate Vocal Remover GUI

Komeza Ubushakashatsi

Ibaruwa y'amakuru y'ubuntu

Ibuka amakuru ya buri munsi ya AI

Inkuru eshatu zizewe za AI buri gitondo cy'iminsi y'akazi, zanditse mu rurimi rworoshye. Ubuntu iteka ryose, nta kwamamaza.

Imeri imwe buri munsi y'akazi. Kwiyimura mu buryo bumwe. Ntitwigera tugurisha cyangwa ngo dusangire aderesi yawe.

Isuzume

Take the Demucs and HT-Demucs Music Source Separation quiz

Ibisubizo ako kanya kuri buri kibazo, n'icyemezo gishobora gusangirwa gifite ID yagenzurwa umaze gutsinda isomo.

Tangira ikibazo

Fasha uburezi bwa AI bw'ubuntu. AI Understanding ni umuryango utegamiye kuri leta wa 501(c)(3) — nta kwamamaza, nta kwishyura, iteka ryose. Tanga impano

Ibibazo bikunze kubazwa

What is Demucs and HT-Demucs Music Source Separation?

Demucs ni __AIU_PROTECTED_13 __ 'ifunguye-isoko yumuziki isoko yo gutandukanya; Hybrid Transformer Demucs (HT-Demucs) igabanya indirimbo mumajwi, ingoma, bass, nibindi biti ukoresheje flake ya flake na sprogramu.

Niki Demucs ikora indirimbo irangiye?

Demucs ikora inkomoko yumuziki itandukanya, igabanya stereo ivanze mubikoresho byihariye nibiti byijwi.

Niki gituma Hybrid Demucs 'hybrid'?

Hybrid Demucs ikomatanya igihe-domaine ishami ryumurongo nishami rya spekrogramu, bigahuza imbaraga zabo.

Which metric did the original Demucs paper report for separation quality on MUSDB?

The original paper reports average SDR for music-source separation on MUSDB; the result belongs to that benchmark and setup.

Which research organization is associated with the original Demucs project?

The original Demucs paper lists FAIR affiliations and the official project repository is under facebookresearch. This describes the research origin of the project.

Ni ukubera iki Demucs yo hambere yakoraga muburyo butaziguye?

Gukorera muri domaine yumurongo ubika icyiciro, ubwo buryo bwa spekrogramu-ubunini bukunze guta kandi bugomba kugereranya.