Tách thân cây Spleeter

Music source separation is the task of taking a finished, mixed recording and estimating the individual parts that went into it. Those parts are called stems: a vocal stem, a drum stem, a bass stem, and so on. Separation is hard because the mix is a sum of sounds that overlap in time and frequency, and the original multitrack session is usually unavailable. Spleeter is a tool that does this with pretrained neural networks. It was built by Romain Hennequin, Anis Khlif, Felix Voituret and Manuel Moussallam at Deezer Research in Paris, released on GitHub in 2019 under the MIT license, and described in the Journal of Open Source Software (JOSS) on June 24, 2020.

Spleeter ships three pretrained configurations. The 2stems model separates vocals from accompaniment. The 4stems model follows the categories of the SiSEC 2018 evaluation campaign: vocals, drums, bass and other. The 5stems model adds a piano stem, and the authors described it as, to their knowledge, the first released model to perform that separation. Users install a Python package and run one command, for example spleeter separate -p spleeter:2stems -o output song.mp3. The tool is built on TensorFlow, Google's machine learning library, so it runs on either a CPU or a GPU, and it is also packaged for conda and Docker.

The method works on a spectrogram, a picture of sound that shows how much energy is present at each frequency over time. Spleeter computes a short-time Fourier transform (STFT), which slices the audio into overlapping frames and measures the frequencies in each. A U-Net, a convolutional neural network shaped like an encoder that compresses the input followed by a decoder that expands it again, with skip connections that pass detail across, looks at the mixture's magnitude spectrogram. The paper specifies 12-layer U-Nets, six encoder and six decoder layers. For each stem, a U-Net outputs a soft mask: a grid of values between 0 and 1 saying what share of the energy in each time-frequency cell belongs to that stem. Multiplying the mixture spectrogram by the mask gives an estimate of the stem, which is then converted back to audio. Because Spleeter normalizes the masks so that, in every cell, the shares for all stems add up to one, the separated stems sum back to approximately the original mix within the processed frequency range. Instead of plain masking, users can choose multichannel Wiener filtering, a classic statistical step that refines the estimates using all stems together.

Training used Deezer's internal datasets, including one called Bean, with the Adam optimizer and an L1 loss, the average absolute difference between the masked mixture spectrogram and the true stem spectrogram. Training took about a week on a single GPU. The training audio could not be published for copyright reasons, which is why Deezer released trained models instead: it was the only way to share the results.

Speed was a design goal. The JOSS paper reports that Spleeter separated the full MUSDB18 test set, about 3 hours and 27 minutes of audio, into four stems in under two minutes on a single GeForce RTX 2080 GPU, including about 15 seconds of model loading. That works out to more than 100 seconds of stereo audio processed per second. The authors note it still cannot run in real time on live input, because the model needs a buffer of audio to work on.

Quality is measured with signal-to-distortion ratio (SDR), in decibels, where higher means the estimate is closer to the true stem. On MUSDB18, the standard benchmark of 150 tracks (100 for training, 50 for testing), Spleeter was evaluated without being trained or tuned on that data. With Wiener filtering it reported vocal SDR of 6.86 dB, drums 6.71, bass 5.51 and other 4.55. In the same table, Open-Unmix scored 6.32 on vocals and the waveform model Demucs scored 7.05 on vocals and 6.70 on bass. The authors summarized Spleeter as competitive with Open-Unmix and almost on par with Demucs, at much higher speed.

Since then, separation quality has moved on. Hybrid Transformer Demucs, posted to arXiv in November 2022 by Simon Rouard, Francisco Massa and Alexandre Defossez, reported 9.20 dB average SDR on MUSDB HQ using extra training data, sparse attention and per-source fine-tuning. Models like this combine waveform and spectrogram processing and are larger and slower. Spleeter's appeal today is mainly its simplicity, speed, permissive license and long track record as a baseline.

The limitations are concrete. Masking cannot create sound that is hidden under another instrument, so stems contain bleed, where traces of other instruments leak through, and artifacts such as watery or phasey textures. The other and piano stems score lowest, because they lump together many instruments. Spleeter's default configurations only process the lower part of the spectrum (see the technical section), so high-frequency detail can be lost. MUSDB18 itself is limited to academic use and contains mostly Western popular music, so benchmark numbers may not reflect results on other genres. And separating a commercial recording does not change who owns its copyright.