Audio AI ÚTMUTATÓ

Spleeter szár szétválasztása

Spleeter is an open-source music source separation tool from Deezer Research that splits a mixed song into 2, 4 or 5 stems, such as vocals, drums, bass and piano, using pretrained U-Net models.

Hallgassa meg ezt az útmutatótAI-narrált · ~7 min

Ezen az oldalon8 perc olvasás
  1. Áttekintés
  2. Főbb tanulságok
  3. Mély merülés
  4. Separating a song into four stems and checking the result
  5. Valós megvalósítás
  6. A Spleeter szárleválasztás jövője
  7. A gyakorlatba ültetés
  8. Források és további olvasmányok
  9. Folytassa a felfedezést
  10. Gyakran ismételt kérdések

Áttekintés

It matters because it made near state-of-the-art separation free, fast and one command away, and it still serves as a common baseline in research and audio software.

Főbb tanulságok

  1. Spleeter is Deezer Research's MIT-licensed music separation tool with pretrained 2, 4 and 5 stem models built on TensorFlow.
  2. Each Spleeter model is a 12-layer U-Net that predicts a soft mask over the mixture spectrogram for every stem.
  3. On one RTX 2080 GPU, Spleeter separated about 3.5 hours of MUSDB18 test audio into four stems in under two minutes.
  4. With Wiener filtering, Spleeter reported 6.86 dB vocal SDR on MUSDB18, compared with 6.32 dB for Open-Unmix and 7.05 dB for Demucs.
  5. Spleeter's default configuration processes frequencies up to about 11 kHz, so higher frequency content is not separated.
  6. Later models such as Hybrid Transformer Demucs reported 9.20 dB average SDR on MUSDB HQ in 2022, well above Spleeter.

Mély merülés

Music source separation is the task of taking a finished, mixed recording and estimating the individual parts that went into it. Those parts are called stems: a vocal stem, a drum stem, a bass stem, and so on. Separation is hard because the mix is a sum of sounds that overlap in time and frequency, and the original multitrack session is usually unavailable. Spleeter is a tool that does this with pretrained neural networks. It was built by Romain Hennequin, Anis Khlif, Felix Voituret and Manuel Moussallam at Deezer Research in Paris, released on GitHub in 2019 under the MIT license, and described in the Journal of Open Source Software (JOSS) on June 24, 2020.

Spleeter ships three pretrained configurations. The 2stems model separates vocals from accompaniment. The 4stems model follows the categories of the SiSEC 2018 evaluation campaign: vocals, drums, bass and other. The 5stems model adds a piano stem, and the authors described it as, to their knowledge, the first released model to perform that separation. Users install a Python package and run one command, for example spleeter separate -p spleeter:2stems -o output song.mp3. The tool is built on TensorFlow, Google's machine learning library, so it runs on either a CPU or a GPU, and it is also packaged for conda and Docker.

The method works on a spectrogram, a picture of sound that shows how much energy is present at each frequency over time. Spleeter computes a short-time Fourier transform (STFT), which slices the audio into overlapping frames and measures the frequencies in each. A U-Net, a convolutional neural network shaped like an encoder that compresses the input followed by a decoder that expands it again, with skip connections that pass detail across, looks at the mixture's magnitude spectrogram. The paper specifies 12-layer U-Nets, six encoder and six decoder layers. For each stem, a U-Net outputs a soft mask: a grid of values between 0 and 1 saying what share of the energy in each time-frequency cell belongs to that stem. Multiplying the mixture spectrogram by the mask gives an estimate of the stem, which is then converted back to audio. Because Spleeter normalizes the masks so that, in every cell, the shares for all stems add up to one, the separated stems sum back to approximately the original mix within the processed frequency range. Instead of plain masking, users can choose multichannel Wiener filtering, a classic statistical step that refines the estimates using all stems together.

Training used Deezer's internal datasets, including one called Bean, with the Adam optimizer and an L1 loss, the average absolute difference between the masked mixture spectrogram and the true stem spectrogram. Training took about a week on a single GPU. The training audio could not be published for copyright reasons, which is why Deezer released trained models instead: it was the only way to share the results.

Speed was a design goal. The JOSS paper reports that Spleeter separated the full MUSDB18 test set, about 3 hours and 27 minutes of audio, into four stems in under two minutes on a single GeForce RTX 2080 GPU, including about 15 seconds of model loading. That works out to more than 100 seconds of stereo audio processed per second. The authors note it still cannot run in real time on live input, because the model needs a buffer of audio to work on.

Quality is measured with signal-to-distortion ratio (SDR), in decibels, where higher means the estimate is closer to the true stem. On MUSDB18, the standard benchmark of 150 tracks (100 for training, 50 for testing), Spleeter was evaluated without being trained or tuned on that data. With Wiener filtering it reported vocal SDR of 6.86 dB, drums 6.71, bass 5.51 and other 4.55. In the same table, Open-Unmix scored 6.32 on vocals and the waveform model Demucs scored 7.05 on vocals and 6.70 on bass. The authors summarized Spleeter as competitive with Open-Unmix and almost on par with Demucs, at much higher speed.

Since then, separation quality has moved on. Hybrid Transformer Demucs, posted to arXiv in November 2022 by Simon Rouard, Francisco Massa and Alexandre Defossez, reported 9.20 dB average SDR on MUSDB HQ using extra training data, sparse attention and per-source fine-tuning. Models like this combine waveform and spectrogram processing and are larger and slower. Spleeter's appeal today is mainly its simplicity, speed, permissive license and long track record as a baseline.

The limitations are concrete. Masking cannot create sound that is hidden under another instrument, so stems contain bleed, where traces of other instruments leak through, and artifacts such as watery or phasey textures. The other and piano stems score lowest, because they lump together many instruments. Spleeter's default configurations only process the lower part of the spectrum (see the technical section), so high-frequency detail can be lost. MUSDB18 itself is limited to academic use and contains mostly Western popular music, so benchmark numbers may not reflect results on other genres. And separating a commercial recording does not change who owns its copyright.

04Működő példa

Separating a song into four stems and checking the result

  1. Install the package with pip install spleeter in a Python environment, and pick a short stereo song you have the right to use.

  2. Run spleeter separate -p spleeter:4stems -o output song.wav to produce vocals, drums, bass and other files.

  3. Open the four output files in an audio editor and play each one alone, noting where other instruments bleed through.

  4. Import all four stems on parallel tracks, sum them, and invert the polarity of the original mix on another track to compare.

  5. Look at a spectrogram of the vocal stem and confirm that little energy appears above about 11 kHz.

  6. Repeat with the 2stems model and compare the accompaniment file with the sum of drums, bass and other.

Mit mutat

Four WAV stems whose sum closely matches the original song, with audible bleed in some stems and a visible cutoff near 11 kHz. The polarity-inverted comparison should leave mostly quiet residue, which shows how much of the mix the masks redistributed rather than lost.

Valós megvalósítás

Making a karaoke or practice track by running the 2stems model, which writes vocals.wav and accompaniment.wav from a single command such as spleeter separate -p spleeter:2stems.

Remixing inside commercial audio editors: the Spleeter README lists integrations including iZotope RX 8 Music Rebalance, Steinberg SpectraLayers 7 Unmix and Acon Digital Acoustica 7.

Live DJ stem control, where VirtualDJ stem isolation and Algoriddim's NeuralMix in djay are listed by the project as built on or integrating Spleeter.

Music information retrieval research, where Deezer's paper suggests isolated vocals or drums can help lyrics alignment, chord and drum transcription, singer identification and cover detection.

Batch-processing a large catalog: the JOSS paper reports separating the roughly 3.5-hour MUSDB18 test set into 4 stems in under 2 minutes on one RTX 2080 GPU.

A Spleeter szárleválasztás jövője

Open problems in music source separation are mostly about data and fidelity. Deezer's 2020 paper notes that the most widely used benchmark, MUSDB18, is often the only dataset on which separation is reported, and it cannot be redistributed for commercial work, so results on other genres, recording styles and instruments remain less well measured. The Hybrid Transformer Demucs paper (November 2022) reported that its transformer model performed poorly when trained only on MUSDB and needed 800 extra songs to excel, which points to training data as a central constraint. Research since Spleeter has moved toward models that work on raw waveforms or combine waveform and spectrogram branches, toward more stems than the classic four, and toward reducing bleed and artifacts that masking alone cannot fix. The Spleeter repository itself said it might later release models with more than five stems. Questions that remain open include how to separate overlapping instruments in the same register, how to run high-quality separation on live audio with low delay, and how to evaluate quality in ways that match what listeners hear rather than SDR alone.

A gyakorlatba ültetés

Miért számít

  • Hozzáférés és elérés. Javítja a hozzáférhetőséget az átírás, a narráció és a hangfelületek révén.
  • Költség és költségvetés. A médiacsapatok kisebb költségvetéssel gyorsabban szállíthatják a csiszolt hanganyagot.
  • Sebesség és méretarány. Az ügyfélközpontú rendszerek nagyobb léptékben képesek feldolgozni a beszélt interakciókat.

Vigyázz

  • A beleegyezés hiányában nő a hanggal való visszaélés és a megszemélyesítés kockázata.
  • A pontosság csökkenhet az akcentusok, dialektusok vagy zajos környezetben.
  • A szintetikus hang összetéveszthető a hiteles beszéddel egyértelmű címkézés nélkül.

Következő lépések

  1. Kérjen kifejezett hozzájárulást a hangrögzítéshez, klónozáshoz és újrafelhasználáshoz.
  2. Tesztelje a minőséget különféle hangszórókon és háttérviszonyok között.
  3. Határozza meg, mikor kell egy embernek felülvizsgálnia vagy jóváhagynia a kimeneteket.
  4. Címkézze fel a szintetikus hanganyagot, és vezessen származási nyilvántartást az elszámoltathatóság érdekében.

Források és további olvasmányok

  1. Journal of Open Source SoftwareSpleeter: a fast and efficient music source separation tool with pre-trained models
  2. Deezer Research (GitHub)deezer/spleeter: Deezer source separation library including pretrained models
  3. Deezer Research (GitHub)Spleeter wiki: Models
  4. Deezer Research (GitHub)Spleeter 4stems base configuration
  5. SigSepMUSDB18 dataset
  6. arXivHybrid Transformers for Music Source Separation
  7. Deezer Research (GitHub)spleeter/model/__init__.py (mask computation)

Folytassa a felfedezést

Ingyenes hírlevél

Szerezze meg a napi AI összefoglalót

Minden hétköznap reggel három ellenőrzött AI történet, egyszerű angol nyelven. Örökké ingyen, hirdetések nélkül.

Minden hétköznap egy e-mail. Egy kattintással lemondhat. Soha nem értékesítjük vagy osztjuk meg az Ön címét.

Teszteld magad

Take the Spleeter Stem Separation quiz

Azonnali visszajelzés minden válaszra, és megosztható tanúsítvány egy ellenőrizhető azonosítóval, miután sikeresen teljesít egy kurzust.

Kezdő kvíz

Támogassa az ingyenes AI oktatást. AI Understanding egy 501(c)(3) nonprofit – nincs hirdetés, nincs fizetőfal, soha. Tegyen adományt

Gyakran ismételt kérdések

What is Spleeter Stem Separation?

Spleeter is an open-source music source separation tool from Deezer Research that splits a mixed song into 2, 4 or 5 stems, such as vocals, drums, bass and piano, using pretrained U-Net models. It matters because it made near state-of-the-art separation free, fast and one command away, and it still serves as a common baseline in research and audio software.

Melyik cég adta ki eredetileg a Spleeter-t?

A Spleeter-t nyílt forráskódúként 2019-ben adta ki a Deezer, a francia zenei streaming cég.

Mire választja szét a hangot a Spleeter 4 szárú modellje?

A 4 szárú konfiguráció éneket, dobot, basszust és egy „egyéb” szárat ad ki minden máshoz.

Milyen neurális hálózati architektúrát használ a Spleeter?

A Spleeter U-Net konvolúciós hálózatokat használ, amelyek előrejelzik a maszkokat a hang spektrogramján.

Hogyan von ki a Spleeter egyetlen forrást a keverékből?

A hálózat megjósol egy forrásonkénti maszkot (0 és 1 közötti értékek), amelyet megszoroz a keverék spektrogramjával.

What did Deezer report about the speed of Spleeter’s pretrained 4-stem model?

Deezer reported roughly 100-times-real-time throughput for the 4-stem model on one GPU, while explaining that buffering is needed, so this does not mean live real-time operation.