SeterusnyaPanduan seterusnya
Pemisahan Masa-Domain Conv-TasNet
AI Audio
PANDUAN AI Audio
Spleeter ialah alat pemisahan sumber muzik sumber terbuka dari Deezer Research yang memisahkan lagu campuran kepada 2, 4 atau 5 stem, seperti vokal, drum, bass dan piano, menggunakan model U‑Net pra‑latih.
Dengar panduan iniAI-diceritakan · ~7 min
Ia penting kerana ia menjadikan pemisahan hampir setaraf dengan keadaan seni menjadi percuma, pantas dan hanya satu arahan, dan ia masih berfungsi sebagai asas umum dalam penyelidikan dan perisian audio.
Music source separation is the task of taking a finished, mixed recording and estimating the individual parts that went into it. Those parts are called stems: a vocal stem, a drum stem, a bass stem, and so on. Separation is hard because the mix is a sum of sounds that overlap in time and frequency, and the original multitrack session is usually unavailable. Spleeter is a tool that does this with pretrained neural networks. It was built by Romain Hennequin, Anis Khlif, Felix Voituret and Manuel Moussallam at Deezer Research in Paris, released on GitHub in 2019 under the MIT license, and described in the Journal of Open Source Software (JOSS) on June 24, 2020.
Spleeter ships three pretrained configurations. The 2stems model separates vocals from accompaniment. The 4stems model follows the categories of the SiSEC 2018 evaluation campaign: vocals, drums, bass and other. The 5stems model adds a piano stem, and the authors described it as, to their knowledge, the first released model to perform that separation. Users install a Python package and run one command, for example spleeter separate -p spleeter:2stems -o output song.mp3. The tool is built on TensorFlow, Google's machine learning library, so it runs on either a CPU or a GPU, and it is also packaged for conda and Docker.
The method works on a spectrogram, a picture of sound that shows how much energy is present at each frequency over time. Spleeter computes a short-time Fourier transform (STFT), which slices the audio into overlapping frames and measures the frequencies in each. A U-Net, a convolutional neural network shaped like an encoder that compresses the input followed by a decoder that expands it again, with skip connections that pass detail across, looks at the mixture's magnitude spectrogram. The paper specifies 12-layer U-Nets, six encoder and six decoder layers. For each stem, a U-Net outputs a soft mask: a grid of values between 0 and 1 saying what share of the energy in each time-frequency cell belongs to that stem. Multiplying the mixture spectrogram by the mask gives an estimate of the stem, which is then converted back to audio. Because Spleeter normalizes the masks so that, in every cell, the shares for all stems add up to one, the separated stems sum back to approximately the original mix within the processed frequency range. Instead of plain masking, users can choose multichannel Wiener filtering, a classic statistical step that refines the estimates using all stems together.
Training used Deezer's internal datasets, including one called Bean, with the Adam optimizer and an L1 loss, the average absolute difference between the masked mixture spectrogram and the true stem spectrogram. Training took about a week on a single GPU. The training audio could not be published for copyright reasons, which is why Deezer released trained models instead: it was the only way to share the results.
Speed was a design goal. The JOSS paper reports that Spleeter separated the full MUSDB18 test set, about 3 hours and 27 minutes of audio, into four stems in under two minutes on a single GeForce RTX 2080 GPU, including about 15 seconds of model loading. That works out to more than 100 seconds of stereo audio processed per second. The authors note it still cannot run in real time on live input, because the model needs a buffer of audio to work on.
Quality is measured with signal-to-distortion ratio (SDR), in decibels, where higher means the estimate is closer to the true stem. On MUSDB18, the standard benchmark of 150 tracks (100 for training, 50 for testing), Spleeter was evaluated without being trained or tuned on that data. With Wiener filtering it reported vocal SDR of 6.86 dB, drums 6.71, bass 5.51 and other 4.55. In the same table, Open-Unmix scored 6.32 on vocals and the waveform model Demucs scored 7.05 on vocals and 6.70 on bass. The authors summarized Spleeter as competitive with Open-Unmix and almost on par with Demucs, at much higher speed.
Since then, separation quality has moved on. Hybrid Transformer Demucs, posted to arXiv in November 2022 by Simon Rouard, Francisco Massa and Alexandre Defossez, reported 9.20 dB average SDR on MUSDB HQ using extra training data, sparse attention and per-source fine-tuning. Models like this combine waveform and spectrogram processing and are larger and slower. Spleeter's appeal today is mainly its simplicity, speed, permissive license and long track record as a baseline.
The limitations are concrete. Masking cannot create sound that is hidden under another instrument, so stems contain bleed, where traces of other instruments leak through, and artifacts such as watery or phasey textures. The other and piano stems score lowest, because they lump together many instruments. Spleeter's default configurations only process the lower part of the spectrum (see the technical section), so high-frequency detail can be lost. MUSDB18 itself is limited to academic use and contains mostly Western popular music, so benchmark numbers may not reflect results on other genres. And separating a commercial recording does not change who owns its copyright.
04Contoh yang dikerjakan
Pasang pakej dengan pip install spleeter dalam persekitaran Python, dan pilih lagu stereo pendek yang anda ada hak untuk gunakan.
Jalankan spleeter separate -p spleeter:4stems -o output song.wav untuk menghasilkan vokal, drum, bass dan fail lain.
Buka empat fail output dalam penyunting audio dan mainkan setiap satu secara berasingan, sambil mencatat di mana instrumen lain bocor masuk.
Import semua empat stem ke dalam trek selari, jumlahkan mereka, dan songsangkan polariti campuran asal pada trek lain untuk dibandingkan.
Lihat spektrogram stem vokal dan sahkan bahawa tenaga sedikit muncul di atas kira-kira 11 kHz.
Ulangi dengan model 2stems dan bandingkan fail iringan dengan jumlah drum, bass dan lain-lain.
Apa yang ditunjukkan
Empat stem WAV yang jumlahnya hampir sepadan dengan lagu asal, dengan kebocoran yang dapat didengar pada beberapa stem dan pemotongan yang kelihatan berhampiran 11 kHz. Perbandingan dengan polariti songsang sepatutnya meninggalkan sisa yang kebanyakannya senyap, yang menunjukkan berapa banyak campuran yang dipindahkan semula oleh topeng berbanding yang hilang.
Membuat trek karaoke atau latihan dengan menjalankan model 2stems, yang menulis vocals.wav dan accompaniment.wav melalui satu arahan seperti spleeter separate -p spleeter:2stems.
Menyusun semula dalam penyunting audio komersial: README Spleeter menyenaraikan integrasi termasuk iZotope RX 8 Music Rebalance, Steinberg SpectraLayers 7 Unmix dan Acon Digital Acoustica 7.
Kawalan stem DJ secara langsung, di mana pengasingan stem VirtualDJ dan NeuralMix Algoriddim dalam djay disenaraikan oleh projek sebagai dibina atas atau mengintegrasikan Spleeter.
Penyelidikan pemulihan maklumat muzik, di mana kertas kerja Deezer mencadangkan vokal atau drum yang terasing dapat membantu penjajaran lirik, transkripsi kord dan drum, pengenalan penyanyi serta pengesanan tiruan.
Pemprosesan batch katalog besar: kertas JOSS melaporkan memisahkan set ujian MUSDB18 kira-kira 3.5 jam menjadi 4 stem dalam kurang daripada 2 minit pada satu GPU RTX 2080.
Open problems in music source separation are mostly about data and fidelity. Deezer's 2020 paper notes that the most widely used benchmark, MUSDB18, is often the only dataset on which separation is reported, and it cannot be redistributed for commercial work, so results on other genres, recording styles and instruments remain less well measured. The Hybrid Transformer Demucs paper (November 2022) reported that its transformer model performed poorly when trained only on MUSDB and needed 800 extra songs to excel, which points to training data as a central constraint. Research since Spleeter has moved toward models that work on raw waveforms or combine waveform and spectrogram branches, toward more stems than the classic four, and toward reducing bleed and artifacts that masking alone cannot fix. The Spleeter repository itself said it might later release models with more than five stems. Questions that remain open include how to separate overlapping instruments in the same register, how to run high-quality separation on live audio with low delay, and how to evaluate quality in ways that match what listeners hear rather than SDR alone.
Newsletter percuma
Tiga cerita AI yang disahkan setiap pagi hari bekerja, ditulis dalam Bahasa Inggeris biasa. Percuma selamanya, tiada iklan.
Satu e-mel setiap hari bekerja. Berhenti melanggan dalam satu klik. Kami tidak pernah menjual atau berkongsi alamat anda.
Uji diri anda
Maklum balas segera pada setiap jawapan, dan sijil yang boleh dikongsi dengan ID yang boleh disahkan sebaik sahaja anda lulus kursus.
Sokong pendidikan AI percuma. AI Understanding ialah pertubuhan bukan keuntungan 501(c)(3) — tiada iklan, tiada bayaran dinding, selamanya. Buat derma
Spleeter ialah alat pemisahan sumber muzik sumber terbuka daripada Deezer Research yang memisahkan lagu campuran menjadi 2, 4 atau 5 stem, seperti vokal, drum, bass dan piano, menggunakan model U-Net pra-latih. Ia penting kerana ia menjadikan pemisahan hampir setaraf dengan teknologi terkini menjadi percuma, pantas dan hanya satu arahan sahaja, dan ia masih berfungsi sebagai asas biasa dalam penyelidikan dan perisian audio.
Spleeter dikeluarkan sebagai sumber terbuka pada 2019 oleh Deezer, syarikat penstriman muzik Perancis.
Konfigurasi 4 batang mengeluarkan vokal, dram, bes dan batang 'lain' untuk semua yang lain.
Spleeter menggunakan rangkaian konvolusi U-Net yang meramalkan topeng pada spektrogram audio.
Rangkaian meramalkan topeng setiap sumber (nilai 0 hingga 1) yang didarab dengan spektrogram campuran.
Deezer melaporkan kadar aliran hampir 100 kali masa nyata untuk model 4-stem pada satu GPU, sambil menjelaskan bahawa penimbal diperlukan, jadi ini tidak bermakna operasi masa nyata secara langsung.
Teruskan belajar
Lebih banyak panduan dipilih untuk topik ini
SeterusnyaPanduan seterusnya
Pemisahan Masa-Domain Conv-TasNet
AI Audio