概述
它於 2002 年開始採用手工設計的功能和簡單的分類器,後來轉向聲譜圖上的捲積網絡,現在依賴大型預訓練音頻模型。它支撐著音樂搜尋和組織,但流派標籤很模糊,流行的基準也存在已知的缺陷。
重點摘要
- 音樂流派分類按流派對音訊錄音進行標記,通常是將聲音轉換為梅爾頻譜圖並將其輸入經過訓練的分類器。
- Tzanetakis 和 Cook 2002 年的系統使用手工製作的音色、節奏和音高特徵,在 10 種流派上達到了 61% 的準確率。
- GTZAN 的 1,000 個 30 秒剪輯基準包含精確重複、錯誤標籤和大量藝術家重複,這可能會誇大報告的準確性。
- 對大約 190 萬個 AudioSet 片段進行預訓練的 PANN 微調將 GTZAN 準確率從從頭開始訓練時的 75.8% 提高到 91.5%。
- 流派標籤是主觀的且特定於文化的,因此多標籤標籤、層次結構和藝術家過濾的評估比單一標籤提供更值得信賴的結果。
深入探討
Music genre classification takes an audio file and assigns it one or more genre labels such as classical, country, disco, hip-hop, jazz, metal, pop, reggae, rock or blues. It is one of the oldest tasks in music information retrieval (MIR), the research field that builds software to search, organize and analyze music. The task sounds simple but has a philosophical problem at its core: genres are, as Tzanetakis and Cook wrote in 2002, labels created by humans with no strict definitions or boundaries, shaped by marketing, history and culture. A classifier can only learn whatever regularities its training labels happen to encode.
Every system follows the same basic pipeline. First, audio is converted to a standard format, often mono at 22,050 or 16,000 samples per second, and cut into excerpts, commonly 30 seconds. Second, the audio is turned into a representation that exposes musical structure. The most common is a mel-spectrogram, an image-like grid showing how much energy falls in each frequency band over time, with bands spaced to match human pitch perception. Third, a model maps that representation to scores for each genre. Fourth, predictions from many short windows are averaged to label the whole track. Systems differ mainly in step three: whether the features are designed by hand or learned.
The classic hand-crafted approach was set out by George Tzanetakis and Perry Cook in IEEE Transactions on Speech and Audio Processing in July 2002. They computed three families of features. Timbral texture features describe the sound's color: the spectral centroid (where the center of mass of the spectrum lies, which correlates with brightness), spectral rolloff, spectral flux (how quickly the spectrum changes), zero-crossing rate (a rough measure of noisiness) and the first five mel-frequency cepstral coefficients (MFCCs, a compact summary of the spectral envelope borrowed from speech recognition). Rhythmic features came from a beat histogram covering 40 to 200 beats per minute. Pitch features came from a histogram of detected pitches. Simple statistical classifiers, including Gaussian mixture models and k-nearest neighbors, then labeled the clips, reaching 61% accuracy on ten genres.
Their dataset of 1,000 thirty-second clips, 100 for each of ten genres, recorded at 22,050 Hz from radio, CDs and MP3 files, became known as GTZAN and the standard benchmark. In 2013, Bob Sturm counted at least 100 published works that used it and documented serious faults: 50 exact repeated excerpts, 21 further repetitions of the same recording, mislabeled clips and distorted audio. Artist repetition is heavy; the 100 blues excerpts come from only nine artists, and more than a third of the reggae excerpts are by Bob Marley. When the same artist appears in both training and test sets, a model can score well by recognizing the artist or recording rather than the genre, which inflates reported accuracy.
Deep learning shifted the field around 2016. Keunwoo Choi, George Fazekas and Mark Sandler showed at ISMIR 2016 that fully convolutional networks reading mel-spectrograms reached state-of-the-art music tagging on the MagnaTagATune dataset, and that deeper networks did better on the larger Million Song Dataset, concluding that mel-spectrograms are an effective input. Convolutional networks learn their own filters for patterns such as drum hits, harmonic textures and vocal styles, removing the need to design features by hand. Larger datasets followed, notably FMA in 2017 with 106,574 tracks and 161 genres arranged in a hierarchy, with Creative Commons audio that researchers can share.
The current approach is transfer learning from large pretrained models. PANNs (Kong and colleagues, 2020) trained convolutional networks on AudioSet, a collection of about 1.9 million YouTube clips labeled with 527 sound classes, then fine-tuned them on smaller tasks. On GTZAN, the fine-tuned network scored 91.5% accuracy against 75.8% when trained from scratch, while a specialized prior system had reported 93.9%. MERT (ICLR 2024) went further with self-supervised pretraining on music audio, learning to predict hidden targets from an acoustic codec and a constant-Q transform, a pitch-aligned spectrogram, and was evaluated across 14 tasks including genre classification.
Compared with the alternatives, audio-based classification has distinct strengths and weaknesses. Metadata-based approaches use artist, label or user tags; they are cheap and often accurate for well-known music but fail for new uploads with no history. Collaborative filtering infers similarity from listening behavior rather than sound, which captures cultural context that audio cannot. Lyrics-based models help for genres defined by content or language. Audio classifiers work on any recording, including brand-new or unreleased tracks, which is why they are useful for cold-start tagging, but they see only the sound and miss context.
The limitations are significant. Genre boundaries are subjective, overlapping and culture-specific, so a single label per track is often wrong by design; FMA's hierarchy and multi-label tagging are partial answers. Benchmark numbers can mislead: Sturm showed that GTZAN's repetitions and mislabelings affect different systems differently, so accuracy rankings on it are not reliable without artist-filtered splits. GTZAN's ten categories are all Western popular and classical styles, so models trained on it say nothing about most of the world's music. Models can latch onto production artifacts, such as recording era or audio compression, instead of musical traits. In practice, fine-tuning a pretrained model on a dataset the size of GTZAN is a small job, while pretraining models such as PANNs on about 1.9 million clips, or MERT at up to 330 million parameters, requires far larger compute.
04工作範例
建構一個具有藝術家意識分割的小型流派分類器
下載 FMA 小子集,其中包含流派標籤和預先定義的訓練、驗證和測試拆分。
將每個 30 秒剪輯重新取樣為 22,050 Hz 單聲道,並使用 librosa 計算 128 頻段對數梅爾頻譜圖。
作為基線,計算譜質心、滾降、過零率和前 13 個 MFCC 的平均值和方差,然後訓練邏輯回歸分類器。
在梅爾頻譜圖上訓練小型捲積網絡,或使用新的流派輸出層微調預先訓練的音訊模型,例如 PANN 檢查點。
對測試拆分進行評估,並列印一個混淆矩陣,顯示哪些流派與哪些流派混淆。
檢查訓練集和測試集中是否都沒有藝術家出現,並將結果與隨機分割進行比較,以了解有多少藝術家重疊會誇大準確性。
它顯示了什麼
具有測試精度和混淆矩陣的兩個模型。期望學習或預訓練的模型能夠超越手工製作的基線,並期望音樂上相鄰的流派(例如搖滾和流行)之間出現混淆;隨機分割和藝術家過濾的準確性之間的巨大差距表明該模型正在識別藝術家,而不是流派。
現實世界的實施
Tzanetakis 和 Cook 的 2002 年系統將音色、節奏和音高特徵與高斯混合分類器相結合,在 10 個流派中達到了 61% 的準確率,作者稱其與報告的人類結果相當。
在 ISMIR 2017 上發布的免費音樂檔案 (FMA) 資料集提供了來自 16,341 名藝術家的 106,574 首知識共享曲目,標記為 161 個流派的層次結構,用於訓練和基準測試。
PANN 是在大約 190 萬個 AudioSet 剪輯上進行預訓練的音頻網絡,經過微調後在十種類型的 GTZAN 基準上達到了 91.5% 的準確率,而從頭開始訓練的同一網絡的準確率則為 75.8%。
MERT (ICLR 2024) 是一種具有 9500 萬至 3.3 億個參數的自監督音樂模型,在 14 項音樂理解任務上進行了評估,包括 GTZAN 流派分類、標記、調檢測和節拍追蹤。
音樂庫和串流媒體目錄可以使用流派分類器為未標記的上傳建議標籤,組織檔案和種子推薦,這是 Tzanetakis 和 Cook 預計用於大型線上音樂收藏的用途。
音樂流派分類的未來
Research is moving from single-task genre classifiers toward general music foundation models evaluated on many tasks at once, as with MERT (ICLR 2024), which reports results across 14 music understanding tasks. Models that embed audio and text in a shared space allow free-text queries such as describing a mood or style rather than picking from a fixed genre list, which sidesteps some of the rigidity of fixed taxonomies. The evaluation problems identified by Sturm in 2013 remain relevant: GTZAN is still widely reported, and its repetitions and mislabelings mean small accuracy differences are hard to interpret. Larger and better-documented datasets such as FMA (2017) help, but labeling remains culturally specific. Open questions include how to represent hierarchical and overlapping genres, how to evaluate models fairly across cultures, how to avoid shortcut learning from production artifacts, and how to classify music that was itself generated by AI, whose audio statistics may differ from the recordings models were trained on. None of these has a settled answer.
付諸實踐
為什麼這很重要
- 交通與覆蓋範圍. 它透過轉錄、旁白和語音介面提高了可訪問性。
- 成本與預算. 媒體團隊可以用更少的預算更快地交付精美的音訊。
- 速度與規模. 面向客戶的系統可以處理更大規模的語音互動。
留意
- 如果未徵得同意,語音濫用和冒充風險就會增加。
- 由於口音、方言或嘈雜的環境,準確性可能會下降。
- 如果沒有明確的標籤,合成音訊可能會被誤認為是真實的語音。
後續步驟
- 獲得語音捕獲、克隆和重用的明確同意。
- 測試不同揚聲器和背景條件下的品質。
- 定義人員必須審查或批准輸出的時間。
- 標記合成音訊並保留來源記錄以供問責。
資料來源與延伸閱讀
- IEEE Transactions on Speech and Audio Processing (2002)音訊訊號的音樂類型分類
- arXivGTZAN 資料集:它的內容、它的缺陷、它們對評估的影響以及它的未來使用
- arXiv (ISMIR 2017)FMA:音樂分析資料集
- arXiv (ISMIR 2016)使用深度卷積神經網路自動標記
- arXivPANNs:用於音訊模式識別的大規模預訓練音訊神經網路
- arXiv (ICLR 2024)MERT:大規模自監督訓練的原聲音樂理解模型
不斷探索
常見問題
什麼是音樂流派分類?
音樂流派分類是根據音訊自動將錄音標記為搖滾、爵士、嘻哈或其他流派的任務。它於 2002 年開始採用手工設計的功能和簡單的分類器,後來轉向聲譜圖上的捲積網絡,現在依賴大型預訓練音頻模型。它支撐著音樂搜尋和組織,但流派標籤很模糊,流行的基準也存在已知的缺陷。
什麼是梅爾譜圖,通常用作流派分類神經網路的輸入?
梅爾頻譜圖表示不同頻率的能量如何隨時間變化,頻率軸扭曲到與人類音調感知相符的梅爾標度。
哪個經典基準資料集包含 10 個流派的 1,000 個 30 秒剪輯?
GTZAN 由 George Tzanetakis 組建,長期以來一直是標準的流派分類基準,儘管它現在因錯誤標籤和重複藝術家而受到批評。
為什麼要達到完美的流派分類準確度從根本上來說是困難的?
流派是一個文化上的、模糊的概念;歌曲融合了不同的風格,而人們本身對標籤的看法也不一致,因此通常不存在單一的正確答案。
哪一種手工製作的特徵在早期流派分類系統中被廣泛使用?
MFCC 捕捉音訊的音色、頻譜形狀,是深度學習之前 SVM 等分類器的主要功能。
流派分類器末端的 softmax 層會產生什麼?
Softmax 將網路的原始分數轉換為機率,所有流派類別的總和為 1,並選擇最高的作為預測。