音訊人工智慧指南

聲學場景分類

聲學場景分類(ASC)是僅使用聲音來識別錄音來自哪種場所或情境(例如公車、公園或繁忙街道)的任務。

聆聽此指南AI 語音敘述 · ~8 min

本頁閱讀時間 9 分鐘
  1. 概述
  2. 重點摘要
  3. 深入探討
  4. 在公開資料集上訓練小型場景分類器
  5. 現實世界的實施
  6. 聲學場景分類的未來
  7. 付諸實踐
  8. 資料來源與延伸閱讀
  9. 不斷探索
  10. 常見問題

概述

它讓助聽器和手機等裝置能適應周遭環境,並且自 2010 年代中期以來已透過 DCASE 挑戰賽進行公開基準測試。

重點摘要

  1. 聲學場景分類將整個錄音標記為其所處的環境,例如公車、公園或辦公室。
  2. 大多數現代系統將音訊轉換為對數梅爾頻譜圖,並使用卷積神經網路進行分類。
  3. 在 DCASE 2016 中,48 個系統中的最佳系統在 15 個場景的開發測試中達到了 89.7% 的準確率,而 MFCC 和 GMM 的基準測試為 72.5%。
  4. DCASE 2024 任務將模型限制在 128 kB 的參數以及每秒音訊 3000 萬次乘積累加(MACs),以模擬小型裝置。
  5. Phonak 的 AutoSense OS 在助聽器中使用場景分類,從六個聲音類別中自動選擇聆聽程式。

深入探討

Acoustic scene classification (ASC) asks a computer to name the environment a recording was made in, such as a metro station, an office, a shopping mall or a park, from its sound alone. Barchiesi, Giannoulis, Stowell and Plumbley, in a 2015 review in IEEE Signal Processing Magazine, define it simply as classifying environments from the sounds they produce. It differs from sound event detection, which locates individual sounds like a door slam or a dog bark in time. A scene label describes the whole clip, so the model must summarize the overall texture of many overlapping sounds: traffic hum, footsteps, announcements, birdsong and reverberation.

A typical modern system works in four steps. First, the audio is cut into fixed-length clips, for example 1 second in the 2024 DCASE challenge or 30 seconds in the 2016 edition. Second, each clip is converted into a time-frequency picture, usually a log-mel spectrogram, which shows how much energy falls into perceptually spaced frequency bands over time. Third, a classifier, today usually a convolutional neural network (CNN) that scans the spectrogram like an image, produces a score for each scene class. Fourth, the highest score becomes the label, sometimes after averaging over several clips to smooth out brief misleading sounds.

Earlier systems used hand-built features. The benchmark method in the 2015 review, and the baseline in the DCASE 2016 challenge, computed mel-frequency cepstral coefficients (MFCCs), compact summaries of the spectrum's shape, and modeled each scene with a Gaussian mixture model (GMM), choosing the scene whose model gave the input the highest likelihood. In the challenge the review describes, only three submitted algorithms significantly outperformed that baseline, and the best one matched the median accuracy of human listeners. Some scene pairs confused both people and machines.

The field's main milestones track the DCASE (Detection and Classification of Acoustic Scenes and Events) challenges. The 2016 acoustic scene task used the TUT Acoustic Scenes 2016 dataset: 15 scenes including bus, cafe or restaurant, car, city center, forest path, grocery store, home, library, metro station, office and urban park, recorded with binaural in-ear microphones at 44.1 kHz and 24 bits and split into 30-second segments. Its MFCC and GMM baseline reached 72.5% accuracy in development cross-validation, and the best of 48 submitted systems reached 89.7% on the evaluation set. Later editions shifted toward real-world constraints. By 2024 the task used the TAU Urban Acoustic Scenes 2022 Mobile dataset, with 1-second clips from 3 real and 6 simulated recording devices in development, extra unseen devices in evaluation, a 128 kB limit on model size and a 30 million multiply-accumulate limit per second of audio. Teams also had to train on subsets as small as 5% of the data, and the baseline's accuracy ranged from 42.40% at 5% to 56.99% with all the data.

ASC sits beside several neighboring approaches. Sound event detection gives finer detail but needs time-stamped labels that are costly to produce. General audio taggers trained on large collections, such as the PANNs models trained on AudioSet, recognize hundreds of sound types and can be fine-tuned for scenes; the PANNs authors report a mean average precision of 0.439 on AudioSet tagging and transfer to acoustic scene classification among other tasks. Location data such as GPS can say where a device is but not whether it is in a quiet car or a noisy one, which is why hearing aids rely on sound. Phonak's AutoSense OS, for example, uses classical machine learning trained on audio labeled by audiology engineers to pick among six sound classes.

Hearing aids show how a scene label is used in practice. Sonova's fact sheet describes AutoSense OS as a combination of supervised and unsupervised learning, trained on its own media database, whose output selects the hearing program best suited to the current sound, with no input from the wearer. The fact sheet names the main risk plainly: the classifier may choose a program that is not the best one, though the wearer still receives amplification. That framing is typical of scene classification in products, where a wrong label degrades comfort or clarity rather than causing outright failure, so designers trade accuracy against power, memory and response time.

The hard problems are well known from the challenges. Device mismatch is the biggest: a model trained on one microphone performs worse on another, which is why DCASE evaluates on unseen devices. Scenes overlap, so a street with a tram passing may sound like a tram, and an indoor market may sound like a shopping mall. Short clips carry less evidence, and the 2024 baseline's accuracy below 57% on 1-second clips shows how hard the setting is. Class lists are arbitrary and culture-specific, since a metro station in Helsinki may not sound like one in Mumbai. Finally, recordings of public places can capture conversations, so collecting data raises privacy questions.

Deployment costs vary with the target. On a server, a large pretrained network can classify scenes cheaply. On a hearing aid or earbud, power and memory are tight, which is exactly what DCASE's complexity limits simulate: 128 kB holds only about 128,000 parameters in 8-bit integers, and the 2024 baseline used 61,148 parameters and about 29.4 million multiply-accumulates per inference. Training a competitive model needs labeled recordings from many places and devices; the 2024 development training split alone contained 139,620 one-second segments.

04工作範例

在公開資料集上訓練小型場景分類器

  1. 下載公開的 DCASE 開發資料集,例如 TAU Urban Acoustic Scenes 2022 Mobile,並注意其訓練與測試分割中出現的裝置。

  2. 將每個 1 秒片段轉換為對數梅爾頻譜圖,例如使用 256 個頻帶,以符合 DCASE 2024 基準的形狀。

  3. 訓練一個小型卷積網路,包含幾個卷積塊、全局平均池化和 10 類 softmax,使用交叉熵損失。

  4. 計算參數數量與乘加運算,並將其與 DCASE 2024 使用的 128 kB 與 3,000 萬 MAC 限制進行比較。

  5. 報告測試分割的宏平均準確率,然後分別報告從未在訓練中見過的裝置的片段準確率。

  6. 加入 Frequency-MixStyle 與頻率遮蔽,重新訓練,並比較未見裝置的準確率。

它顯示了什麼

學習者最終得到一個緊湊的場景分類器和兩個準確率數值,分別為總體與未見裝置。兩者之間的差距直接顯示裝置不匹配,與公開的 DCASE 2024 基準約 57% 的完整訓練資料進行比較,可作為合理性檢驗。

現實世界的實施

Phonak 的 AutoSense OS 分類器,內建於 Sonova 聽力助聽器中,將進入的聲音分為「安靜時語音」、「噪音時語音」、「車內」、「混響語音」、「噪音」與「音樂」,並利用結果選擇聽力程式,使用者不需要任何操作。

DCASE 2024 挑戰要求團隊將 1 秒片段分類為 10 個城市場景,例如機場、地鐵站與電車,模型參數上限為 128 kB,並限制每秒音訊的乘加運算不超過 3,000 萬。

TUT Acoustic Scenes 2016 資料集以雙耳麥克風於 44.1 kHz 於 15 個環境(從咖啡館到湖邊海灘)錄製,為研究人員提供了 30 秒片段的共享基準,用於比較場景分類器。

大型預訓練音訊網路,例如 PANNs,訓練於 Google 的 AudioSet,並被微調用於場景分類與其他音訊任務,PANNs 作者報告在多個基準上達到最先進的結果。

早期的基準測試將機器與人類進行了比較:Barchiesi 及其同事在 2015 年報告稱,最佳挑戰系統的平均準確率與人類聽眾在相同場景下的中位數準確率相當。

聲學場景分類的未來

The DCASE challenge series shows the direction of the field: from 30-second binaural clips in 2016 to 1-second clips, unseen recording devices, tight memory and compute caps and small training subsets in 2024. These constraints mirror what hearing aids, earbuds and phones need, and the 2024 task explicitly measured how accuracy falls when only 5% of the training data is available. Large pretrained audio models are the other trend. PANNs (2019 to 2020) showed that networks trained on AudioSet transfer to scene classification and other tasks, and the DCASE 2024 rules explicitly allow pretrained models such as PaSST and EfficientAT as teachers for small students that fit device budgets. Whether general-purpose audio models will replace task-specific scene classifiers, or mainly serve as teachers for them, remains open. Open problems include robustness to new microphones and cities, label sets that reflect real users rather than one dataset's ten or fifteen classes, fair performance across regions, and privacy-preserving data collection and on-device processing. Manufacturer claims about product classifiers, such as those in hearing aids, are mostly published as white papers and fact sheets, and independent evaluations remain limited.

付諸實踐

為什麼這很重要

  • 交通與覆蓋範圍. 它透過轉錄、旁白和語音介面提高了可訪問性。
  • 成本與預算. 媒體團隊可以用更少的預算更快地交付精美的音訊。
  • 速度與規模. 面向客戶的系統可以處理更大規模的語音互動。

留意

  • 如果未徵得同意,語音濫用和冒充風險就會增加。
  • 由於口音、方言或嘈雜的環境,準確性可能會下降。
  • 如果沒有明確的標籤,合成音訊可能會被誤認為是真實的語音。

後續步驟

  1. 獲得語音捕獲、克隆和重用的明確同意。
  2. 測試不同揚聲器和背景條件下的品質。
  3. 定義人員必須審查或批准輸出的時間。
  4. 標記合成音訊並保留來源記錄以供問責。

資料來源與延伸閱讀

  1. arXiv (IEEE Signal Processing Magazine, 2015)聲學場景分類:根據聲音產生的環境對其進行分類
  2. DCASE CommunityDCASE 2016 任務 1:聲學場景分類
  3. DCASE CommunityDCASE 2024 任務 1:資料高效且低複雜度的聲學場景分類
  4. arXivPANNs:用於音訊模式識別的大規模預訓練音訊神經網路
  5. Sonova / Phonak機器學習概況表:聲音分類器 AutoSense OS

不斷探索

免費電子報

獲取每日 AI 簡報

每個工作日上午提供三則經過驗證的 AI 新聞,以淺顯易懂的英文撰寫。永久免費,無廣告。

每個工作日一封電子郵件。一鍵取消訂閱。我們絕不出售或分享您的地址。

測試一下自己

參加聲學場景分類測驗

每個答案皆有即時回饋,通過課程後可獲得具備可驗證 ID 的可分享證書。

開始測驗

支持免費的 AI 教育。AI Understanding 是一個 501(c)(3) 非營利組織——絕無廣告,絕無付費牆。 捐款

常見問題

什麼是聲學場景分類?

聲學場景分類 (ASC) 是一項僅利用聲音來識別錄音來源地點或情境(例如公車、公園或繁忙的街道)的任務。它能讓助聽器和手機等裝置適應周圍環境,自 2010 年代中期以來,已透過 DCASE 挑戰賽進行了公開的基準測試。

聲學場景分類與聲音事件偵測有何不同?

ASC 標記整個環境場景(例如「街道」、「公園」),而聲音事件偵測則定位單一聲音,例如警報器或樹皮聲。

ASC 中的「設備不符」問題是什麼?

不同的麥克風具有不同的頻率響應,因此在一台設備上訓練的模型通常會在另一台設備上的錄音中表現下降,這是 ASC 的關鍵挑戰。

哪一種輸入表示形式是 ASC 模型的標準?

與許多音訊 AI 一樣,ASC 將剪輯轉換為 CNN 或 Transformer 可以處理的 log-mel 頻譜圖。

為什麼 ASC 模型會在很多秒內而不是單一時刻匯集特徵?

場景的身份來自於其隨時間變化的整體紋理和氛圍,因此模型會聚合整個剪輯的資訊。

哪些研究挑戰在很大程度上推動了 ASC 的進步?

一年一度的 DCASE(聲學場景和事件的偵測和分類)挑戰賽是推動 ASC 前進的核心基準。