音訊人工智慧指南

音頻的恆定 Q 變換

The constant-Q transform (CQT) turns audio into a time-frequency picture whose frequency bins are spaced like musical notes, a fixed number per octave, instead of evenly in hertz like the Fourier transform.

聆聽此指南AI 語音敘述 · ~8 min

本頁閱讀時間 9 分鐘
  1. 概述
  2. 重點摘要
  3. 深入探討
  4. Find a note in a CQT with librosa
  5. 現實世界的實施
  6. 音頻恆定 Q 變換的未來
  7. 付諸實踐
  8. 資料來源與延伸閱讀
  9. 不斷探索
  10. 常見問題

概述

Because a change of key simply slides the picture up or down, the CQT is a standard input for music transcription, pitch tracking and chord recognition models.

重點摘要

  1. The constant-Q transform spaces frequency bins geometrically, a fixed number per octave, so each bin can line up with a musical note.
  2. Every CQT bin has the same ratio of center frequency to bandwidth, so low notes get long windows and high notes short ones.
  3. Transposing music shifts a CQT up or down without changing its shape, which suits convolutional neural networks.
  4. Judith C. Brown 於 1991 年在《美國聲學學會雜誌》上描述了音樂的恆定 Q 頻譜變換。
  5. Spotify 的 Basic Pitch 使用和聲 CQT 輸入將音訊轉錄為 MIDI,模型參數少於 17,000 個。

深入探討

A spectrogram shows how much energy a sound has at each frequency over time. The usual way to compute one is the short-time Fourier transform (STFT), which cuts audio into frames and measures energy in frequency bins spaced evenly in hertz, for example every 10.8 Hz. Music does not work in even steps. Each octave doubles the frequency, and Western tuning splits the octave into 12 semitones, each about 5.9% higher than the last. Near the bottom of a piano, adjacent notes C1 and C#1 are only 1.9 Hz apart, while near the top they are more than 200 Hz apart. An evenly spaced spectrogram therefore blurs low notes together and spends many bins between high notes. The constant-Q transform fixes this by spacing bins geometrically: every bin is a fixed ratio above the previous one, so each bin can sit exactly on a musical note.

The name comes from the quality factor Q, the ratio of a bin's center frequency to its bandwidth. In the CQT every bin has the same Q, so bandwidth grows in proportion to frequency. To measure a narrow band precisely you must listen for longer, so low bins use long analysis windows and high bins use short ones. Step by step, the computation is: choose a lowest frequency, such as C1 at 32.70 Hz; choose bins per octave, typically 12 for semitones or 36 to 60 for finer pitch detail; place each center frequency a factor of two to the power of one over the bins-per-octave above the last; give each bin a window whose length is Q times the sample rate divided by its center frequency; and at regular time steps, multiply the audio by each bin's windowed complex sinusoid and sum, which yields the magnitude and phase of that note at that moment.

The idea was formalized for music by Judith C. Brown in "Calculation of a Constant Q Spectral Transform", published in the Journal of the Acoustical Society of America in 1991 (volume 89, pages 425 to 434). Her MIT Media Lab page still hosts Matlab code for the direct method. Computing every bin with its own long window was slow, so later work focused on efficiency. Christian Schörkhuber and Anssi Klapuri's 2010 constant-Q toolbox, the method cited by librosa, computes the top octave and then repeatedly halves the sample rate to compute each lower octave cheaply. In 2011 Gino Angelo Velasco, Nicki Holighaus, Monika Dörfler and Thomas Grill showed at the DAFx conference how to build a perfectly invertible CQT using nonstationary Gabor frames, the basis of MATLAB's cqt and icqt functions.

Deep learning gave the CQT a new role. A convolutional neural network learns small patterns and slides them across an image. On a CQT, playing the same chord in a different key moves the whole pattern by a fixed number of bins without changing its shape, so one learned filter recognizes the chord in every key. On a linear spectrogram the spacing between harmonics changes with pitch, so that trick fails. In 2017 Rachel Bittner and colleagues at New York University introduced the harmonic CQT (HCQT), a stack of CQTs whose lowest frequencies are multiplied by 0.5, 1, 2, 3, 4 and 5, so a note's overtones line up in the same position across layers, like the color channels of an image. Spotify's Basic Pitch, released in June 2022, uses this input to transcribe polyphonic audio to MIDI with fewer than 17,000 parameters and under 20 MB of peak memory.

Compared with the alternatives, the CQT's strength is pitch alignment and its cost is everything else. The STFT is fast, simple and exactly invertible but has the wrong frequency layout for notes. The mel spectrogram, widely used for speech, compresses frequency on a perceptual scale that is nearly linear at low frequencies, so its bins do not fall on musical notes. Chroma features, often computed from a CQT, fold all octaves into 12 pitch classes, which helps chord and key detection but discards which octave a note is in. Learned front ends let a network discover its own filters, at the price of more data and less interpretability.

The CQT has real limitations. Long low-frequency windows smear time: with 12 bins per octave at a 22,050 Hz sample rate, the window for C1 is about 11,300 samples, roughly half a second, so fast bass notes and drum hits blur at the bottom of the picture while the top is sharp. Classical implementations were not invertible, which mattered for editing and source separation until frame-based methods like Velasco's appeared. Computation is heavier than an STFT; the nnAudio authors measured librosa taking about 103 seconds for 1,770 piano recordings on a CPU, against 0.26 seconds for their GPU version. Tuning matters too: a recording tuned slightly away from A at 440 Hz puts notes between bins unless the transform is shifted to match.

In practice the CQT is cheap to try. librosa computes it with one function call, with defaults of a 512-sample hop, 84 bins and 12 bins per octave from C1 nnAudio and similar libraries compute it on a GPU as a layer of the network, which avoids storing spectrograms on disk and makes the filters trainable. The main design choices are the lowest frequency, the number of octaves, bins per octave, which trades pitch precision against time precision, and hop size. The 2017 deep salience model, for example, used 60 bins per octave, 20 cents per bin, to track fine pitch, while tasks that only need note names can use 12.

04工作範例

Find a note in a CQT with librosa

  1. 產生或錄製 440 Hz 的 2 秒音調(中間 C 上方的 A),取樣頻率為 22,050 Hz。

  2. 計算 C = librosa.cqt(y, sr=22050),保留預設值:跳數 512,最低頻率 C1 為 32.70 Hz,84 個 bin,每倍頻程 12 個。

  3. 使用 numpy.abs(C) 取得振幅並使用 librosa.amplitude_to_db 轉換為分貝。

  4. 預測亮行:A4 比 C1 高 45 個半音(36 個半音到達 C4,加上 9 個半音到達 A4),因此預計 bin 索引為 45。

  5. 檢查 librosa.cqt_frequencies(84, fmin=librosa.note_to_hz('C1')),其條目 45 應約為 440 Hz。

  6. 以 466.16 Hz 音調 (A#4) 重複,並確認亮行恰好向上移動了一個 bin。

它顯示了什麼

一張 CQT 影像,在 bin 45 處有一條明亮的水平線,每半音移動一個 bin。檢查預測的箱子和觀察到的箱子是否一致確認了幾何間距,並且比較低音與高音的銳度顯示了時間頻率權衡。

現實世界的實施

Spotify 的開源 Basic Pitch 於 2022 年 6 月發布,使用諧波 CQT 輸入和參數少於 17,000 個的模型,將包括語音在內的許多樂器的錄音轉換為 MIDI。

紐約大學 (ISMIR 2017) 的深度顯著模型從 6 個八度音階的和聲 CQT 中估計旋律和多個音高,從 C1 (32.7 Hz) 開始,每個八度音階有 60 個 bin。

librosa Python 函式庫的 cqt 函數是音樂研究的常見起點,預設產生 84 個半音階,從 C1 向上七個八度。

nnAudio 工具箱在神經網路內的 GPU 上計算 CQT,其作者在一組 1,770 個鋼琴錄音上測得的速度比 librosa 快約 400 倍。

MATLAB 的 cqt 和 icqt 函數提供可逆 CQT,因此工程師可以在音符對齊域中編輯聲音,例如刪除音高,然後將其轉換回音訊。

音頻恆定 Q 變換的未來

Current research uses the CQT less as a final feature and more as a structured input or a layer inside networks. Since nnAudio (2019) made GPU and trainable CQT layers practical, models can start from note-aligned filters and fine-tune them during training. The harmonic CQT introduced at ISMIR 2017 and reused in Spotify's Basic Pitch in 2022 shows a continuing trend toward representations that encode musical structure so that smaller models suffice; Basic Pitch runs faster than real time on most modern computers according to Spotify. Open problems remain. The trade-off between time and frequency resolution at low frequencies is fundamental, so fast bass lines and percussion stay blurred unless combined with another representation. Non-Western music that uses intervals other than 12 equal semitones does not map neatly onto standard bin spacings, although finer bins and adjustable tuning partly address this. Invertible CQTs based on nonstationary Gabor frames, available in MATLAB and other toolboxes, make note-domain editing and separation possible, but how they compare with learned encoders for generative music models is still being studied rather than settled.

付諸實踐

為什麼這很重要

  • 交通與覆蓋範圍. 它透過轉錄、旁白和語音介面提高了可訪問性。
  • 成本與預算. 媒體團隊可以用更少的預算更快地交付精美的音訊。
  • 速度與規模. 面向客戶的系統可以處理更大規模的語音互動。

留意

  • 如果未徵得同意,語音濫用和冒充風險就會增加。
  • 由於口音、方言或嘈雜的環境,準確性可能會下降。
  • 如果沒有明確的標籤,合成音訊可能會被誤認為是真實的語音。

後續步驟

  1. 獲得語音捕獲、克隆和重用的明確同意。
  2. 測試不同揚聲器和背景條件下的品質。
  3. 定義人員必須審查或批准輸出的時間。
  4. 標記合成音訊並保留來源記錄以供問責。

資料來源與延伸閱讀

  1. MIT Media Lab, Judith C. Brown常數 Q 變換(Matlab 程式碼和參考 Brown 1991)
  2. librosa documentation庫檔.cqt
  3. DAFx 2011使用非平穩 Gabor 框架建構可逆常 Q 變換
  4. MathWorkscqt:恆定 Q 非平穩 Gabor 變換
  5. ISMIR 2017和弦音樂中 F0 估計的深度顯著表示
  6. Spotify Engineering認識基本音高:Spotify 的開源音訊到 MIDI 轉換器
  7. arXiv (IEEE Access)nnAudio:使用 1D 卷積神經網路的即時 GPU 音訊到頻譜圖轉換工具箱

不斷探索

免費電子報

獲取每日 AI 簡報

每個工作日上午提供三則經過驗證的 AI 新聞,以淺顯易懂的英文撰寫。永久免費,無廣告。

每個工作日一封電子郵件。一鍵取消訂閱。我們絕不出售或分享您的地址。

測試一下自己

進行音頻測驗的恆定 Q 變換

每個答案皆有即時回饋,通過課程後可獲得具備可驗證 ID 的可分享證書。

開始測驗

支持免費的 AI 教育。AI Understanding 是一個 501(c)(3) 非營利組織——絕無廣告,絕無付費牆。 捐款

常見問題

什麼是音訊的恆定 Q 變換?

恆定 Q 變換 (CQT) 將音訊轉換為時頻影像,其頻率箱像音符一樣間隔,每個八度音程固定數量,而不是像傅立葉變換那樣均勻地以赫茲為單位。由於更改音調只是向上或向下滑動圖片,因此 CQT 是音樂轉錄、音高追蹤和和弦識別模型的標準輸入。

恆定 Q 變換中的頻率倉是如何間隔的?

CQT 使用對數間隔的 bin,因此每個 bin 對應於一個音高間隔,這與標準 FFT 的線性間隔不同。

Constant-Q 中的「Q」指的是什麼?

Q 是濾波器中心頻率與其頻寬的比率,CQT 在所有頻段上保持該比率恆定。

為什麼 CQT 在低頻下使用較長的分析視窗?

低音音符在絕對赫茲上非常接近,因此較長的視窗可以提供分離它們所需的精細頻率解析度。

當和弦移調一個八度時,CQT 音高模式會發生什麼事?

由於 CQT bin 是對數間隔的,因此每個頻率加倍都會將模式移動固定數量的 bin,等於每個倍頻程的 bin。

每個八度音階通常使用多少 CQT bin 來與半音對齊?

每個八度音階有 12 個音軌,每個音軌與西方半音階的半音對齊; 24 個 bin 提供四分色調解析度。