오디오에 대한 Constant-Q 변환

A spectrogram shows how much energy a sound has at each frequency over time. The usual way to compute one is the short-time Fourier transform (STFT), which cuts audio into frames and measures energy in frequency bins spaced evenly in hertz, for example every 10.8 Hz. Music does not work in even steps. Each octave doubles the frequency, and Western tuning splits the octave into 12 semitones, each about 5.9% higher than the last. Near the bottom of a piano, adjacent notes C1 and C#1 are only 1.9 Hz apart, while near the top they are more than 200 Hz apart. An evenly spaced spectrogram therefore blurs low notes together and spends many bins between high notes. The constant-Q transform fixes this by spacing bins geometrically: every bin is a fixed ratio above the previous one, so each bin can sit exactly on a musical note.

The name comes from the quality factor Q, the ratio of a bin's center frequency to its bandwidth. In the CQT every bin has the same Q, so bandwidth grows in proportion to frequency. To measure a narrow band precisely you must listen for longer, so low bins use long analysis windows and high bins use short ones. Step by step, the computation is: choose a lowest frequency, such as C1 at 32.70 Hz; choose bins per octave, typically 12 for semitones or 36 to 60 for finer pitch detail; place each center frequency a factor of two to the power of one over the bins-per-octave above the last; give each bin a window whose length is Q times the sample rate divided by its center frequency; and at regular time steps, multiply the audio by each bin's windowed complex sinusoid and sum, which yields the magnitude and phase of that note at that moment.

The idea was formalized for music by Judith C. Brown in "Calculation of a Constant Q Spectral Transform", published in the Journal of the Acoustical Society of America in 1991 (volume 89, pages 425 to 434). Her MIT Media Lab page still hosts Matlab code for the direct method. Computing every bin with its own long window was slow, so later work focused on efficiency. Christian Schörkhuber and Anssi Klapuri's 2010 constant-Q toolbox, the method cited by librosa, computes the top octave and then repeatedly halves the sample rate to compute each lower octave cheaply. In 2011 Gino Angelo Velasco, Nicki Holighaus, Monika Dörfler and Thomas Grill showed at the DAFx conference how to build a perfectly invertible CQT using nonstationary Gabor frames, the basis of MATLAB's cqt and icqt functions.

Deep learning gave the CQT a new role. A convolutional neural network learns small patterns and slides them across an image. On a CQT, playing the same chord in a different key moves the whole pattern by a fixed number of bins without changing its shape, so one learned filter recognizes the chord in every key. On a linear spectrogram the spacing between harmonics changes with pitch, so that trick fails. In 2017 Rachel Bittner and colleagues at New York University introduced the harmonic CQT (HCQT), a stack of CQTs whose lowest frequencies are multiplied by 0.5, 1, 2, 3, 4 and 5, so a note's overtones line up in the same position across layers, like the color channels of an image. Spotify's Basic Pitch, released in June 2022, uses this input to transcribe polyphonic audio to MIDI with fewer than 17,000 parameters and under 20 MB of peak memory.

Compared with the alternatives, the CQT's strength is pitch alignment and its cost is everything else. The STFT is fast, simple and exactly invertible but has the wrong frequency layout for notes. The mel spectrogram, widely used for speech, compresses frequency on a perceptual scale that is nearly linear at low frequencies, so its bins do not fall on musical notes. Chroma features, often computed from a CQT, fold all octaves into 12 pitch classes, which helps chord and key detection but discards which octave a note is in. Learned front ends let a network discover its own filters, at the price of more data and less interpretability.

The CQT has real limitations. Long low-frequency windows smear time: with 12 bins per octave at a 22,050 Hz sample rate, the window for C1 is about 11,300 samples, roughly half a second, so fast bass notes and drum hits blur at the bottom of the picture while the top is sharp. Classical implementations were not invertible, which mattered for editing and source separation until frame-based methods like Velasco's appeared. Computation is heavier than an STFT; the nnAudio authors measured librosa taking about 103 seconds for 1,770 piano recordings on a CPU, against 0.26 seconds for their GPU version. Tuning matters too: a recording tuned slightly away from A at 440 Hz puts notes between bins unless the transform is shifted to match.

In practice the CQT is cheap to try. librosa computes it with one function call, with defaults of a 512-sample hop, 84 bins and 12 bins per octave from C1 nnAudio and similar libraries compute it on a GPU as a layer of the network, which avoids storing spectrograms on disk and makes the filters trainable. The main design choices are the lowest frequency, the number of octaves, bins per octave, which trades pitch precision against time precision, and hop size. The 2017 deep salience model, for example, used 60 bins per octave, 20 cents per bin, to track fine pitch, while tasks that only need note names can use 12.