HƯỚNG DẪN AI âm thanh

Biến đổi Constant-Q cho âm thanh

The constant-Q transform (CQT) turns audio into a time-frequency picture whose frequency bins are spaced like musical notes, a fixed number per octave, instead of evenly in hertz like the Fourier transform.

Nghe hướng dẫn nàyAI thuật lại · ~8 min

Trên trang nàyđọc 9 phút
  1. Tổng quan
  2. Những điểm chính rút ra
  3. Lặn sâu
  4. Find a note in a CQT with librosa
  5. Triển khai trong thế giới thực
  6. Tương lai của Constant-Q Transform cho âm thanh
  7. Đưa nó vào thực tế
  8. Nguồn tham khảo và đọc thêm
  9. Tiếp tục khám phá
  10. Câu hỏi thường gặp

Tổng quan

Because a change of key simply slides the picture up or down, the CQT is a standard input for music transcription, pitch tracking and chord recognition models.

Những điểm chính rút ra

  1. The constant-Q transform spaces frequency bins geometrically, a fixed number per octave, so each bin can line up with a musical note.
  2. Every CQT bin has the same ratio of center frequency to bandwidth, so low notes get long windows and high notes short ones.
  3. Transposing music shifts a CQT up or down without changing its shape, which suits convolutional neural networks.
  4. Judith C. Brown described the constant-Q spectral transform for music in the Journal of the Acoustical Society of America in 1991.
  5. Spotify's Basic Pitch uses a harmonic CQT input to transcribe audio to MIDI with fewer than 17,000 model parameters.

Lặn sâu

A spectrogram shows how much energy a sound has at each frequency over time. The usual way to compute one is the short-time Fourier transform (STFT), which cuts audio into frames and measures energy in frequency bins spaced evenly in hertz, for example every 10.8 Hz. Music does not work in even steps. Each octave doubles the frequency, and Western tuning splits the octave into 12 semitones, each about 5.9% higher than the last. Near the bottom of a piano, adjacent notes C1 and C#1 are only 1.9 Hz apart, while near the top they are more than 200 Hz apart. An evenly spaced spectrogram therefore blurs low notes together and spends many bins between high notes. The constant-Q transform fixes this by spacing bins geometrically: every bin is a fixed ratio above the previous one, so each bin can sit exactly on a musical note.

The name comes from the quality factor Q, the ratio of a bin's center frequency to its bandwidth. In the CQT every bin has the same Q, so bandwidth grows in proportion to frequency. To measure a narrow band precisely you must listen for longer, so low bins use long analysis windows and high bins use short ones. Step by step, the computation is: choose a lowest frequency, such as C1 at 32.70 Hz; choose bins per octave, typically 12 for semitones or 36 to 60 for finer pitch detail; place each center frequency a factor of two to the power of one over the bins-per-octave above the last; give each bin a window whose length is Q times the sample rate divided by its center frequency; and at regular time steps, multiply the audio by each bin's windowed complex sinusoid and sum, which yields the magnitude and phase of that note at that moment.

The idea was formalized for music by Judith C. Brown in "Calculation of a Constant Q Spectral Transform", published in the Journal of the Acoustical Society of America in 1991 (volume 89, pages 425 to 434). Her MIT Media Lab page still hosts Matlab code for the direct method. Computing every bin with its own long window was slow, so later work focused on efficiency. Christian Schörkhuber and Anssi Klapuri's 2010 constant-Q toolbox, the method cited by librosa, computes the top octave and then repeatedly halves the sample rate to compute each lower octave cheaply. In 2011 Gino Angelo Velasco, Nicki Holighaus, Monika Dörfler and Thomas Grill showed at the DAFx conference how to build a perfectly invertible CQT using nonstationary Gabor frames, the basis of MATLAB's cqt and icqt functions.

Deep learning gave the CQT a new role. A convolutional neural network learns small patterns and slides them across an image. On a CQT, playing the same chord in a different key moves the whole pattern by a fixed number of bins without changing its shape, so one learned filter recognizes the chord in every key. On a linear spectrogram the spacing between harmonics changes with pitch, so that trick fails. In 2017 Rachel Bittner and colleagues at New York University introduced the harmonic CQT (HCQT), a stack of CQTs whose lowest frequencies are multiplied by 0.5, 1, 2, 3, 4 and 5, so a note's overtones line up in the same position across layers, like the color channels of an image. Spotify's Basic Pitch, released in June 2022, uses this input to transcribe polyphonic audio to MIDI with fewer than 17,000 parameters and under 20 MB of peak memory.

Compared with the alternatives, the CQT's strength is pitch alignment and its cost is everything else. The STFT is fast, simple and exactly invertible but has the wrong frequency layout for notes. The mel spectrogram, widely used for speech, compresses frequency on a perceptual scale that is nearly linear at low frequencies, so its bins do not fall on musical notes. Chroma features, often computed from a CQT, fold all octaves into 12 pitch classes, which helps chord and key detection but discards which octave a note is in. Learned front ends let a network discover its own filters, at the price of more data and less interpretability.

The CQT has real limitations. Long low-frequency windows smear time: with 12 bins per octave at a 22,050 Hz sample rate, the window for C1 is about 11,300 samples, roughly half a second, so fast bass notes and drum hits blur at the bottom of the picture while the top is sharp. Classical implementations were not invertible, which mattered for editing and source separation until frame-based methods like Velasco's appeared. Computation is heavier than an STFT; the nnAudio authors measured librosa taking about 103 seconds for 1,770 piano recordings on a CPU, against 0.26 seconds for their GPU version. Tuning matters too: a recording tuned slightly away from A at 440 Hz puts notes between bins unless the transform is shifted to match.

In practice the CQT is cheap to try. librosa computes it with one function call, with defaults of a 512-sample hop, 84 bins and 12 bins per octave from C1 nnAudio and similar libraries compute it on a GPU as a layer of the network, which avoids storing spectrograms on disk and makes the filters trainable. The main design choices are the lowest frequency, the number of octaves, bins per octave, which trades pitch precision against time precision, and hop size. The 2017 deep salience model, for example, used 60 bins per octave, 20 cents per bin, to track fine pitch, while tasks that only need note names can use 12.

04Ví dụ hoạt động

Find a note in a CQT with librosa

  1. Generate or record a 2-second tone at 440 Hz (the A above middle C) sampled at 22,050 Hz.

  2. Compute C = librosa.cqt(y, sr=22050), keeping the defaults: hop 512, lowest frequency C1 at 32.70 Hz, 84 bins, 12 per octave.

  3. Take magnitudes with numpy.abs(C) and convert to decibels with librosa.amplitude_to_db.

  4. Predict the bright row: A4 is 45 semitones above C1 (36 to reach C4, plus 9 to A4), so expect bin index 45.

  5. Check with librosa.cqt_frequencies(84, fmin=librosa.note_to_hz('C1')), whose entry 45 should be about 440 Hz.

  6. Repeat with a 466.16 Hz tone (A#4) and confirm the bright row moves up by exactly one bin.

Những gì nó cho thấy

A CQT image with a single bright horizontal line at bin 45 that moves one bin per semitone. Checking that the predicted and observed bins agree confirms the geometric spacing, and comparing the sharpness of a low note with a high note shows the time-frequency trade-off.

Triển khai trong thế giới thực

Spotify's open-source Basic Pitch, released in June 2022, converts recordings of many instruments, including voice, into MIDI using a harmonic CQT input and a model with fewer than 17,000 parameters.

The deep salience model from NYU (ISMIR 2017) estimates melodies and multiple pitches from a harmonic CQT of 6 octaves at 60 bins per octave, starting at C1 (32.7 Hz).

The librosa Python library's cqt function is a common starting point for music research, by default producing 84 semitone bins, seven octaves upward from C1.

The nnAudio toolbox computes the CQT on a GPU inside a neural network, which its authors measured at roughly 400 times faster than librosa on a set of 1,770 piano recordings.

MATLAB's cqt and icqt functions provide an invertible CQT, so an engineer can edit a sound in the note-aligned domain, for example removing one pitch, and convert it back to audio.

Tương lai của Constant-Q Transform cho âm thanh

Current research uses the CQT less as a final feature and more as a structured input or a layer inside networks. Since nnAudio (2019) made GPU and trainable CQT layers practical, models can start from note-aligned filters and fine-tune them during training. The harmonic CQT introduced at ISMIR 2017 and reused in Spotify's Basic Pitch in 2022 shows a continuing trend toward representations that encode musical structure so that smaller models suffice; Basic Pitch runs faster than real time on most modern computers according to Spotify. Open problems remain. The trade-off between time and frequency resolution at low frequencies is fundamental, so fast bass lines and percussion stay blurred unless combined with another representation. Non-Western music that uses intervals other than 12 equal semitones does not map neatly onto standard bin spacings, although finer bins and adjustable tuning partly address this. Invertible CQTs based on nonstationary Gabor frames, available in MATLAB and other toolboxes, make note-domain editing and separation possible, but how they compare with learned encoders for generative music models is still being studied rather than settled.

Đưa nó vào thực tế

Tại sao nó quan trọng

  • Truy cập và tiếp cận. Nó cải thiện khả năng tiếp cận thông qua phiên âm, tường thuật và giao diện giọng nói.
  • Chi phí và ngân sách. Các nhóm truyền thông có thể gửi âm thanh tinh tế nhanh hơn với ngân sách nhỏ hơn.
  • Tốc độ và tỷ lệ. Các hệ thống hướng tới khách hàng có thể xử lý các tương tác bằng giọng nói ở quy mô lớn hơn.

coi chừng

  • Rủi ro lạm dụng giọng nói và mạo danh sẽ tăng lên khi thiếu sự đồng ý.
  • Độ chính xác có thể giảm đối với các giọng, phương ngữ hoặc môi trường ồn ào.
  • Âm thanh tổng hợp có thể bị nhầm lẫn với lời nói đích thực nếu không có nhãn rõ ràng.

Các bước tiếp theo

  1. Nhận được sự đồng ý rõ ràng để thu âm, sao chép và tái sử dụng giọng nói.
  2. Kiểm tra chất lượng trên nhiều loa và điều kiện nền khác nhau.
  3. Xác định khi nào con người phải xem xét hoặc phê duyệt kết quả đầu ra.
  4. Dán nhãn âm thanh tổng hợp và lưu giữ hồ sơ xuất xứ để đảm bảo trách nhiệm giải trình.

Nguồn tham khảo và đọc thêm

  1. MIT Media Lab, Judith C. BrownConstant Q Transform (Matlab code and reference to Brown 1991)
  2. librosa documentationlibrosa.cqt
  3. DAFx 2011Constructing an invertible constant-Q transform with nonstationary Gabor frames
  4. MathWorkscqt: Constant-Q nonstationary Gabor transform
  5. ISMIR 2017Deep Salience Representations for F0 Estimation in Polyphonic Music
  6. Spotify EngineeringMeet Basic Pitch: Spotify's Open Source Audio-to-MIDI Converter
  7. arXiv (IEEE Access)nnAudio: An on-the-fly GPU Audio to Spectrogram Conversion Toolbox Using 1D Convolution Neural Networks

Tiếp tục khám phá

Bản tin miễn phí

Nhận bản tóm tắt AI hàng ngày

Ba câu chuyện AI đã được xác minh vào mỗi buổi sáng các ngày trong tuần, được viết bằng tiếng Anh đơn giản. Miễn phí mãi mãi, không có quảng cáo.

Một email mỗi ngày trong tuần. Hủy đăng ký trong một cú nhấp chuột. Chúng tôi không bao giờ bán hoặc chia sẻ địa chỉ của bạn.

Tự kiểm tra

Take the Constant-Q Transform for Audio quiz

Phản hồi tức thì về mọi câu trả lời và chứng chỉ có thể chia sẻ với ID có thể xác minh sau khi bạn vượt qua khóa học.

Bắt đầu bài kiểm tra

Hỗ trợ giáo dục AI miễn phí. AI Understanding là tổ chức phi lợi nhuận 501(c)(3) — không bao giờ có quảng cáo, không có tường phí. Hãy quyên góp

Câu hỏi thường gặp

Constant-Q Transform cho Âm thanh là gì?

The constant-Q transform (CQT) turns audio into a time-frequency picture whose frequency bins are spaced like musical notes, a fixed number per octave, instead of evenly in hertz like the Fourier transform. Because a change of key simply slides the picture up or down, the CQT is a standard input for music transcription, pitch tracking and chord recognition models.

Các ngăn tần số được đặt cách nhau như thế nào trong Biến đổi Constant-Q?

CQT sử dụng các khoảng cách theo logarit để mỗi khoảng tương ứng với một khoảng cao độ âm nhạc, không giống như khoảng cách tuyến tính của FFT tiêu chuẩn.

'Q' trong Constant-Q đề cập đến điều gì?

Q là tỷ lệ giữa tần số trung tâm của bộ lọc với băng thông của nó và CQT giữ tỷ lệ này không đổi trên tất cả các thùng.

Tại sao CQT sử dụng cửa sổ phân tích dài hơn ở tần số thấp?

Các nốt nhạc thấp rất gần nhau ở hertz tuyệt đối, vì vậy cửa sổ dài hơn sẽ cung cấp độ phân giải tần số tốt cần thiết để tách chúng ra.

What happens to a CQT pitch pattern when a chord is transposed up one octave?

Because CQT bins are logarithmically spaced, doubling every frequency shifts a pattern by a fixed number of bins equal to the bins per octave.

Có bao nhiêu thùng CQT trên mỗi quãng tám thường được sử dụng để căn chỉnh với nửa cung?

Mười hai ô trên mỗi quãng tám sắp xếp mỗi ô theo một nửa cung của thang màu phương Tây; 24 thùng cho độ phân giải một phần tư tông màu.