HƯỚNG DẪN AI âm thanh

Mảng chùm tia và micrô

Beamforming combines the signals from several microphones at known positions so that sound from one chosen direction adds up while sound from other directions partly cancels.

Nghe hướng dẫn nàyAI thuật lại · ~8 min

Trên trang nàyđọc 9 phút
  1. Tổng quan
  2. Những điểm chính rút ra
  3. Lặn sâu
  4. Steer a two-microphone delay-and-sum beam
  5. Triển khai trong thế giới thực
  6. Tương lai của mảng định dạng chùm tia và micrô
  7. Đưa nó vào thực tế
  8. Nguồn tham khảo và đọc thêm
  9. Tiếp tục khám phá
  10. Câu hỏi thường gặp

Tổng quan

It is how smart speakers, hearing aids and conference systems pick out a voice across a noisy room, and modern versions use neural networks to decide where the speech is before the array does the filtering.

Những điểm chính rút ra

  1. Beamforming delays and weights signals from several microphones so sound from one direction adds up and other directions partly cancel.
  2. Delay-and-sum with 10 elements gives roughly a tenfold signal-to-noise gain against uncorrelated noise, per MathWorks' worked example.
  3. MVDR keeps unit gain toward the target while minimizing output power, which automatically places nulls on strong interfering sources.
  4. Microphones spaced more than half a wavelength apart cause spatial aliasing, which creates ghost beams in wrong directions.
  5. Directional hearing aids improve front-facing speech by about 3 dB in real life, mainly when the talker is within about 10 feet.

Lặn sâu

A microphone array is a set of two or more microphones mounted at fixed, known positions, for example a ring of seven on top of a smart speaker. Beamforming is the signal processing that combines their outputs into one channel that listens mainly in a chosen direction, the look direction. It works like a spotlight for sound: speech from the target direction is reinforced while noise, music and other voices from elsewhere are reduced. Nothing physically moves; the beam is steered entirely in software, which is why one device can follow a person walking around a room.

The simplest method is delay-and-sum. Sound travels at about 343 meters per second, so a voice arriving from the left reaches the leftmost microphone slightly before the rightmost one. The delay between two microphones equals their spacing times the cosine of the arrival angle, divided by the speed of sound; for microphones 5 centimeters apart and sound arriving end-on, that is about 146 microseconds. Delay-and-sum shifts each channel by the delay expected for the look direction, so the target lines up perfectly, then adds the channels. The aligned target adds coherently, while noise that differs at each microphone adds up only partially. MathWorks' worked example shows the payoff: with 10 elements, the array gain is 10, so output signal-to-noise ratio is about ten times that of a single sensor for uncorrelated noise.

In practice audio beamformers work in the frequency domain. The signal is cut into short frames and converted with a Fourier transform into frequency bins, and each bin gets its own complex weight per microphone, which applies a phase shift and gain. This matters because a fixed time delay corresponds to a different phase at every frequency. Geometry sets hard limits. If microphones are more than half a wavelength apart, the array cannot tell some directions apart and produces ghost beams, an effect called spatial aliasing; at 8 kHz the wavelength is about 4.3 centimeters, so spacing must stay near 2 centimeters to avoid it. At low frequencies, where wavelengths are meters long, a small array barely differs from one microphone, so its beam is very wide.

Adaptive beamformers go further by also learning where the noise is. The minimum variance distortionless response (MVDR) beamformer chooses weights that keep the look direction at exactly unit gain, the distortionless constraint, while making total output power as small as possible. Since the target must pass unchanged, the only way to lower output power is to suppress everything else, so MVDR automatically places deep nulls on strong interferers. In MathWorks' example it put deep nulls on interferers at 30 and 50 degrees while keeping 0 dB gain toward a target at 45 degrees, where plain delay-and-sum let the interference mask the signal. The linearly constrained minimum variance (LCMV) beamformer generalizes this with several constraints at once.

Deep learning changed how the look direction is found. In 2016 Heymann, Drude and Haeb-Umbach at Paderborn University, and separately Erdogan and colleagues at MERL, trained neural networks to estimate a mask, a value for every time-frequency point saying how likely it is to be speech rather than noise. The masks are used to estimate the spatial statistics of speech and of noise, which in turn give MVDR weights without knowing the talker's angle in advance. Google described neural networks that perform the multichannel processing jointly with acoustic modeling in its 2017 paper on Google Home, reporting 8 to 28% fewer word errors relative to the existing production system. Apple's December 2018 article on HomePod described a six-microphone chain of echo cancellation, mask-based multichannel Wiener filtering, dereverberation and blind source separation, with the "Hey Siri" detector choosing the best output stream. Apple reported word error rate improvements of about 40% for reverberant speech, 74% in loud background noise and 61% with a competing talker.

Compared with single-microphone noise suppression, beamforming has an advantage that no single-channel model can match: it uses where the sound comes from, not just what it sounds like. A linear beamformer such as MVDR also adds little distortion to the target, which is why it is popular in front of speech recognizers. The trade-off is hardware: more microphones, synchronized sampling and careful placement. Hearing aids show the scale of benefit with only two microphones per ear. Laurel Christensen's 2013 review in The Hearing Review puts first-order directional benefit at about 3 dB in real life, notes that linking the microphones on both ears narrows the beam from about 60 to about 45 degrees, and reports that this added about 1 dB in speech-shaped noise but nothing measurable in babble.

Beamforming has well-known failure modes. Reverberation is the biggest: in a room, a voice reaches the array directly and also via many reflections from every wall, and reflected energy arriving from all sides cannot be removed by aiming. The Hearing Review notes that directional hearing aids help mainly when the talker is within about 10 feet, inside the critical distance where direct sound dominates. Diffuse noise, such as a crowded restaurant, has no single direction to null, so adaptive hearing aids fall back to a fixed pattern. Steering errors are dangerous for MVDR: in MathWorks' example, aiming at 43 degrees when the talker was at 45 made MVDR treat the talker as interference and suppress it, which is called signal self-nulling. Moving talkers, microphone mismatch and loudspeaker echo add further error.

What it takes in practice depends on the product. A hearing aid manages with two microphones a few millimeters apart and tight power limits. A smart speaker uses six or seven microphones in a ring of a few centimeters, multichannel echo cancellation so it can hear while playing music, and enough compute to run continuously; Apple says its HomePod front end uses less than 15% of one core of a 1.4 GHz A8 chip. MVDR itself requires estimating and inverting a small covariance matrix, one row and column per microphone, for every frequency bin, a modest cost on modern chips. The harder engineering is mechanical: keeping microphones matched, sealed from internal vibration and placed according to the spacing rules.

04Ví dụ hoạt động

Steer a two-microphone delay-and-sum beam

  1. Take two microphones 5 centimeters apart on a line and a talker 60 degrees off that line, with sound traveling at 343 meters per second.

  2. Compute the delay: 0.05 times cos(60 degrees) divided by 343, which is 0.025 / 343, about 73 microseconds.

  3. At a 48 kHz sampling rate, that is 73 microseconds times 48,000 per second, about 3.5 samples, so use a fractional delay filter or a frequency-domain phase shift.

  4. Delay the earlier microphone by that amount so the talker's waveform lines up in both channels, then average the two channels.

  5. Check aliasing: half a wavelength equals 343 / (2 f), so 5 centimeters spacing is safe only up to about 3.4 kHz.

  6. Play noise from a different angle and compare its level in the output with the level at a single microphone.

Những gì nó cho thấy

A beam aimed at 60 degrees. Verify it by playing a test tone from 60 degrees and from another angle: the on-target tone should keep its level after averaging, while noise from other directions and uncorrelated microphone noise should drop, up to about 3 dB for two microphones.

Triển khai trong thế giới thực

Amazon's Echo uses a seven-microphone circular array with beamforming, noise reduction and echo cancellation; in April 2017 Amazon offered the same design to other device makers as the Alexa 7-Mic Far-Field Development Kit.

Apple's HomePod runs a six-microphone front end with multichannel echo cancellation, mask-based noise reduction and source separation, using under 15% of one A8 processor core, according to Apple's 2018 research article.

Google Home's speech recognizer, launched in November 2016, uses neural networks that perform multichannel processing jointly with acoustic modeling, cutting word errors 8 to 28% relative to the earlier production system.

Hearing aids with directional microphones improve the signal-to-noise ratio of speech from the front by about 3 dB in real-life measurements, with more benefit when noise sits behind the listener.

Mask-based MVDR beamforming, published by MERL and Paderborn University researchers in 2016, lets a neural network locate speech so the beamformer can aim at a talker whose direction is unknown.

Tương lai của mảng định dạng chùm tia và micrô

Since the mask-based beamformers of 2016, the line between beamforming and neural networks has kept blurring. Google's 2017 Google Home paper already trained multichannel processing jointly with recognition, and Apple's 2018 HomePod system combined classical filters with neural speech-presence estimates and stream selection driven by a wake-word detector. Open problems remain the ones physics imposes: heavy reverberation, diffuse noise with no direction to null, moving talkers and tiny devices whose microphones sit too close together to separate low frequencies. Hearing-aid research illustrates the limits; the 2013 Hearing Review analysis found that a narrower binaural beam added about 1 dB in one noise type and nothing in babble, so narrower is not automatically better for listeners who also need awareness of their surroundings. Active research directions include arrays formed from separate devices such as phones in a meeting room, which face clock-synchronization problems, and end-to-end networks that learn spatial filtering directly from raw multichannel audio. Whether learned approaches will fully replace explicit beamformers, or keep wrapping classical MVDR-style filters that are easy to analyze and add little distortion, is not settled.

Đưa nó vào thực tế

Tại sao nó quan trọng

  • Truy cập và tiếp cận. Nó cải thiện khả năng tiếp cận thông qua phiên âm, tường thuật và giao diện giọng nói.
  • Chi phí và ngân sách. Các nhóm truyền thông có thể gửi âm thanh tinh tế nhanh hơn với ngân sách nhỏ hơn.
  • Tốc độ và tỷ lệ. Các hệ thống hướng tới khách hàng có thể xử lý các tương tác bằng giọng nói ở quy mô lớn hơn.

coi chừng

  • Rủi ro lạm dụng giọng nói và mạo danh sẽ tăng lên khi thiếu sự đồng ý.
  • Độ chính xác có thể giảm đối với các giọng, phương ngữ hoặc môi trường ồn ào.
  • Âm thanh tổng hợp có thể bị nhầm lẫn với lời nói đích thực nếu không có nhãn rõ ràng.

Các bước tiếp theo

  1. Nhận được sự đồng ý rõ ràng để thu âm, sao chép và tái sử dụng giọng nói.
  2. Kiểm tra chất lượng trên nhiều loa và điều kiện nền khác nhau.
  3. Xác định khi nào con người phải xem xét hoặc phê duyệt kết quả đầu ra.
  4. Dán nhãn âm thanh tổng hợp và lưu giữ hồ sơ xuất xứ để đảm bảo trách nhiệm giải trình.

Nguồn tham khảo và đọc thêm

  1. Apple Machine Learning ResearchOptimizing Siri on HomePod in Far-Field Settings
  2. Amazon Alexa Developer BlogBringing Far-Field Voice Technology from the Amazon Echo to Third-Party Devices
  3. ISCA Archive (Interspeech 2017)Acoustic Modeling for Google Home
  4. MERL (Interspeech 2016)Improved MVDR beamforming using single-channel mask prediction networks
  5. Paderborn University (ICASSP 2016)Neural Network Based Spectral Mask Estimation for Acoustic Beamforming
  6. MathWorksConventional and Adaptive Beamformers
  7. The Hearing ReviewThe Evolution of Directionality: Have Developments Led to Greater Benefit for Hearing Aid Users?

Tiếp tục khám phá

Bản tin miễn phí

Nhận bản tóm tắt AI hàng ngày

Ba câu chuyện AI đã được xác minh vào mỗi buổi sáng các ngày trong tuần, được viết bằng tiếng Anh đơn giản. Miễn phí mãi mãi, không có quảng cáo.

Một email mỗi ngày trong tuần. Hủy đăng ký trong một cú nhấp chuột. Chúng tôi không bao giờ bán hoặc chia sẻ địa chỉ của bạn.

Tự kiểm tra

Take the Beamforming and Microphone Arrays quiz

Phản hồi tức thì về mọi câu trả lời và chứng chỉ có thể chia sẻ với ID có thể xác minh sau khi bạn vượt qua khóa học.

Bắt đầu bài kiểm tra

Hỗ trợ giáo dục AI miễn phí. AI Understanding là tổ chức phi lợi nhuận 501(c)(3) — không bao giờ có quảng cáo, không có tường phí. Hãy quyên góp

Câu hỏi thường gặp

What is Beamforming and Microphone Arrays?

Beamforming combines the signals from several microphones at known positions so that sound from one chosen direction adds up while sound from other directions partly cancels. It is how smart speakers, hearing aids and conference systems pick out a voice across a noisy room, and modern versions use neural networks to decide where the speech is before the array does the filtering.

Tín hiệu vật lý nào mà Beamforming chủ yếu khai thác?

Âm thanh đến từng micrô vào những thời điểm hơi khác nhau; Beamforming sử dụng những độ trễ này để lọc theo hướng.

Kỹ thuật tạo chùm tia đơn giản nhất được gọi là gì?

Delay-and-sum căn chỉnh các tín hiệu từ hướng mục tiêu và thêm chúng để chúng củng cố lẫn nhau.

What is the defining constraint in an MVDR beamformer?

MVDR imposes a distortionless-response constraint for the target and minimizes output power subject to it.

Định dạng chùm tia liên quan như thế nào đến khả năng khử tiếng ồn của một micrô?

Beamforming bổ sung tính năng lọc hướng/không gian mà các phương pháp kênh đơn không thể cung cấp, vì vậy chúng hoạt động cùng nhau.

Why can microphone spacing that is too large create spatial aliasing?

At high frequencies, widely spaced microphones can sample phase ambiguously, making different directions appear alike.