Аудио AI РЪКОВОДСТВО

Формиране на лъч и микрофонни масиви

Beamforming combines the signals from several microphones at known positions so that sound from one chosen direction adds up while sound from other directions partly cancels.

Чуйте това ръководствоAI-разказан · ~8 min

На тази страница9 минути четене
  1. Преглед
  2. Основни изводи
  3. Дълбоко гмуркане
  4. Steer a two-microphone delay-and-sum beam
  5. Внедряване в реалния свят
  6. Бъдещето на формирането на лъчи и микрофонните масиви
  7. Прилагайки го на практика
  8. Източници и допълнителна литература
  9. Продължете да изследвате
  10. Често задавани въпроси

Преглед

It is how smart speakers, hearing aids and conference systems pick out a voice across a noisy room, and modern versions use neural networks to decide where the speech is before the array does the filtering.

Основни изводи

  1. Beamforming delays and weights signals from several microphones so sound from one direction adds up and other directions partly cancel.
  2. Delay-and-sum with 10 elements gives roughly a tenfold signal-to-noise gain against uncorrelated noise, per MathWorks' worked example.
  3. MVDR keeps unit gain toward the target while minimizing output power, which automatically places nulls on strong interfering sources.
  4. Microphones spaced more than half a wavelength apart cause spatial aliasing, which creates ghost beams in wrong directions.
  5. Directional hearing aids improve front-facing speech by about 3 dB in real life, mainly when the talker is within about 10 feet.

Дълбоко гмуркане

A microphone array is a set of two or more microphones mounted at fixed, known positions, for example a ring of seven on top of a smart speaker. Beamforming is the signal processing that combines their outputs into one channel that listens mainly in a chosen direction, the look direction. It works like a spotlight for sound: speech from the target direction is reinforced while noise, music and other voices from elsewhere are reduced. Nothing physically moves; the beam is steered entirely in software, which is why one device can follow a person walking around a room.

The simplest method is delay-and-sum. Sound travels at about 343 meters per second, so a voice arriving from the left reaches the leftmost microphone slightly before the rightmost one. The delay between two microphones equals their spacing times the cosine of the arrival angle, divided by the speed of sound; for microphones 5 centimeters apart and sound arriving end-on, that is about 146 microseconds. Delay-and-sum shifts each channel by the delay expected for the look direction, so the target lines up perfectly, then adds the channels. The aligned target adds coherently, while noise that differs at each microphone adds up only partially. MathWorks' worked example shows the payoff: with 10 elements, the array gain is 10, so output signal-to-noise ratio is about ten times that of a single sensor for uncorrelated noise.

In practice audio beamformers work in the frequency domain. The signal is cut into short frames and converted with a Fourier transform into frequency bins, and each bin gets its own complex weight per microphone, which applies a phase shift and gain. This matters because a fixed time delay corresponds to a different phase at every frequency. Geometry sets hard limits. If microphones are more than half a wavelength apart, the array cannot tell some directions apart and produces ghost beams, an effect called spatial aliasing; at 8 kHz the wavelength is about 4.3 centimeters, so spacing must stay near 2 centimeters to avoid it. At low frequencies, where wavelengths are meters long, a small array barely differs from one microphone, so its beam is very wide.

Adaptive beamformers go further by also learning where the noise is. The minimum variance distortionless response (MVDR) beamformer chooses weights that keep the look direction at exactly unit gain, the distortionless constraint, while making total output power as small as possible. Since the target must pass unchanged, the only way to lower output power is to suppress everything else, so MVDR automatically places deep nulls on strong interferers. In MathWorks' example it put deep nulls on interferers at 30 and 50 degrees while keeping 0 dB gain toward a target at 45 degrees, where plain delay-and-sum let the interference mask the signal. The linearly constrained minimum variance (LCMV) beamformer generalizes this with several constraints at once.

Deep learning changed how the look direction is found. In 2016 Heymann, Drude and Haeb-Umbach at Paderborn University, and separately Erdogan and colleagues at MERL, trained neural networks to estimate a mask, a value for every time-frequency point saying how likely it is to be speech rather than noise. The masks are used to estimate the spatial statistics of speech and of noise, which in turn give MVDR weights without knowing the talker's angle in advance. Google described neural networks that perform the multichannel processing jointly with acoustic modeling in its 2017 paper on Google Home, reporting 8 to 28% fewer word errors relative to the existing production system. Apple's December 2018 article on HomePod described a six-microphone chain of echo cancellation, mask-based multichannel Wiener filtering, dereverberation and blind source separation, with the "Hey Siri" detector choosing the best output stream. Apple reported word error rate improvements of about 40% for reverberant speech, 74% in loud background noise and 61% with a competing talker.

Compared with single-microphone noise suppression, beamforming has an advantage that no single-channel model can match: it uses where the sound comes from, not just what it sounds like. A linear beamformer such as MVDR also adds little distortion to the target, which is why it is popular in front of speech recognizers. The trade-off is hardware: more microphones, synchronized sampling and careful placement. Hearing aids show the scale of benefit with only two microphones per ear. Laurel Christensen's 2013 review in The Hearing Review puts first-order directional benefit at about 3 dB in real life, notes that linking the microphones on both ears narrows the beam from about 60 to about 45 degrees, and reports that this added about 1 dB in speech-shaped noise but nothing measurable in babble.

Beamforming has well-known failure modes. Reverberation is the biggest: in a room, a voice reaches the array directly and also via many reflections from every wall, and reflected energy arriving from all sides cannot be removed by aiming. The Hearing Review notes that directional hearing aids help mainly when the talker is within about 10 feet, inside the critical distance where direct sound dominates. Diffuse noise, such as a crowded restaurant, has no single direction to null, so adaptive hearing aids fall back to a fixed pattern. Steering errors are dangerous for MVDR: in MathWorks' example, aiming at 43 degrees when the talker was at 45 made MVDR treat the talker as interference and suppress it, which is called signal self-nulling. Moving talkers, microphone mismatch and loudspeaker echo add further error.

What it takes in practice depends on the product. A hearing aid manages with two microphones a few millimeters apart and tight power limits. A smart speaker uses six or seven microphones in a ring of a few centimeters, multichannel echo cancellation so it can hear while playing music, and enough compute to run continuously; Apple says its HomePod front end uses less than 15% of one core of a 1.4 GHz A8 chip. MVDR itself requires estimating and inverting a small covariance matrix, one row and column per microphone, for every frequency bin, a modest cost on modern chips. The harder engineering is mechanical: keeping microphones matched, sealed from internal vibration and placed according to the spacing rules.

04Работен пример

Steer a two-microphone delay-and-sum beam

  1. Take two microphones 5 centimeters apart on a line and a talker 60 degrees off that line, with sound traveling at 343 meters per second.

  2. Compute the delay: 0.05 times cos(60 degrees) divided by 343, which is 0.025 / 343, about 73 microseconds.

  3. At a 48 kHz sampling rate, that is 73 microseconds times 48,000 per second, about 3.5 samples, so use a fractional delay filter or a frequency-domain phase shift.

  4. Delay the earlier microphone by that amount so the talker's waveform lines up in both channels, then average the two channels.

  5. Check aliasing: half a wavelength equals 343 / (2 f), so 5 centimeters spacing is safe only up to about 3.4 kHz.

  6. Play noise from a different angle and compare its level in the output with the level at a single microphone.

Какво показва

A beam aimed at 60 degrees. Verify it by playing a test tone from 60 degrees and from another angle: the on-target tone should keep its level after averaging, while noise from other directions and uncorrelated microphone noise should drop, up to about 3 dB for two microphones.

Внедряване в реалния свят

Echo на Amazon използва кръгъл масив от седем микрофона с формиране на лъча, намаляване на шума и анулиране на ехото; през април 2017 Amazon предложи същия дизайн на други производители на устройства като Alexa 7-Mic Far-Field Development Kit.

HomePod на Apple работи с преден край с шест микрофона с многоканално анулиране на ехото, намаляване на шума на базата на маска и разделяне на източника, като използва под 15% от едно A8 процесорно ядро, според изследователската статия на Apple от 2018 г.

Разпознаването на реч на Google Home, пуснато през ноември 2016 г., използва невронни мрежи, които извършват многоканална обработка съвместно с акустично моделиране, намалявайки грешките в думите с 8 до 28% в сравнение с по-ранната производствена система.

Слуховите апарати с насочени микрофони подобряват съотношението сигнал/шум на речта отпред с около 3 dB при измервания в реалния живот, с по-голяма полза, когато шумът седи зад слушателя.

MVDR формиране на лъч, базирано на маска, публикувано от изследователи на MERL и университета в Падерборн през 2016 г., позволява на невронна мрежа да локализира речта, така че форматорът на лъч да може да се насочи към говорещ, чиято посока е неизвестна.

Бъдещето на формирането на лъчи и микрофонните масиви

Since the mask-based beamformers of 2016, the line between beamforming and neural networks has kept blurring. Google's 2017 Google Home paper already trained multichannel processing jointly with recognition, and Apple's 2018 HomePod system combined classical filters with neural speech-presence estimates and stream selection driven by a wake-word detector. Open problems remain the ones physics imposes: heavy reverberation, diffuse noise with no direction to null, moving talkers and tiny devices whose microphones sit too close together to separate low frequencies. Hearing-aid research illustrates the limits; the 2013 Hearing Review analysis found that a narrower binaural beam added about 1 dB in one noise type and nothing in babble, so narrower is not automatically better for listeners who also need awareness of their surroundings. Active research directions include arrays formed from separate devices such as phones in a meeting room, which face clock-synchronization problems, and end-to-end networks that learn spatial filtering directly from raw multichannel audio. Whether learned approaches will fully replace explicit beamformers, or keep wrapping classical MVDR-style filters that are easy to analyze and add little distortion, is not settled.

Прилагайки го на практика

Защо има значение

  • Достъп и обхват. Той подобрява достъпността чрез интерфейси за транскрипция, дикторски текст и глас.
  • Разходи и бюджет. Медийните екипи могат да доставят изпипано аудио по-бързо с по-малки бюджети.
  • Скорост и мащаб. Системите, насочени към клиента, могат да обработват устни взаимодействия в по-голям мащаб.

Внимавайте за

  • Рисковете от злоупотреба с глас и имитация се увеличават, когато липсва съгласие.
  • Точността може да спадне при акценти, диалекти или шумна среда.
  • Синтетичното аудио може да бъде сбъркано с автентична реч без ясно етикетиране.

Следващи стъпки

  1. Получете изрично съгласие за улавяне на глас, клониране и повторно използване.
  2. Тествайте качеството при различни високоговорители и фонови условия.
  3. Определете кога човек трябва да прегледа или одобри резултатите.
  4. Етикетирайте синтетичното аудио и поддържайте записи за произход за отчетност.

Източници и допълнителна литература

  1. Apple Machine Learning ResearchOptimizing Siri on HomePod in Far-Field Settings
  2. Amazon Alexa Developer BlogBringing Far-Field Voice Technology from the Amazon Echo to Third-Party Devices
  3. ISCA Archive (Interspeech 2017)Acoustic Modeling for Google Home
  4. MERL (Interspeech 2016)Improved MVDR beamforming using single-channel mask prediction networks
  5. Paderborn University (ICASSP 2016)Neural Network Based Spectral Mask Estimation for Acoustic Beamforming
  6. MathWorksConventional and Adaptive Beamformers
  7. The Hearing ReviewЕволюцията на насочеността: Развитието довело ли е до по-голяма полза за потребителите на слухови апарати?

Продължете да изследвате

Безплатен бюлетин

Получавайте дневния AI бюлетин

Три проверени AI истории всяка делнична сутрин, написани на достъпен език. Безплатно завинаги, без реклами.

Един имейл всеки делничен ден. Отписване с едно кликване. Никога не продаваме и не споделяме вашия имейл адрес.

Тествайте себе си

Take the Beamforming and Microphone Arrays quiz

Незабавна обратна връзка за всеки отговор и сертификат за споделяне с подлежащ на проверка идентификационен номер, след като преминете курс.

Стартирай теста

Подкрепете безплатното AI образование. AI Understanding е 501(c)(3) с нестопанска цел — без реклами, без платен достъп, никога. Направете дарение

Често задавани въпроси

Какво е Beamforming и Microphone Arrays?

Beamforming combines the signals from several microphones at known positions so that sound from one chosen direction adds up while sound from other directions partly cancels. It is how smart speakers, hearing aids and conference systems pick out a voice across a noisy room, and modern versions use neural networks to decide where the speech is before the array does the filtering.

Каква физическа реплика основно използва формирането на лъч?

Звукът достига до всеки микрофон в малко по-различно време; beamforming използва тези забавяния за филтриране по посока.

Как се нарича най-простата техника за формиране на лъч?

Delay-and-sum подравнява сигналите от целевата посока и ги добавя, така че да се подсилват един друг.

Какво е определящото ограничение в MVDR beamformer?

MVDR налага ограничение за реакция без изкривяване за целта и минимизира изходната мощност, която е предмет на това.

Как формирането на лъча е свързано с потискането на шума от един микрофон?

Beamforming добавя насочено/пространствено филтриране, което едноканалните методи не могат да предоставят, така че те работят заедно.

Защо твърде голямото разстояние между микрофоните може да създаде пространствено псевдоним?

При високи честоти широко разположените микрофони могат да семплират двусмислено фазата, което прави различните посоки да изглеждат еднакви.

Продължавай да учиш

Свързани ръководства

Още ръководства, избрани за тази тема

Получете безплатен AI сертификатКратък безплатен курс и изпит. Без разходи, без карта. Около 2,5 часа.Започнете безплатно →