āĻ…āĻĄāĻŋāĻ“ āĻāφāχ āĻ—āĻžāχāĻĄ

āĻ“āϝāĻŧ⧇āĻ­āύ⧇āϟ

WaveNet is an autoregressive model for raw audio waveforms.

āĻāχ āύāĻŋāĻ°ā§āĻĻ⧇āĻļāĻŋāĻ•āĻž āĻļ⧁āύ⧁āύāĻāφāχ-āĻŦāĻ°ā§āĻŖāĻŋāϤ ¡ ~1 min

āĻāχ āĻĒ⧃āĻˇā§āĻ āĻžāϝāĻŧ1 āĻŽāĻŋāύāĻŋāϟ āĻĒāĻĄāĻŧāĻž
  1. āĻ“āĻ­āĻžāϰāĻ­āĻŋāω
  2. āĻ—āĻ­ā§€āϰ āĻĄā§āĻŦ
  3. āĻŦāĻžāĻ¸ā§āϤāĻŦ-āĻŦāĻŋāĻļā§āĻŦ āĻŦāĻžāĻ¸ā§āϤāĻŦāĻžāϝāĻŧāύ
  4. āĻ“āϝāĻŧ⧇āĻ­āύ⧇āĻŸā§‡āϰ āĻ­āĻŦāĻŋāĻˇā§āϝāϤ
  5. āĻāϟāĻžāϕ⧇ āĻŦāĻžāĻ¸ā§āϤāĻŦ⧇ āĻĒā§āϰāϝāĻŧā§‹āĻ— āĻ•āϰāĻž āĻšāĻšā§āϛ⧇
  6. āĻ…āĻ¨ā§āĻŦ⧇āώāĻŖ āϚāĻžāϞāĻŋāϝāĻŧ⧇ āϝāĻžāύ
  7. āĻĒā§āϰāĻžāϝāĻŧāĻļāχ āϜāĻŋāĻœā§āĻžāĻžāϏāĻŋāϤ āĻĒā§āϰāĻļā§āύāĻžāĻŦāϞ⧀

āĻ“āĻ­āĻžāϰāĻ­āĻŋāω

Its original paper describes causal dilated convolutions, a categorical distribution over quantized samples, and conditional speech generation from linguistic and speaker features.

āĻ—āĻ­ā§€āϰ āĻĄā§āĻŦ

WaveNet generates raw audio one sample at a time. The original paper uses causal dilated convolutions and an 8-bit Âĩ-law representation with a categorical distribution over 256 possible values. In conditional speech synthesis, linguistic and speaker features can condition the waveform generation. The paper reports results in its own speech and music experiments; it does not establish current product adoption or universal use in accessibility, singing or music generation.

āĻŦāĻžāĻ¸ā§āϤāĻŦ-āĻŦāĻŋāĻļā§āĻŦ āĻŦāĻžāĻ¸ā§āϤāĻŦāĻžāϝāĻŧāύ

Modeling waveform samples sequentially using prior samples as context.

Using causal dilated convolutions to expand receptive field.

Predicting a categorical distribution over 256 Âĩ-law-quantized sample values.

Generating speech waveforms conditioned on linguistic and speaker features in the paper’s TTS setup.

āĻ“āϝāĻŧ⧇āĻ­āύ⧇āĻŸā§‡āϰ āĻ­āĻŦāĻŋāĻˇā§āϝāϤ

Evaluate the original model on the intended audio task and hardware. Later systems such as parallel or recurrent vocoders require their own primary-source evidence; do not infer a current product roadmap from the WaveNet paper.

āĻāϟāĻžāϕ⧇ āĻŦāĻžāĻ¸ā§āϤāĻŦ⧇ āĻĒā§āϰāϝāĻŧā§‹āĻ— āĻ•āϰāĻž āĻšāĻšā§āϛ⧇

āϕ⧇āύ āĻāϟāĻž āϗ⧁āϰ⧁āĻ¤ā§āĻŦāĻĒā§‚āĻ°ā§āĻŖ

  • āĻ…ā§āϝāĻžāĻ•ā§āϏ⧇āϏ āĻāĻŦāĻ‚ āĻĒ⧌āρāĻ›āĻžāύ⧋. āĻāϟāĻŋ āĻŸā§āϰāĻžāĻ¨ā§āϏāĻ•ā§āϰāĻŋāĻĒāĻļāύ, āĻŦāĻ°ā§āĻŖāύāĻž āĻāĻŦāĻ‚ āĻ­āϝāĻŧ⧇āϏ āχāĻ¨ā§āϟāĻžāϰāĻĢ⧇āϏ⧇āϰ āĻŽāĻžāĻ§ā§āϝāĻŽā§‡ āĻ…ā§āϝāĻžāĻ•ā§āϏ⧇āϏāϝ⧋āĻ—ā§āϝāϤāĻž āωāĻ¨ā§āύāϤ āĻ•āϰ⧇āĨ¤
  • āĻ–āϰāϚ āĻ“ āĻŦāĻžāĻœā§‡āϟ. āĻŽāĻŋāĻĄāĻŋāϝāĻŧāĻž āĻĻāϞāϗ⧁āϞāĻŋ āϛ⧋āϟ āĻŦāĻžāĻœā§‡āĻŸā§‡āϰ āϏāĻžāĻĨ⧇ āĻĒāĻžāϞāĻŋāĻļ āĻ•āϰāĻž āĻ…āĻĄāĻŋāĻ“ āĻĻā§āϰ⧁āϤ āĻĒāĻžāĻ āĻžāϤ⧇ āĻĒāĻžāϰ⧇āĨ¤
  • āĻ—āϤāĻŋ āĻāĻŦāĻ‚ āĻ¸ā§āϕ⧇āϞ. āĻ—ā§āϰāĻžāĻšāĻ•-āĻŽā§āĻ–ā§€ āϏāĻŋāĻ¸ā§āĻŸā§‡āĻŽāϗ⧁āϞāĻŋ āĻŦ⧃āĻšāĻ¤ā§āϤāϰ āĻ¸ā§āϕ⧇āϞ⧇ āĻ•āĻĨā§āϝ āĻŽāĻŋāĻĨāĻ¸ā§āĻ•ā§āϰāĻŋāϝāĻŧāĻž āĻĒā§āϰāĻ•ā§āϰāĻŋāϝāĻŧāĻž āĻ•āϰāϤ⧇ āĻĒāĻžāϰ⧇āĨ¤

āϜāĻ¨ā§āϝ āϏāϤāĻ°ā§āĻ•

  • āϏāĻŽā§āĻŽāϤāĻŋ āĻ…āύ⧁āĻĒāĻ¸ā§āĻĨāĻŋāϤ āĻšāϞ⧇ āĻ­āϝāĻŧ⧇āϏ āĻ…āĻĒāĻŦā§āϝāĻŦāĻšāĻžāϰ āĻāĻŦāĻ‚ āĻ›āĻĻā§āĻŽāĻŦ⧇āĻļ⧇āϰ āĻā§āρāĻ•āĻŋ āĻŦ⧇āĻĄāĻŧ⧇ āϝāĻžāϝāĻŧāĨ¤
  • āωāĻšā§āϚāĻžāϰāĻŖ, āωāĻĒāĻ­āĻžāώāĻž āĻŦāĻž āϕ⧋āϞāĻžāĻšāϞāĻĒā§‚āĻ°ā§āĻŖ āĻĒāϰāĻŋāĻŦ⧇āĻļ⧇ āϝāĻĨāĻžāĻ°ā§āĻĨāϤāĻž āĻšā§āϰāĻžāϏ āĻĒ⧇āϤ⧇ āĻĒāĻžāϰ⧇āĨ¤
  • āϏāĻŋāĻ¨ā§āĻĨ⧇āϟāĻŋāĻ• āĻ…āĻĄāĻŋāĻ“ āĻĒāϰāĻŋāĻˇā§āĻ•āĻžāϰ āϞ⧇āĻŦ⧇āϞ āĻ›āĻžāĻĄāĻŧāĻž āĻ–āĻžāρāϟāĻŋ āĻŦāĻ•ā§āϤ⧃āϤāĻž āĻšāĻŋāϏāĻžāĻŦ⧇ āϭ⧁āϞ āĻšāϤ⧇ āĻĒāĻžāϰ⧇āĨ¤

āĻĒāϰāĻŦāĻ°ā§āϤ⧀ āĻĒāĻĻāĻ•ā§āώ⧇āĻĒ

  1. āĻ­āϝāĻŧ⧇āϏ āĻ•ā§āϝāĻžāĻĒāϚāĻžāϰ, āĻ•ā§āϞ⧋āύāĻŋāĻ‚ āĻāĻŦāĻ‚ āĻĒ⧁āύāσāĻŦā§āϝāĻŦāĻšāĻžāϰ⧇āϰ āϜāĻ¨ā§āϝ āϏ⧁āĻ¸ā§āĻĒāĻˇā§āϟ āϏāĻŽā§āĻŽāϤāĻŋ āĻĒāĻžāύāĨ¤
  2. āĻŦāĻŋāĻ­āĻŋāĻ¨ā§āύ āĻ¸ā§āĻĒāĻŋāĻ•āĻžāϰ āĻāĻŦāĻ‚ āĻŦā§āϝāĻžāĻ•āĻ—ā§āϰāĻžāωāĻ¨ā§āĻĄ āĻ•āĻ¨ā§āĻĄāĻŋāĻļāύ āϜ⧁āĻĄāĻŧ⧇ āĻŽāĻžāύ āĻĒāϰ⧀āĻ•ā§āώāĻž āĻ•āϰ⧁āύāĨ¤
  3. āĻ•āĻ–āύ āĻāĻ•āϜāύ āĻŽāĻžāύ⧁āώāϕ⧇ āφāωāϟāĻĒ⧁āϟ āĻĒāĻ°ā§āϝāĻžāϞ⧋āϚāύāĻž āĻŦāĻž āĻ…āύ⧁āĻŽā§‹āĻĻāύ āĻ•āϰāϤ⧇ āĻšāĻŦ⧇ āϤāĻž āύāĻŋāĻ°ā§āϧāĻžāϰāĻŖ āĻ•āϰ⧁āύāĨ¤
  4. āϏāĻŋāĻ¨ā§āĻĨ⧇āϟāĻŋāĻ• āĻ…āĻĄāĻŋāĻ“ āϞ⧇āĻŦ⧇āϞ āĻ•āϰ⧁āύ āĻāĻŦāĻ‚ āĻĻāĻžāϝāĻŧāĻŦāĻĻā§āϧāϤāĻžāϰ āϜāĻ¨ā§āϝ āĻŽā§‚āϞ āϰ⧇āĻ•āĻ°ā§āĻĄ āϰāĻžāϖ⧁āύāĨ¤

āĻ…āĻ¨ā§āĻŦ⧇āώāĻŖ āϚāĻžāϞāĻŋāϝāĻŧ⧇ āϝāĻžāύ

āĻĢā§āϰāĻŋ āύāĻŋāωāϜāϞ⧇āϟāĻžāϰ

āĻĒā§āϰāϤāĻŋāĻĻāĻŋāύ⧇āϰ āĻāφāχ āĻŦā§āϰāĻŋāĻĢāĻŋāĻ‚ āĻĒāĻžāύ

āĻĒā§āϰāϤāĻŋ āϏāĻĒā§āϤāĻžāĻšā§‡āϰ āĻ•āĻžāĻœā§‡āϰ āĻĻāĻŋāύ āϏāĻ•āĻžāϞ⧇ āϤāĻŋāύāϟāĻŋ āϝāĻžāϚāĻžāχāĻ•ā§ƒāϤ āĻāφāχ āĻ¸ā§āĻŸā§‹āϰāĻŋ, āϏāĻšāϜ āχāĻ‚āϰ⧇āϜāĻŋāϤ⧇ āϞ⧇āĻ–āĻžāĨ¤ āϚāĻŋāϰāĻ•āĻžāϞ⧇āϰ āϜāĻ¨ā§āϝ āĻĢā§āϰāĻŋ, āϕ⧋āύ⧋ āĻŦāĻŋāĻœā§āĻžāĻžāĻĒāύ āύ⧇āχāĨ¤

āĻĒā§āϰāϤāĻŋ āϏāĻĒā§āϤāĻžāĻšā§‡āϰ āĻ•āĻžāĻœā§‡āϰ āĻĻāĻŋāύ⧇ āĻāĻ•āϟāĻŋ āχāĻŽā§‡āϞāĨ¤ āĻāĻ• āĻ•ā§āϞāĻŋāϕ⧇āχ āφāύāϏāĻžāĻŦāĻ¸ā§āĻ•ā§āϰāĻžāχāĻŦ āĻ•āϰ⧁āύāĨ¤ āφāĻŽāϰāĻž āĻ•āĻ–āύ⧋āχ āφāĻĒāύāĻžāϰ āĻ āĻŋāĻ•āĻžāύāĻž āĻŦāĻŋāĻ•ā§āϰāĻŋ āĻŦāĻž āĻļā§‡ā§ŸāĻžāϰ āĻ•āϰāĻŋ āύāĻžāĨ¤

āύāĻŋāĻœā§‡āϕ⧇ āĻĒāϰ⧀āĻ•ā§āώāĻž āĻ•āϰ⧁āύ

Take the WaveNet quiz

āĻĒā§āϰāϤāĻŋāϟāĻŋ āωāĻ¤ā§āϤāϰ⧇āϰ āϤāĻžā§ŽāĻ•ā§āώāĻŖāĻŋāĻ• āĻĢāĻŋāĻĄāĻŦā§āϝāĻžāĻ•, āĻāĻŦāĻ‚ āϕ⧋āĻ°ā§āϏ āĻĒāĻžāϏ āĻ•āϰāĻžāϰ āĻĒāϰ āϝāĻžāϚāĻžāχāϝ⧋āĻ—ā§āϝ āφāχāĻĄāĻŋ āϏāĻš āĻāĻ•āϟāĻŋ āĻļā§‡ā§ŸāĻžāϰāϝ⧋āĻ—ā§āϝ āϏāĻžāĻ°ā§āϟāĻŋāĻĢāĻŋāϕ⧇āϟāĨ¤

āϕ⧁āχāϜ āĻļ⧁āϰ⧁ āĻ•āϰ⧁āύ

āĻĢā§āϰāĻŋ āĻāφāχ āĻļāĻŋāĻ•ā§āώāĻžāϕ⧇ āϏāĻŽāĻ°ā§āĻĨāύ āĻ•āϰ⧁āύāĨ¤ AI Understanding āĻāĻ•āϟāĻŋ 501(c)(3) āĻ…āϞāĻžāĻ­āϜāύāĻ• āĻĒā§āϰāϤāĻŋāĻˇā§āĻ āĻžāύ — āϕ⧋āύ⧋ āĻŦāĻŋāĻœā§āĻžāĻžāĻĒāύ āύ⧇āχ, āϕ⧋āύ⧋ āĻĒ⧇āĻ“ā§ŸāĻžāϞ āύ⧇āχ, āĻ•āĻ–āύ⧋āχ āύāĻžāĨ¤ āĻāĻ•āϟāĻŋ āĻĻāĻžāύ āĻ•āϰ⧁āύ

āĻĒā§āϰāĻžāϝāĻŧāĻļāχ āϜāĻŋāĻœā§āĻžāĻžāϏāĻŋāϤ āĻĒā§āϰāĻļā§āύāĻžāĻŦāϞ⧀

What is WaveNet?

WaveNet is an autoregressive model for raw audio waveforms. Its original paper describes causal dilated convolutions, a categorical distribution over quantized samples, and conditional speech generation from linguistic and speaker features.

āĻ•āĻŋāĻ­āĻžāĻŦ⧇ WaveNet āĻ…āĻĄāĻŋāĻ“ āϤ⧈āϰāĻŋ āĻ•āϰ⧇?

WaveNet āĻšāϞ āĻ…āĻŸā§‹āϰāĻŋāĻ—ā§āϰ⧇āϏāĻŋāĻ­: āĻāϟāĻŋ āφāϗ⧇ āφāϏāĻž āύāĻŽā§āύāĻžāϰ āωāĻĒāϰ āĻ­āĻŋāĻ¤ā§āϤāĻŋ āĻ•āϰ⧇ āĻĒā§āϰāϤāĻŋāϟāĻŋ āĻ…āĻĄāĻŋāĻ“ āύāĻŽā§āύāĻžāϰ āĻĒā§‚āĻ°ā§āĻŦāĻžāĻ­āĻžāϏ āĻĻ⧇āϝāĻŧāĨ¤

āĻ“āϝāĻŧ⧇āĻ­āύ⧇āĻŸā§‡āϰ āĻŽā§‚āϞ āĻĒāϰāĻŋāĻŦāĻ°ā§āϤāύāĻŽā§‚āϞāĻ• āωāĻĻā§āĻ­āĻžāĻŦāύ āϕ⧀?

āĻĒā§āϰāϏāĻžāϰāĻŋāϤ āĻ•āĻžāĻ°ā§āϝāĻ•āĻžāϰāĻŖ āĻ•āύāĻ­ā§‹āϞāĻŋāωāĻļāύāϗ⧁āϞāĻŋ āύ⧇āϟāĻ“āϝāĻŧāĻžāĻ°ā§āĻ•āϟāĻŋāϕ⧇ āĻ•āϝāĻŧ⧇āĻ• āĻšāĻžāϜāĻžāϰ āĻ…āϤ⧀āϤ⧇āϰ āύāĻŽā§āύāĻžāϗ⧁āϞāĻŋāϕ⧇ āĻĻāĻ•ā§āώāϤāĻžāϰ āϏāĻžāĻĨ⧇ āĻ•āĻ­āĻžāϰ āĻ•āϰāϤ⧇ āĻĻ⧇āϝāĻŧ āϝāĻ–āύ āĻļ⧁āϧ⧁āĻŽāĻžāĻ¤ā§āϰ āϏāĻŽāϝāĻŧ⧇āϰ āĻĒāĻŋāĻ›āύ⧇ āϤāĻžāĻ•āĻžāύ⧋ āĻšāϝāĻŧāĨ¤

How did the original WaveNet represent quantized waveform samples for prediction?

The original WaveNet paper uses 8-bit Âĩ-law companding and predicts a categorical distribution over 256 quantized audio values at each step.

āĻŽā§‚āϞ āĻ“āϝāĻŧ⧇āĻ­āύ⧇āĻŸā§‡āϰ āĻāĻ•āϟāĻŋ āĻŦāĻĄāĻŧ āĻŦā§āϝāĻŦāĻšāĻžāϰāĻŋāĻ• āĻ¤ā§āϰ⧁āϟāĻŋ āϕ⧀ āĻ›āĻŋāϞ?

āϝ⧇āĻšā§‡āϤ⧁ āĻĒā§āϰāϤāĻŋāϟāĻŋ āύāĻŽā§āύāĻž āĻĒā§‚āĻ°ā§āĻŦāĻŦāĻ°ā§āϤ⧀āϗ⧁āϞāĻŋāϰ āωāĻĒāϰ āύāĻŋāĻ°ā§āĻ­āϰ āĻ•āϰ⧇, āĻĒā§āϰāϜāĻ¨ā§āĻŽ āĻ›āĻŋāϞ āĻ•ā§āϰāĻŽāĻŋāĻ• āĻāĻŦāĻ‚ āϤāĻžāχ āϧ⧀āϰ, āϏāĻŽāĻžāĻ¨ā§āϤāϰāĻžāϞ āĻ“āϝāĻŧ⧇āĻ­āύ⧇āϟ āĻāĻŦāĻ‚ āĻ…āĻ¨ā§āϝāĻžāĻ¨ā§āϝ āωāĻ¤ā§āϤāϰāϏ⧂āϰāĻŋāĻĻ⧇āϰ āĻ…āύ⧁āĻĒā§āϰāĻžāĻŖāĻŋāϤ āĻ•āϰ⧇āĨ¤

In the speech-synthesis experiments described in the WaveNet paper, what does the model generate?

The WaveNet paper uses a conditional WaveNet to generate raw speech waveforms from linguistic and speaker conditioning features; it does not make the spectrogram the final output.

āĻļāĻŋāĻ–āϤ⧇ āĻĨāĻžāϕ⧁āύ

āϏāĻŽā§āĻĒāĻ°ā§āĻ•āĻŋāϤ āĻ—āĻžāχāĻĄ

āĻāχ āĻŦāĻŋāώāϝāĻŧ⧇āϰ āϜāĻ¨ā§āϝ āĻŦāĻžāĻ›āĻžāχ āĻ•āϰāĻž āφāϰāĻ“ āĻ—āĻžāχāĻĄ

āĻĒāϰāĻŦāĻ°ā§āϤ⧀ āφāĻĒāĻĒāϰāĻŦāĻ°ā§āϤ⧀ āĻ—āĻžāχāĻĄ

āĻ…āĻĄāĻŋāĻ“ āĻĄāĻŋāĻĒāĻĢ⧇āĻ• āϏāύāĻžāĻ•ā§āϤāĻ•āϰāĻŖ

āĻ…āĻĄāĻŋāĻ“ āĻāφāχ

āĻāĻ•āϟāĻŋ āĻŦāĻŋāύāĻžāĻŽā§‚āĻ˛ā§āϝ⧇ AI āĻļāĻ‚āϏāĻžāĻĒāĻ¤ā§āϰ āĻĒāĻžāύāĻāĻ•āϟāĻŋ āϏāĻ‚āĻ•ā§āώāĻŋāĻĒā§āϤ āĻŦāĻŋāύāĻžāĻŽā§‚āĻ˛ā§āϝ⧇ āϕ⧋āĻ°ā§āϏ āĻāĻŦāĻ‚ āĻĒāϰ⧀āĻ•ā§āώāĻž. āϕ⧋āύ⧋ āĻ–āϰāϚ āύ⧇āχ, āϕ⧋āύ⧋ āĻ•āĻžāĻ°ā§āĻĄ āύ⧇āχāĨ¤ āĻĒā§āϰāĻžāϝāĻŧ 2.5 āϘāĻ¨ā§āϟāĻžāĨ¤āĻŦāĻŋāύāĻžāĻŽā§‚āĻ˛ā§āϝ⧇ āĻļ⧁āϰ⧁ āĻ•āϰ⧁āύ →