AudioGen синтез на текст към аудио

AudioGen: Textually Guided Audio Generation studies a model that predicts learned discrete acoustic representations from descriptive text. Its general-audio setting includes environmental events and overlapping sources, differing from a system devoted to accurate transcript-to-speech rendering. A learned audio autoencoder connects waveforms with discrete token sequences; the language model predicts tokens, and the codec decoder reconstructs waveform audio from them. An autoregressive transformer uses earlier acoustic tokens and text conditioning when producing subsequent predictions. This gives a different generation procedure from a parallel masked acoustic model or continuous latent diffusion. The original paper’s augmentation mixes two audio samples at a sampled relative offset and signal-to-noise ratio, then concatenates their text captions. The audio is mixed while the captions are joined; describing this as simply concatenating waveform clips would miss the stated procedure. The authors study whether that augmentation helps composition of several described sources. Their classifier-free guidance combines conditional and unconditional model distributions in the stated log-probability formulation. This gives a tunable quality/diversity tradeoff in the experiments rather than a universal setting that makes every sound faithful. The study also investigates audio continuation, using an initial audio prompt with or without additional text conditioning. Continuing an acoustic prefix differs from starting only with a text caption. AudioGen is documented in AudioCraft alongside other audio research tools. The published paper and a later implementation can use different configurations, so codec names, model sizes, sampling rules, and resource requirements should be associated with the actual version being evaluated.