Èdè AI Itọsọna
Encoder-Decoder Architectures
Encoder-decoder architectures map an input sequence to an output sequence that may differ in length.
Gbọ si itọsọna yiiAI-ṣe afihan · ~1 min
Lori iwe yi1 iseju kika
Akopọ
The encoder builds source representations, and the decoder generates target elements conditioned on those representations.
Jin Dive
Early seq2seq models used recurrent encoders and decoders. Bahdanau attention addressed the fixed-vector bottleneck by computing a source context for each decoder step. The Transformer replaces recurrence with self-attention and cross-attention. BART pretrains a denoising encoder-decoder, while T5 casts many language tasks as text-to-text. These papers evaluate specific models and tasks; the architecture alone does not guarantee quality.
Real-World imuse
A neural translation encoder represents a source sentence while a decoder generates a translated sentence.
Bahdanau attention lets each decoder step focus on different encoder states.
The Transformer uses encoder-decoder attention and masked decoder self-attention.
BART and T5 use encoder-decoder architectures for denoising or text-to-text tasks.
Ojo iwaju ti Encoder-Decoder Architectures
Choose an encoder-decoder when a task naturally maps one sequence to another, and compare it with alternatives on the target data and latency. Track input-length behavior and output quality. The cited work does not establish that this architecture is best for every language task.
Fifi o sinu iwa
Kini idi ti o ṣe pataki
- Iyara ati iwọn. Ṣiṣan iṣẹ ede le gbe ni iyara laisi irubọ aitasera.
- Wiwọle ati arọwọto. O faagun iraye si kọja awọn ede ati awọn aza ibaraẹnisọrọ.
- Awọn ipinnu diẹ sii. Awọn ẹgbẹ le lo akoko diẹ sii lori idajọ lakoko ti adaṣe n kapa atunwi.
Ṣọra fun
- Awọn otitọ ti a sọ di mimọ le tẹ awọn ijabọ sii ni idakẹjẹ, awọn ṣiṣan atilẹyin, tabi awọn abajade iwadii.
- Ifamọ kiakia le ṣẹda awọn abajade aisedede kọja awọn ibeere ti o jọra.
- Awọn data ọrọ ifarabalẹ le farahan ti awọn idari wiwọle ko lagbara.
Next awọn igbesẹ
- Ṣetumo ọna kika iṣẹjade, ohun orin, ati awọn iṣedede didara ṣaaju ṣiṣejade.
- Awọn idahun ilẹ pẹlu awọn orisun ti o gbẹkẹle nigbakugba ti deede ba ṣe pataki.
- Jeki aaye ayẹwo atunyẹwo eniyan fun awọn abajade ti o ga julọ.
- Tọpinpin awọn ilana ikuna ati tunṣe awọn itọsi tabi ṣiṣan iṣẹ nigbagbogbo.
Tesiwaju Ṣiṣawari
Ìwé ìròyìn ọfẹ
Gba ìkànìyàn AI ojoojúmọ́
Ìtàn AI mẹta tó jẹ́rìí ní gbogbo owurọ́ ọjọ́ iṣẹ́, kọ́wé ní Gẹ̀ẹ́sì tó rọrùn. Ọfẹ́ fún gbogbo àkókò, kò sí ìpolówó.
Ìmeèlì kan ní gbogbo ọjọ́ iṣẹ́. Yọkúrò ní ìkànìyàn ní ìkànìyàn kan. A kò ta tàbí pín àdírẹ́sì rẹ.
Ṣe idanwo fun ara rẹ
Take the Encoder-Decoder Architectures quiz
Ìdáhùn lẹ́sẹ̀kẹsẹ̀ lórí gbogbo ìdáhùn, àti iwe-ẹ̀rí tó lè pin pẹ̀lú ID tó lè jẹ́rìí lẹ́yìn tí o bá kọ́ ẹ̀kọ́.
Ṣe àtìlẹ́yìn ẹ̀kọ́ AI ọfẹ́. AI Understanding jẹ́ àjọ àìní èrè 501(c)(3) — kò sí ìpolówó, kò sí àkúnya owó, ní gbogbo àkókò. Ṣe ẹbun kan
Awọn ibeere ti a beere nigbagbogbo
Kini Encoder-Decoder Architectures?
Encoder-decoder architectures map an input sequence to an output sequence that may differ in length. The encoder builds source representations, and the decoder generates target elements conditioned on those representations.
Kini iṣẹ akọkọ ti kooduopo ni awoṣe koodu-iyipada kan?
Awọn kooduopo n gba gbogbo ọna titẹ sii ati ṣe agbejade awọn abala ọrọ-ọrọ ti o mu itumọ rẹ mu, eyiti oluyipada naa nlo.
Kilode ti olupilẹṣẹ ṣe lo boju-boju (okunfa) akiyesi ara ẹni?
Masking ṣe idaniloju ipo iṣelọpọ kọọkan nikan wa si awọn ipo iṣaaju, titoju aṣẹ iran autoregressive ti osi-si-ọtun.
Ilana wo ni o so kooduopo pọ mọ awọn aṣoju kooduopo?
Ifarabalẹ-agbelebu jẹ ki Layer decoder kọọkan ṣe ibeere awọn ipinlẹ ti o farapamọ koodu koodu, sisopọ oye ti igbewọle si iran ti iṣelọpọ.
Eyi ti awọn awoṣe jẹ ẹya kooduopo-decoder (akọọkan-si-ọkọọkan) faaji?
T5 (ati BART) jẹ awọn awoṣe encoder-decoder Ayebaye ti o ṣe awọn iṣẹ ṣiṣe bi ọrọ-si-ọrọ. BERT jẹ encoder-nikan ati GPT-3 jẹ decoder-nikan.
Kini idi ti apẹrẹ koodu-iyipada jẹ ibamu adayeba fun itumọ?
Awọn maapu itumọ ni ọna kan si ọkan ti o yatọ, nitorinaa kika gbogbo orisun (apoti) ati ṣiṣẹda ibi-afẹde tuntun (oluyipada) baamu ni pipe.
Tesiwaju kikọ
Jẹmọ awọn itọsọna
Awọn itọsọna diẹ sii ti a yan fun koko yii