Architecture encodeur-décodeur
Architecture encodeur-decodeur dafay xaaj model bi ñaari xaaj: benn bi dafay jàng ba noppi di kompresse ay dugal ci biir representation bu am solo, ak beneen biy defar ay output ci biir.
Résumé
This design powers translation, summarization, and any task where the input and output are different sequences.
Plongeur bu xóot
Benn xeetu encoder-decoder dafay saafara jafe-jafe ci ñaari pàcc. Encoder bi dafay jàng limu dugal yépp (nu wax, ab frase ci angale) ba noppi mu soppi ko ci ay vecteur contextuel yuy jàpp luñuy tekki. Dekodeer bi dafay génne limu génne (maanaam, français) benn-benn, xoolaat limu njëkka génne ak li enkodeer bi wane. Transformer 2017 bi njëkk, enkodeer-dekodeer la woon buñ defar ngir tekki làkk. Modèle yu melni T5 ak BART dañuy jëfandikoo jëmm jii, ba noppi ñu kaadre bépp liggéey ni bind-dugg, bind-génn. Xaajale bi amna doole ndax encoder bi mën na gis duggal bi yépp benn yoon (contexte bidirectionnel), fekk decoder bi dafay defar cammoy-ndeyjoor. Loolu mooy tax konseptioŋ bi méngoo bu baax ak jafe-jafe yi ci toppalante, ndax guddaayi li ñuy génne ak ëmbiit li wuute ak li ñuy dugal.
Gis-gis xarala
Encoder bi dafay jëfandikoo self-attention bu am ñaari yoon, suko defee bépp token buñ dugal dafay bàyyi xel ci beneen token bi ci saasi. Decodeur bi autorégresif la, te dafay jëfandikoo maskewu boppam, loolu dafay tekki ni position bu nekk mën na gis position yu njëkk yi rek ngir mëna am generation causal. boole leen mooy cross-attention: couche decodeur yi dañuy laaj stade yu mujj yu nëbbu yi ci encodeur bi. Tàqale bi dafay may encoder bi mu tabax xam-xam bu mat te amul benn yoon, fekk decoder bi dafay jëfandikoo benn token benn yoon.
njeextalu pexe
Gaawaay ak yaatuwaay
Liggéeyukaay yi ci làkk yi mën nañu gëna gaaw te duñu yàq deggoo gi.
Dugg ak yegg
Dafay yaatal jëfandikoo gi ci làkk yi ak ci anam yi ñuy jokkoo.
dogal yu gëna leer
Ekip yi mën nañu gëna yàgg ci àtte ci jamono ji otomatisation di liggéey ci baamtu.
Ëlëgu arsitektir enkodeer-dekodeer
Modèle decoder-only yu melni GPT leegi ñoo ëpp doole ci waxtaan yi ndax benn stack dafay yombal ba noppi di def liggéey yu bari jaaraleko ci laaj. Waaye jëmmal encoder-decoder dafay wéy fu xam-xam bi dugg ak génne yi wuute dëgg: xàmmee kàddu (Whisper), tënk dokimaa, ak sistem multimodal yuy boole encoder vision ak decoder bind. Xaarandil architecture hybrid yuy leble xam-xam bi encoder bi am ci ñaari yoon ngir seet ak grounding ci noonu lañuy tëye decoder bi, rawatina bi model yi di boole bind, audio ak nataal.
Doxal ci àdduna dëgg
Google Tekki ak DeepL dañuy jëfandikoo enkodeer-dekodeer Transformatër ngir méngale ab frase ci benn làkk ak beneen.
__AIU_PROTECTED_10_'s Whisper dafay kode spectrogram audio yi ba noppi dekode leen ñu nekk mbind buñ bind wala tekki.
T5 ak BART dañu am doole ci tënk abstrak, boole xët yu gudd yi ci tënk yu gàtt.
Sistem yiy joxe sous-titre ci nataal yi dañuy boole encodeur vision ak decodeur text ngir mëna joxe leeral ci nataal yi ci kàddu.
Risk yi ak balustrade yi
Lépp lu jaarul yoon mën na dugg ci rapoor yi, jàppale ci liggéey bi, wala ci njariñu gëstu bi.
Sensibilite bu gaaw mën na jur njariñ yu wuute ci laajte yu noonu mel.
Done yu am solo mën nañu feeñ sudee seytu jëfandikoo gi néew doole.
Roadmap ngir samp gi
Mandargal formaa génne gi, melokaan bi, ak standard kalite yi laata ngay dugal ko.
Tontu yu am solo ak balluwaay yu wóor saa yu dëggu bi di am solo.
Fexeel am barabu xool nit ñi ngir am njariñ yu am solo.
Toppal anami gacce yi ak di faral di tàggataat ay laaj wala def-liggéey.
Weyal di banneexu
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Encoder-Decoder Architectures quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gis bi ci topp
Encodeurs croisés vs Bi-encodeurs
Laaj yi ñuy faral di laaj
What is Encoder-Decoder Architectures?
Architecture encodeur-decodeur dafay xaaj model bi ñaari xaaj: benn bi dafay jàng ba noppi di kompresse ay dugal ci biir representation bu am solo, ak beneen biy defar ay output ci biir. Design bii dafay jàppale tekki làkk, tënk, ak bépp liggéey bu dugal ak genn wuute ci toppalante.
Lan mooy liggéey bi gëna am solo ci encoder ci xeetu encoder-decoder?
Encoder bi dafay lekk limu dugal yépp ba noppi defar ay vecteur contextuel yuy jàpp li muy tekki, decoder bi daal di koy jëfandikoo.
Lu tax dekodeer bi di jëfandikoo maske (causal) bàyyi xel ci boppam?
Maskering dafay fexe ba position bu nekk ci genn gi nekk ci position yi ko jiitu rek, ba noppi baña yàq doxalinu generation autoregressive bu cammoñ bi dem ci ndijoor ji.
Ban mekaniism mooy boole dekodeer bi ak misaali enkodeer bi?
Cross-attention dafay may bépp couche decodeur mu laaj stade yu nëbbu yi ci encodeur bi, muy lëkkale xam-xam bi ci dugal ak defar ci génne gi.
Ci xeetu architecture yii, yan ci ñoom mooy architecture encodeur-decodeur (toppal-ci-toppal)?
T5 (ak BART) ay xeetu encoder-decoder yu yàgg lañu yuy kaadre liggéey yi ni bind-ci-bind. BERT encodeur kese la, GPT-3 decodeur kese la.
Lan moo waral jëmmu encoder-decoder bi méngoo bu baax ak tekki làkk?
Tekki làkk dafay méngale benn toppalante ak beneen bu wuute, kon jàng source bi yépp (encoder) ak defar target bu bees (decoder) mën nañu ko méngale bu baax.