Skeleton-y-Igitekerezo Iringaniza Decoding
Skeleton-y-Ibitekerezo (SoT) nubuhanga bwihuse kandi bwa decoding ibanza gusaba icyitegererezo cyururimi kwerekana skeleti ngufi yibisubizo, hanyuma ikagura buri ngingo muburyo bubangikanye.
Incamake
It matters because it can cut the wall-clock latency of long answers by roughly 2x without retraining the model.
Kwibira cyane
Ururimi runini rusanzwe rutanga ikimenyetso kimwe icyarimwe, igisubizo kirekire rero kiratinda gusa kuberako buri jambo ritegereza rimwe mbere yaryo. Skeleton-y-Igitekerezo, yatangijwe nabashakashatsi bo muri Tsinghua na Microsoft muri 2023, ivugurura imirimo. Ihamagarwa ryambere risaba icyitegererezo kuri terse skeleton: urutonde rufite numero 3 kugeza 10 kumutwe, buri jambo rito. Icyiciro cya kabiri cyo guhamagara noneho cyagura ingingo yose yigenga kandi icyarimwe, kuko ingingo ntizishingiye kuri mugenzi we. Kwaguka byashizwe hamwe mubisubizo byanyuma. Kuberako intambwe yo kwaguka gahoro ikora muburyo bubangikanye, ubukererwe bwaragabanutse cyane kubibazo ibisubizo bisanzwe bibora mubice byigenga, nkurutonde rwinama cyangwa kugereranya amahitamo.
Ubushishozi
SoT ikoresha iyo myanzuro ya decoder ihujwe nubukererwe, ntabwo buri gihe iba ibarwa: icyifuzo kimwe akenshi gisiga GPU idakoreshwa. Kwiruka kwaguka nkicyiciro gikomeza ibyuma bikora kandi bigahuza kuri buri gisekuru. Hamwe na moderi ya API, kwaguka gutangwa nkibisabwa hamwe; hamwe nicyitegererezo cyaho, basangiye icyiciro kimwe imbere. Icyiciro cya skeleton cyongeweho kigufi hejuru, bityo net yihuta ikura hamwe nuburebure bwibisubizo n'umubare w'ingingo zigenga.
Ingaruka z'Ingamba
Umuvuduko n'igipimo
Ururimi rwakazi rushobora kugenda byihuse nta gutamba guhuzagurika.
Kugera no kugera
Yagura uburyo bwindimi nuburyo bwo gutumanaho.
Ibyemezo bisobanutse
Amakipe arashobora kumara umwanya munini murubanza mugihe automatike ikora gusubiramo.
Kazoza ka Skeleton-y-Igitekerezo Kuringaniza Decoding
Tegereza ibitekerezo bya SoT kugirango bihuze muburyo bwo guhuza n'imihindagurikire y'ikirere: sisitemu izamenya igihe ikibazo cyangirika neza kandi gihindurwe no kwaguka, gusubira mubitekerezo bikurikiranye kubikorwa bishingiye cyane nkibimenyetso by'imibare. Ibihinduka nka SoT hamwe nigishushanyo mbonera cya graphique yemerera ingingo zikorana. Nka serivise yo kongeramo kavukire-sub-gusaba inkunga hamwe na decoding idasanzwe, ingamba zibangikanye-gusenyuka zizahinduka urwego rusanzwe rwo kugabanya ubukererwe aho kuba amayeri yihuse.
Gushyira mu bikorwa Isi
Kwihutisha ikiganiro gisubiza 'mpa inama 8 zo kugabanya ibiciro byigicu' mugura inama umunani zose icyarimwe.
Umukiriya-wunganira umufasha kubyara ibyiciro byinshi byuburyo bwo gukemura ibibazo hamwe nubukererwe buke.
Gutanga igisubizo cyo kugereranya (ibyiza nibibi byibicuruzwa bibiri) aho buri sasu ryuzuyemo icyarimwe.
Inyuma ya sisitemu itanga ibice byigenga kugirango uzamure imikoreshereze ya GPU mugihe kirekire.
Ingaruka & Kurinda
Ibintu bifatika bishobora kwinjiza bucece raporo, gushyigikira imigendekere, cyangwa ibisubizo byubushakashatsi.
Kwihuta byihuse birashobora gukora ibisubizo bidahuye mubisabwa bisa.
Ibyanditswe byumvikana birashobora kugaragara niba kugenzura kugenzura ari ntege.
Igishushanyo mbonera
Sobanura imiterere isohoka, amajwi, hamwe nubuziranenge mbere yo gutangira.
Ibisubizo byibanze hamwe nisoko yizewe igihe cyose ukuri kwingirakamaro.
Komeza kugenzura abantu kugenzura ibisubizo byinshi.
Kurikirana uburyo bwo kunanirwa no kongera imyitozo cyangwa akazi gahoraho.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Skeleton-of-Thought Parallel Decoding quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
MaskGIT Iringaniza Token Decoding
Ibibazo bikunze kubazwa
What is Skeleton-of-Thought Parallel Decoding?
Skeleton-y-Ibitekerezo (SoT) nubuhanga bwihuse kandi bwa decoding ibanza gusaba icyitegererezo cyururimi kwerekana skeleti ngufi yibisubizo, hanyuma ikagura buri ngingo muburyo bubangikanye. Nibyingenzi kuko irashobora guca urukuta-isaha yubukererwe bwibisubizo birebire hafi 2x utarinze kwitoza icyitegererezo.
Niki cyiciro cya mbere cya Skeleton-y-Ibitekerezo bitanga?
SoT ibanza gusaba icyitegererezo cyo gusohora terse skeleton yimitwe yimitwe, hanyuma ikagurwa ukwayo.
Ni ukubera iki icyiciro cyo kwagura icyiciro gishobora kugereranywa?
Ingingo ya skeleton yagenewe kwigenga, kubwagura rero ntibisaba gutegereza abavandimwe.
Niyihe ntego nyamukuru SoT igamije muri decoding isanzwe ya LLM?
Ubusanzwe decoding ihujwe nubukererwe kuko buri kimenyetso gitegereza icyambere; SoT ihuza akazi kugirango igabanye igihe-isaha.
Ni ubuhe bwoko bw'ikibazo SoT itanga umuvuduko mwinshi?
Ibisubizo bibora mubice byinshi byigenga byunguka byinshi, kuva buri gice cyaguka icyarimwe.
Ni ubuhe buryo bwo hejuru SoT yongeyeho ugereranije nigisekuru kimwe gikurikiranye?
Icyiciro cya skeleton kongeramo akantu gato gahoro gahoro, karengerwa no kwaguka kuburinganire kubisubizo birebire.