Ƙwararren Ƙwararren-Tunani
Skeleton-of-Thought (SoT) dabara ce mai tunzurawa da yanke hukunci wacce ta fara tambayar ƙirar harshe don zayyana taƙaitaccen kwarangwal na maki amsa, sannan ta faɗaɗa kowane batu a layi daya.
Dubawa
It matters because it can cut the wall-clock latency of long answers by roughly 2x without retraining the model.
Zurfafa nutsewa
Manya-manyan ƙirar harshe galibi suna haifar da alama ɗaya a lokaci ɗaya, don haka doguwar amsa tana jinkirin kawai saboda kowace kalma tana jiran wanda ke gabanta. Skeleton-of-Thought, wanda masu bincike a Tsinghua suka gabatar da Microsoft a cikin 2023, ya sake fasalin aikin. Kira na farko yana tambayar ƙirar kwarangwal mai ƙima: jerin ƙididdiga na kanun maki 3 zuwa 10, kowanne kaɗan kaɗan. Kashi na biyu na kira yana faɗaɗa kowane batu da kansa kuma a lokaci ɗaya, saboda maki ba ya dogara da juna. An dinke abubuwan fadadawa tare zuwa amsa ta ƙarshe. Saboda jinkirin fadada matakin yana gudana a layi daya, jimlar latency yana faɗuwa sosai don tambayoyin da amsoshinsu a zahiri ke lalacewa zuwa sassa masu zaman kansu, kamar jeri nasiha ko kwatanta zaɓuɓɓuka.
Fahimtar Fasaha
SoT yana yin amfani da cewa ƙaddamar da ƙaddamarwa yana da latency, ba koyaushe ake ƙididdigewa ba: buƙatu ɗaya galibi tana barin GPU mara amfani. Faɗawar wurin gudu azaman tsari yana sa kayan aiki su shagaltu kuma suna mamaye tsara kowane lokaci. Tare da samfuran API, ana ba da faɗaɗawa azaman buƙatun lokaci guda; tare da samfura na gida, suna raba fas ɗin gaba guda ɗaya. Matakin kwarangwal yana ƙara ƙayyadadden ɗan gajeren sama, don haka saurin net ɗin yana girma tare da tsayin amsa da adadin maki masu zaman kansu.
Dabarun Tasiri
Gudu da sikelin
Gudun aikin harshe na iya tafiya da sauri ba tare da sadaukar da daidaito ba.
Shiga ku isa
Yana faɗaɗa damar shiga cikin harsuna da salon sadarwa.
Shawarwari masu haske
Ƙungiyoyi za su iya ciyar da ƙarin lokaci akan hukunci yayin da aiki da kai ke sarrafa maimaitawa.
Makomar kwarangwal-na-Tunani Daidaitacce
Yi tsammanin ra'ayoyin SoT don haɗawa zuwa hanyoyin daidaitawa: tsarin zai gano lokacin da tambaya ta lalace cikin tsafta kuma ta canza zuwa faɗaɗa layi ɗaya, komawa zuwa dalilai na jeri don ayyuka masu dogaro sosai kamar hujjojin lissafi. Bambance-bambancen kamar SoT tare da dogaron jadawali mai ƙarfi suna ba da damar maki waɗanda ke yin la'akari da juna. Kamar yadda tsarin ba da sabis na ƙara tallafin ƙaramar buƙatu na asali da ƙididdige ƙididdiga, dabarun lalata-daidaita za su zama madaidaicin layin rage jinkiri maimakon dabarar gaggawar hannu.
Aiwatar da Gaskiyar Duniya
Ƙaddamar da bot ɗin hira wanda ke amsa 'ba ni shawarwari 8 don rage farashin girgije' ta hanyar faɗaɗa duk shawarwari takwas lokaci guda.
Mataimakin goyon bayan abokin ciniki yana samar da ingantaccen jagorar warware matsalar yanki da yawa tare da ƙarancin jinkirin amsawa.
Samar da amsa kwatancen (riba da fursunoni na samfura biyu) inda kowane harsashi ke cika a lokaci guda.
Tsarukan ba da baya na ba da amsa ga sassan amsa masu zaman kansu don haɓaka amfani da GPU yayin tsara dogon tsari.
Hatsari & Tsare-tsare
Abubuwan da aka ruɗe suna iya shigar da rahotanni cikin nutsuwa, kwararar tallafi, ko abubuwan bincike.
Hankali na gaggawa na iya ƙirƙirar sakamako mara daidaituwa a cikin buƙatun iri ɗaya.
Za a iya fallasa bayanan rubutu mai ma'ana idan ikon samun dama yana da rauni.
Taswirar Hanya
Ƙayyade tsarin fitarwa, sautin, da ma'auni masu inganci kafin fitowa.
Amsa a ƙasa tare da amintattun tushe a duk lokacin da daidaito ya shafi mahimmanci.
Ajiye wurin binciken ɗan adam don abubuwan da ake samu masu girma.
Bibiyar tsarin gazawar kuma sake horar da tsokaci ko tafiyar aiki akai-akai.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Skeleton-of-Thought Parallel Decoding quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
MaskGIT Parallel Token Decoding
Tambayoyin da ake yawan yi
What is Skeleton-of-Thought Parallel Decoding?
Skeleton-of-Thought (SoT) dabara ce mai tunzurawa da yanke hukunci wacce ta fara tambayar ƙirar harshe don zayyana taƙaitaccen kwarangwal na maki amsa, sannan ta faɗaɗa kowane batu a layi daya. Yana da mahimmanci saboda yana iya yanke jinkirin agogon bango na dogon amsoshi ta kusan 2x ba tare da sake horar da ƙirar ba.
Menene matakin farko na kwarangwal-na-Thought ke samarwa?
SoT da farko ya sa samfurin ya fitar da kwarangwal na taken taken, wanda sai a fadada shi daban.
Me yasa matakin fadada batu zai iya gudana a layi daya?
An tsara wuraren kwarangwal don zama masu zaman kansu, don haka fadada su baya buƙatar jiran 'yan'uwa.
Menene babban maƙasudin ƙulli na SoT a cikin ƙaddamarwar LLM na al'ada?
Daidaitaccen ƙaddamarwa yana da latency saboda kowane alama yana jiran wanda ya gabata; SoT overlaps yana aiki don rage lokacin agogon bango.
Wace irin tambaya ce SoT ke ba da babbar gudu?
Amsoshin da ke lalacewa zuwa sassa masu zaman kansu da yawa suna amfana sosai, tunda kowane bangare yana faɗaɗa a lokaci guda.
Wane kari ne SoT ya ƙara idan aka kwatanta da tsara guda ɗaya?
Matakin kwarangwal yana ƙara ƙaramin tsayayyen jinkiri, wanda ya fi girma ta hanyar faɗaɗa layi ɗaya akan dogayen amsoshi.