Hankalin Tambaya-Rukuni
Hankalin Rukuni-Tambaya (GQA) hanya ce ta rage ƙwaƙwalwar da ake buƙata yayin tsara rubutu ta barin shugabannin tambaya da yawa su raba maɓalli ɗaya da shugabannin ƙima.
Dubawa
It makes large models much faster to serve with almost no quality loss.
Zurfafa nutsewa
A cikin madaidaitan kula da kai mai yawan kai, kowane kai yana da nasa tambayoyin, maɓallai, da ƙima. A lokacin tsarawa, maɓallai da ƙimar duk alamun da suka gabata ana adana su ('KV cache') don haka ƙirar baya ƙididdige su. Tare da shugabannin da yawa da dogayen mahallin, wannan cache ya zama babba kuma yana mamaye bandwidth na ƙwaƙwalwar ajiya a lokacin ƙididdigewa. GQA, wanda Google masu bincike suka gabatar a cikin 2023, suna tattara shugabannin tambayar kuma suna ba kowane rukuni saitin maɓalli da ƙima ɗaya. Idan kuna da shugabannin tambaya 32 amma ƙungiyoyin KV 8 kawai, ma'ajin KV ɗin yana raguwa kusan ninki huɗu. Wannan yana zaune tsakanin cikakken kulawa da kai (kowane kai daban) da kulawar tambaya da yawa (wanda aka raba KV don duk shugabannin), yana ɗaukar mafi yawan saurin MQA yayin kiyaye inganci kusa da cikakkiyar hankali. Llama 2 70B kuma yawancin samfura daga baya sun karbe shi.
Fahimtar Fasaha
Ingantacciyar kulawa ya dogara sosai akan samun takamaiman kwatancen tambaya, amma yana jure wa raba maɓalli da ƙima. GQA tana amfani da wannan asymmetry: tana kiyaye duk shugabannin tambaya amma tana maimaita kowane shugaban KV da aka raba a cikin tambayoyin da ke cikin rukuninsa. Ajiye ya zo ne a cikin mahimmanci, inda KV cache shine babban mabukaci na bandwidth na ƙwaƙwalwar ajiya; Ƙananan shugabannin KV suna nufin ƙarancin bayanai don karanta kowace alamar da aka samar. Sau da yawa ana 'ɗayan samfura' a taƙaice don canza wurin bincike mai yawan kai zuwa GQA ɗaya.
Dabarun Tasiri
Gudu da sikelin
Gudun aikin harshe na iya tafiya da sauri ba tare da sadaukar da daidaito ba.
Shiga ku isa
Yana faɗaɗa damar shiga cikin harsuna da salon sadarwa.
Shawarwari masu haske
Ƙungiyoyi za su iya ciyar da ƙarin lokaci akan hukunci yayin da aiki da kai ke sarrafa maimaitawa.
Makomar Hankalin Rukuni-Tambaya
GQA is now a standard default in open-weight models because it cleanly trades a tiny quality cost for big serving wins. Yi tsammanin zai haɗu da haɓaka tare da wasu dabaru masu inganci kamar FlashAttention, KV-cache quantization, da sabbin tsare-tsare irin su latent hankali da yawa waɗanda ke damfara cache har ma da ƙari. Kamar yadda mahallin windows ke girma, sarrafa girman KV-cache zai zama babban matsalar ƙira, kuma raba kai-style GQA zai kasance maɓalli mai mahimmanci.
Aiwatar da Gaskiyar Duniya
Llama 2 70B da Llama 3 suna amfani da GQA don yin aiki na dogon lokaci tare da ƙaramin cache na KV
Rage žwažwalwar ajiyar GPU don haka babban samfurin taɗi ya dace da ƴan ƙaranci ko rahusa accelerators
Ƙaddamar da haɓaka alamar-ta-alama a cikin samar da APIs inda KV-cache bandwidth shine ƙulli
Ba da damar girma dabam dabam don hidimar masu amfani da yawa lokaci guda ba tare da gajiyar ƙwaƙwalwar ajiya ba
Hatsari & Tsare-tsare
Abubuwan da aka ruɗe suna iya shigar da rahotanni cikin nutsuwa, kwararar tallafi, ko abubuwan bincike.
Hankali na gaggawa na iya ƙirƙirar sakamako mara daidaituwa a cikin buƙatun iri ɗaya.
Za a iya fallasa bayanan rubutu mai ma'ana idan ikon samun dama yana da rauni.
Taswirar Hanya
Ƙayyade tsarin fitarwa, sautin, da ma'auni masu inganci kafin fitowa.
Amsa a ƙasa tare da amintattun tushe a duk lokacin da daidaito ya shafi mahimmanci.
Ajiye wurin binciken ɗan adam don abubuwan da ake samu masu girma.
Bibiyar tsarin gazawar kuma sake horar da tsokaci ko tafiyar aiki akai-akai.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Grouped-Query Attention quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Hankalin Tambaya da yawa
Tambayoyin da ake yawan yi
What is Grouped-Query Attention?
Hankalin Rukuni-Tambaya (GQA) hanya ce ta rage ƙwaƙwalwar da ake buƙata yayin tsara rubutu ta barin shugabannin tambaya da yawa su raba maɓalli ɗaya da shugabannin ƙima. Yana sa manyan samfura da sauri don yin hidima tare da kusan babu asarar inganci.
A cikin Hankalin Tambaya-Rukuni, menene aka raba tsakanin rukunin shugabannin tambaya?
GQA yana adana shugabannin tambaya daban amma yana barin kowane rukuni daga cikinsu ya raba saitin maɓalli da ƙima, yana raguwar cache na KV.
An fi kwatanta GQA a matsayin tsaka-tsaki tsakanin wace matsananci biyu?
Multi-head yana ba kowane shugaban KV kansa; Multi-questery yana raba KV ɗaya don duk shugabannin; GQA yana zaune tsakanin ƴan KV ƙungiyoyi.
Wane bangare na ra'ayi ne GQA da farko ya fi rahusa?
Adadin yana nunawa a lokacin tsarawa, inda karanta cache na KV shine babban ƙimar ƙwaƙwalwar ajiya-bandwidth.
Idan samfurin yana da shugabannin tambaya 32 da ƙungiyoyin KV 8, ana rage ma'ajin KV da kusan wane dalili?
Shuwagabannin tambaya guda 32 da ƙungiyoyin KV guda 8 suka raba suna ba da kusan ninki huɗu a cikin maɓallan da aka adana da ƙima.
Me yasa GQA zai iya adana yawancin ingancin samfurin duk da raba kawunan KV?
Hankali yana jure wa maɓalli da ƙima mafi kyau fiye da yadda yake jurewa rasa takamaiman kwatancen tambaya, don haka GQA yana kiyaye inganci mai girma.