Gumbel-Softmax na Reparameterization
Gumbel-Softmax bụ aghụghọ nke na-ahapụ netwọkụ akwara ozi 'nlere anya' site na ngalaba dị iche iche ebe a ka na-azụ ya site na mgbada gradient.
Nchịkọta
Ọ dị mkpa n'ihi na ịgbasa azụ azụ enweghị ike isi na nhọrọ enweghị usoro.
Ime miri emi
Netwọk akwara na-amụta site na izipu gradients azụ site na ọrụ ọ bụla. Mana ịlele ụdị pụrụ iche (dị ka ịhọrọ okwu #7 nke 50,000) bụ ihe siri ike, nke na-enweghị isi, yabụ gradients na-anwụ ebe ahụ. The reparameterization aghụghọ rewrite random sampling mere randomness na-abịa site a ofu mpụta mkpọtụ isi iyi, na-ahapụ a ire ụtọ, dị iche iche ụzọ gradients. Gumbel-Softmax na-emetụta nke a na ụdị mgbanwe dị iche iche: ọ na-agbakwụnye mkpọtụ nkesa Gumbel na logits, wee dochie argmax siri ike na softmax na-achịkwa okpomọkụ. N'ebe okpomọkụ dị elu, a na-emepụta ihe na-egbuke egbuke n'elu edemede; ka okpomọkụ na-adaba na efu, ọ na-amụba n'ebe dị nso na vector dị ọkụ, na-enweta ezi nlele mgbe ọ na-anọpụ iche n'oge niile.
Nghọta nka nka
Aghụghọ Gumbel-Max na-ekwu: ịgbakwunye mkpọtụ Gumbel (0,1) nọọrọ onwe ya na logit ọ bụla na iwere argmax na-enye ezigbo nlele site na nkesa softmax. Gumbel-Softmax gbanwere argmax siri ike maka softmax ((log p + g)/tau). The okpomọkụ tau interpolates n'etiti ire ụtọ, elu-entropy nkesa (nnukwu tau) na nso-pụrụ iche otu-ekpo ọkụ (obere tau). N'ihi na a na-esepụta mkpọtụ g na mpụga netwọkụ, ụzọ site na logits gaa na mmepụta ga-adị iche.
Mmetụta atụmatụ
Ọnụ ego na mmefu ego
Mkpebi ihe owuwu ụlọ na-akwalite arụmọrụ yana ọnụ ahịa ọrụ ruo ọtụtụ afọ.
Mkpebi doro anya
Nkà mmụta nka na-enyere ndị otu egwuregwu aka ịhọrọ nchịkọta ziri ezi, ọ bụghị naanị nke kachasị ọhụrụ.
Quality akara
Nhọrọ injinia ka mma na-ebelata ihe omume ntụkwasị obi na mmepụta.
Ọdịnihu nke Gumbel-Softmax na Reparameterization
Gumbel-Softmax ka bụ ngwá ọrụ ndabara maka mgbanwe dị iche iche latent dị iche, nyocha ụlọ dị iche iche, ụdị vector-quatized, yana ụzọ mmụta mmụta na sistemụ ndị ọkachamara. Nchọpụta na-aga n'ihu na nhụsianya dị ala, ntụrụndụ dị ala (dị ka Rao-Blackwellized na njikwa-iche estimators) yana n'usoro ihe nhụsianya nke na-edozi nhụsianya nke okpomọkụ na-ekpo ọkụ megide nnukwu gradient mgbanwe nke oyi. Dị ka ụdị na-esiwanye ike na-eme mkpebi doro anya, na-atụ anya ka ezumike ndị a na-aga n'ihu ga-abụ isi n'ime nhọrọ ndị dị otú ahụ a ga-amụta site na njedebe ruo ọgwụgwụ.
Mmejuputa n'ezie n'ụwa
Ọzụzụ variational autoencoders nwere categorical (pụrụ iche) koodu latent kama naanị ndị Gaussian na-aga n'ihu.
Ọchụchọ ihe owuwu akwara dị iche (dịka ọmụmaatụ, ụzọ ụdị DARTS) na-ahọpụta ọrụ a ga-etinye na oyi akwa ọ bụla.
Ịmụta nhọrọ akwụkwọ koodu pụrụiche n'ụdị VQ na ụdị nnochite anya pụrụ iche.
Mkpebi ụzọ ụzọ dị iche iche ma ọ bụ gating dị na ngwakọta-nke-ọkachamara na netwọọdụ mgbakọ na mwepụ.
Ihe ize ndụ & okporo ụzọ nche
Ịkwalite otu akara ngosi nwere ike zoo adịghị ike sistemụ sara mbara.
A na-eledakarị ihe akụrụngwa na ụgwọ ọrụ anya.
Ọdịiche nchekwa na nleba anya nwere ike itolite ka sistemu na-adịwanye mgbagwoju anya.
Map mmejuputa
Kọwaa latency, ịdịmma na ebumnuche ọnụ ahịa tupu mmejuputa ya.
Benchmark n'okpuru ibu dị adị na ọnọdụ data.
Nleba anya akụrụngwa maka mperi, ịkpafu na mmetụta onye ọrụ.
Kwadebe ụzọ nzaghachi azụghachi azụ na ihe omume tupu ịchachaa.
Nọgide na-eme nchọpụta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Gumbel-Softmax and Reparameterization quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ntuziaka na-esote
Netwọk ugboro ugboro abụọ
Ajụjụ a na-ajụkarị
Kedu ihe bụ Gumbel-Softmax na Reparameterization?
Gumbel-Softmax bụ aghụghọ nke na-ahapụ netwọkụ akwara ozi 'nlere anya' site na ngalaba dị iche iche ebe a ka na-azụ ya site na mgbada gradient. Ọ dị mkpa n'ihi na ịgbasa azụ azụ enweghị ike isi na nhọrọ enweghị usoro.
Kedu isi nsogbu Gumbel-Softmax na-edozi?
Nlere anya pụrụ iche site na argmax abụghị ihe dị iche, na-egbochi gradients. Gumbel-Softmax na-enye ntụrụndụ dị iche iche ka netwọk wee nwee ike ịzụ ya site na njedebe ruo ọgwụgwụ.
Na aghụghọ reparameterization, kedụ ka randomness si abịa?
Ndozigharị na-akpali enweghị usoro gaa na mgbanwe mkpọtụ nọọrọ onwe ya, na-ahapụ ọrụ njiri mara, ọrụ dị iche iche nke paramita netwọkụ.
Dị ka aghụghọ Gumbel-Max si dị, kedu ka ị ga-esi nweta ezigbo nlele site na nkesa softmax?
A na-ekesa aghụghọ Gumbel-Max: argmax over (logits + i.d. Gumbel mkpọtụ) kpọmkwem dị ka ihe atụ categorical si softmax nke logit ndị ahụ.
Kedu ọrụ nke oke okpomọkụ (tau) na Gumbel-Softmax?
Low tau na-akwali softmax n'ebe dị nso otu vector na-ekpo ọkụ (nke dị nso na nlele ezi); elu tau na-eme ka ọ dị nro na elu-entropy. Ọ na-azụta echiche ọjọọ megide mgbanwe gradient.
Ka tau na-abịaru nso efu, ihe nrụpụta Gumbel-Softmax na-abịaru nso gịnị?
Ime ka ọnọdụ okpomọkụ dị n'ebe efu na-amụba softmax ruo mgbe ọ fọrọ nke nta ka ọ họrọ otu otu, na-eweghachi omume nlere anya pụrụ iche.