Maswali yanayounganishwa na mwongozo · Kati Kiwango

Maswali ya Kutoweka na Kulipuka kwa Gradients

Hukagua bidhaa za gradient ya kanuni ya mnyororo, kueneza, urefu wa mfuatano unaojirudia, njia za mabaki na mipaka ya heuristics ya spectral-radius.

Njia za mwongozo zinazohusianaKutoweka na Kulipuka Gradients
Swali 1 ya 8

Ni operesheni gani ya kihesabu katika uenezaji wa nyuma ndio sababu kuu ya kutoweka na kulipuka kwa gradients?

Hakiki kila swali8 maswali

Majibu na maelezo huonekana unapojibu maswali.

  1. Ni operesheni gani ya kihesabu katika uenezaji wa nyuma ndio sababu kuu ya kutoweka na kulipuka kwa gradients?
    • Kurudia kuzidisha vipengele kwa kila safu kupitia kanuni ya mnyororo
    • Kuongeza gradients kwenye safu moja
    • Kugawanya hasara kwa saizi ya kundi
    • Kuchukua mizizi ya mraba ya uzani
  2. Why can sigmoid or tanh units contribute to vanishing gradients when they are saturated?
    • Their derivatives are always larger than one
    • Their derivatives become small in saturated regions, so repeated chain-rule products can shrink
    • They remove the chain rule from backpropagation
    • They force every weight to zero
  3. Je, ni usanifu gani ambao umeathiriwa zaidi na matatizo ya upinde rangi juu ya mlolongo mrefu?
    • Perceptron ya safu mbili
    • Urejeshaji wa mstari mmoja
    • Mtandao wa kawaida wa neva ulifunguliwa kwa hatua nyingi za nyakati
    • Jedwali la kutazama
  4. Kuona upotezaji wa mafunzo unabadilika ghafla kuwa NaN uwezekano mkubwa unaonyesha ni shida gani?
    • Gradients zinazopotea
    • Muunganiko kamili
    • Kiwango kidogo sana cha kujifunza
    • Gradients zinazolipuka
  5. Je, miunganisho ya mabaki (ya ruka) husaidia vipi na gradient zinazopotea?
    • Huipa upinde rangi njia ya mkato ya moja kwa moja kurudi nyuma, ikikwepa kuzidisha kupungua
    • Wanaondoa yote yasiyo ya mstari
    • Wao huongeza kiwango cha kujifunza moja kwa moja
    • Wanafuta tabaka za mapema
  6. In a simplified analysis of a recurrent linear transformation repeated over many steps, what matrix quantity is a rough signal of growth versus decay?
    • The matrix trace relative to the batch size
    • The spectral radius relative to one
    • The number of hidden units alone
    • The determinant relative to the learning rate
  7. A backpropagated gradient is multiplied by a factor below one at each of many layers. What trend should you expect?
    • It must become exactly one
    • It grows exponentially regardless of the factor
    • The signal can shrink exponentially with depth
    • It stops depending on earlier layers after one step
  8. Why can a recurrent network be especially vulnerable over a long sequence?
    • The network has no hidden state
    • The output layer is used only once
    • The sequence is processed without a chain rule
    • The recurrent transformation is applied repeatedly across timesteps, compounding its gradient effect