Idanwo ti o sopọ mọ itọsọna · Alabọde Ipele

Yiyan ati bugbamu Gradients adanwo

Checks chain-rule gradient products, saturation, recurrent sequence length, residual paths and the limits of spectral-radius heuristics.

Jẹmọ itọsọna awọn ọnaParẹ Ati Exploding Gradients
Ibeere 1 ti 8

Iṣiṣẹ mathematiki wo ni isọdọtun jẹ idi gbòǹgbò ti sisọnu ati bugbamu gradients?

Awotẹlẹ gbogbo ibeere8 ibeere

Awọn idahun ati awọn alaye han bi o ṣe n mu ibeere naa.

  1. Iṣiṣẹ mathematiki wo ni isọdọtun jẹ idi gbòǹgbò ti sisọnu ati bugbamu gradients?
    • Leralera isodipupo fun-Layer ifosiwewe nipasẹ awọn pq ofin
    • Fifi gradients kọja kan nikan Layer
    • Pipin pipadanu nipasẹ iwọn ipele
    • Mu awọn square root ti awọn òṣuwọn
  2. Why can sigmoid or tanh units contribute to vanishing gradients when they are saturated?
    • Their derivatives are always larger than one
    • Their derivatives become small in saturated regions, so repeated chain-rule products can shrink
    • They remove the chain rule from backpropagation
    • They force every weight to zero
  3. Iru faaji wo ni o ni ipa pupọ julọ nipasẹ awọn iṣoro gradient lori awọn ilana gigun?
    • A meji-Layer perceptron
    • A nikan laini ifaseyin
    • Nẹtiwọọki ti nwaye loorekoore ṣiṣi silẹ lori ọpọlọpọ awọn igba
    • tabili wiwa
  4. Ri ipadanu ikẹkọ lojiji yipada si NaN ṣe afihan iṣoro wo?
    • Awọn gradients ti nparun
    • Ipejọpọ pipe
    • Oṣuwọn ẹkọ kere ju
    • Exploding gradients
  5. Bawo ni awọn asopọ ti o ku (foo) ṣe ṣe iranlọwọ pẹlu sisọnu gradients?
    • Wọn fun iwọn didun ni ọna ọna abuja taara sẹhin, ni ikọja awọn isodipupo idinku
    • Wọn yọ gbogbo awọn aiṣedeede kuro
    • Wọn mu iwọn ẹkọ pọ si laifọwọyi
    • Wọn pa awọn ipele ibẹrẹ
  6. In a simplified analysis of a recurrent linear transformation repeated over many steps, what matrix quantity is a rough signal of growth versus decay?
    • The matrix trace relative to the batch size
    • The spectral radius relative to one
    • The number of hidden units alone
    • The determinant relative to the learning rate
  7. A backpropagated gradient is multiplied by a factor below one at each of many layers. What trend should you expect?
    • It must become exactly one
    • It grows exponentially regardless of the factor
    • The signal can shrink exponentially with depth
    • It stops depending on earlier layers after one step
  8. Why can a recurrent network be especially vulnerable over a long sequence?
    • The network has no hidden state
    • The output layer is used only once
    • The sequence is processed without a chain rule
    • The recurrent transformation is applied repeatedly across timesteps, compounding its gradient effect