Idanwo ti o sopọ mọ itọsọna · Alabọde Ipele

Idanwo Clipping Gradient

Checks norm and value clipping, direction preservation, exploding-gradient control and per-example clipping in differentially private training.

Jẹmọ itọsọna awọn ọnaGidigidi Clipping
Ibeere 1 ti 8

Kini gige gige-nipasẹ-iwuwasi gradient ni akọkọ ṣe itọju lakoko diwọn imudojuiwọn naa?

Awotẹlẹ gbogbo ibeere8 ibeere

Awọn idahun ati awọn alaye han bi o ṣe n mu ibeere naa.

  1. Kini gige gige-nipasẹ-iwuwasi gradient ni akọkọ ṣe itọju lakoko diwọn imudojuiwọn naa?
    • Awọn itọsọna ti awọn ìwò gradient imudojuiwọn
    • Iwọn gangan ti gbogbo gradient
    • Awọn nọmba ti paramita awoṣe
    • Iṣeto oṣuwọn ẹkọ
  2. Ni agekuru-nipasẹ-iwuwasi pẹlu ala-ilẹ c, kini yoo ṣẹlẹ nigbati iwuwasi gradient g_norm ti kọja c?
    • Awọn ipele girama ti ṣeto si odo
    • Ilọsiwaju kọọkan jẹ isodipupo nipasẹ c / g_norm
    • Ikẹkọ duro
    • Oṣuwọn ẹkọ jẹ ilọpo meji
  3. Iṣoro wo ni gige gige ni pataki ti a ṣe apẹrẹ lati dojuko?
    • Awọn gradients ti nparun
    • Ikojọpọ data ti o lọra
    • Exploding gradients
    • Overfitting to afọwọsi ṣeto
  4. Bawo ni agekuru-nipasẹ-iye ṣe yatọ si agekuru-nipasẹ-iwuwasi?
    • O tọju itọsọna imudojuiwọn nigbagbogbo
    • O ṣiṣẹ nikan lori awọn fẹlẹfẹlẹ convolutional
    • O yọ awọn gradients patapata
    • O di paati gradient kọọkan ni ominira sinu iwọn kan, eyiti o le yi itọsọna naa pada
  5. A minibatch produces an unusually large global gradient norm. What does clip-by-norm do before the optimizer update?
    • Scales the full gradient vector down to the configured threshold when the norm exceeds it
    • Changes the model’s labels
    • Drops every gradient component independently
    • Increases the learning rate until the loss recovers
  6. Ni ikẹkọ ikọkọ ti o yatọ (DP-SGD), ipa wo ni fun apẹẹrẹ-pipe gradient clipping?
    • O encrypts awọn data
    • O ṣe agbedemeji ipa apẹẹrẹ kọọkan nitorina ariwo iwọntunwọnsi le ṣe iṣeduro asiri
    • O mu iwọn ipele pọ si
    • O yọ awọn nilo fun a isonu iṣẹ
  7. The global gradient norm is twice a clipping threshold c. By what factor is the gradient scaled?
    • Meji
    • One quarter
    • One half (c divided by 2c)
    • It is left unchanged
  8. Why does DP-SGD clip each example’s gradient before adding calibrated noise?
    • To make all examples have the same label
    • To remove the need for a privacy budget
    • To increase the norm of rare-class gradients
    • To bound an individual example’s influence on the released update