Maswali yanayounganishwa na mwongozo · Kati Kiwango

Maswali ya Kupunguza Gradient

Hukagua kanuni na upunguzaji wa thamani, uhifadhi wa mwelekeo, udhibiti wa gradient unaolipuka na upunguzaji kwa kila mfano katika mafunzo ya kibinafsi tofauti.

Njia za mwongozo zinazohusianaUpunguzaji wa Gradient
Swali 1 ya 8

Je, upunguzaji wa klipu kwa-kawaida huhifadhi nini hasa wakati unazuia sasisho?

Hakiki kila swali8 maswali

Majibu na maelezo huonekana unapojibu maswali.

  1. Je, upunguzaji wa klipu kwa-kawaida huhifadhi nini hasa wakati unazuia sasisho?
    • Mwelekeo wa sasisho la jumla la upinde rangi
    • Ukubwa halisi wa kila upinde rangi
    • Idadi ya vigezo vya mfano
    • Ratiba ya kiwango cha kujifunza
  2. Katika klipu-kwa-kawaida na kizingiti c, nini hufanyika wakati kawaida ya gradient g_norm inazidi c?
    • Gradients zimewekwa hadi sifuri
    • Kila upinde rangi unazidishwa na c/g_norm
    • Mafunzo yanaacha
    • Kiwango cha kujifunza kinaongezeka mara mbili
  3. Ni shida gani ambayo upunguzaji wa upinde rangi umeundwa mahsusi kupambana?
    • Gradients zinazopotea
    • Upakiaji wa data polepole
    • Gradients zinazolipuka
    • Kutosha kwa seti ya uthibitishaji
  4. Je, klipu-kwa-thamani inatofautiana vipi na klipu-kwa-kawaida?
    • Daima huhifadhi mwelekeo wa sasisho
    • Inafanya kazi tu kwenye tabaka za ubadilishaji
    • Huondoa gradients kabisa
    • Inabana kila sehemu ya gradient kwa kujitegemea katika masafa, ambayo inaweza kubadilisha mwelekeo
  5. A minibatch produces an unusually large global gradient norm. What does clip-by-norm do before the optimizer update?
    • Scales the full gradient vector down to the configured threshold when the norm exceeds it
    • Changes the model’s labels
    • Drops every gradient component independently
    • Increases the learning rate until the loss recovers
  6. Katika mafunzo tofauti ya kibinafsi (DP-SGD), upunguzaji wa upinde rangi kwa kila mfano una jukumu gani?
    • Inasimba data kwa njia fiche
    • Inaweka ushawishi wa kila mfano ili kelele iliyorekebishwa iweze kuhakikisha faragha
    • Inaongeza ukubwa wa kundi
    • Huondoa hitaji la utendaji wa upotezaji
  7. The global gradient norm is twice a clipping threshold c. By what factor is the gradient scaled?
    • Mbili
    • One quarter
    • One half (c divided by 2c)
    • It is left unchanged
  8. Why does DP-SGD clip each example’s gradient before adding calibrated noise?
    • To make all examples have the same label
    • To remove the need for a privacy budget
    • To increase the norm of rare-class gradients
    • To bound an individual example’s influence on the released update