引導式測驗 · 中等 等級
梯度消失和爆炸測驗
檢視鍊式法則梯度積、飽和度、循環序列長度、剩餘路徑和光譜半徑啟發法的限制。
問題 1 的 8
反向傳播中的什麼數學運算是梯度消失和爆炸的根本原因?
預覽每個問題8 問題
當您參加測驗時,就會出現答案和解釋。
- 反向傳播中的什麼數學運算是梯度消失和爆炸的根本原因?
- 透過鍊式法則重複乘以每層因子
- 在單層上新增漸變
- 將損失除以批量大小
- 取權重的平方根
- Why can sigmoid or tanh units contribute to vanishing gradients when they are saturated?
- Their derivatives are always larger than one
- Their derivatives become small in saturated regions, so repeated chain-rule products can shrink
- They remove the chain rule from backpropagation
- They force every weight to zero
- 哪一種架構受長序列梯度問題的影響最嚴重?
- 兩層感知器
- 單一線性迴歸
- 循環神經網路在多個時間步上展開
- 一個查找表
- 看到訓練損失突然變成 NaN 最有可能顯示哪個問題?
- 梯度消失
- 完美融合
- 學習率太小
- 梯度爆炸
- 殘餘(跳過)連接如何幫助解決梯度消失問題?
- 他們為梯度提供了一條直接向後的捷徑,繞過了縮小乘法
- 他們消除了所有非線性
- 他們自動提高學習率
- 他們刪除早期圖層
- In a simplified analysis of a recurrent linear transformation repeated over many steps, what matrix quantity is a rough signal of growth versus decay?
- The matrix trace relative to the batch size
- The spectral radius relative to one
- The number of hidden units alone
- The determinant relative to the learning rate
- A backpropagated gradient is multiplied by a factor below one at each of many layers. What trend should you expect?
- It must become exactly one
- It grows exponentially regardless of the factor
- The signal can shrink exponentially with depth
- It stops depending on earlier layers after one step
- Why can a recurrent network be especially vulnerable over a long sequence?
- The network has no hidden state
- The output layer is used only once
- The sequence is processed without a chain rule
- The recurrent transformation is applied repeatedly across timesteps, compounding its gradient effect