概述
我們的目標不是記住這些例子,而是在新例子上表現良好。了解這個循環可以解釋為什麼人工智慧需要如此多的數據和運算能力,以及為什麼它會以令人驚訝的方式失敗。
重點摘要
- 人工智慧模型透過調整其參數來減少損失來學習,損失是衡量其預測在訓練樣本上的錯誤程度的數字。
- 反向傳播由 Rumelhart、Hinton 和 Williams 於 1986 年在《自然》雜誌上發表,計算每個權重應如何變化以縮小誤差。
- 監督學習、無監督學習、自我監督學習和強化學習的主要差異在於學習訊號的來源、標籤、結構、資料本身或獎勵。
- 對新資料的泛化是真正的目標,因此開發人員保留單獨的訓練、驗證和測試集,並注意過度擬合。
- 大多數部署的模型在訓練結束時都會停止學習,因此它們的知識保持不變,直到開發人員重新訓練或微調它們。
- DeepMind 的 2022 年 Chinchilla 研究發現,模型大小和訓練資料應該一起成長,才能最有效地利用計算。
深入探討
When people say an AI system learns, they mean something narrower and more mechanical than human learning. Most modern AI is built with machine learning: instead of a programmer writing rules such as if the email contains this word, mark it as spam, the programmer supplies examples and a procedure that finds the rules automatically. The thing that learns is a model, which is a mathematical function with adjustable settings called parameters or weights. A small model may have thousands of parameters; AlexNet, the image model that reshaped computer vision in 2012, had 60 million. Learning means finding values for those parameters that make the model's outputs useful.
Training follows a repeating loop. First, the model receives an example, such as a photo, and makes a prediction, such as cat with 70% confidence. Second, a loss function compares the prediction with the correct answer and turns the mistake into a single number: the bigger the mistake, the bigger the loss. Third, the system works out how each parameter contributed to that loss. For neural networks this is done with backpropagation, a procedure described by Rumelhart, Hinton and Williams in Nature in October 1986, which passes the error backward through the layers to compute a gradient, meaning the direction in which each parameter should move to reduce the error. Fourth, an optimizer called gradient descent nudges every parameter a small step in that direction. The size of the step is set by the learning rate. Repeat this over millions of examples, often in small batches and over several full passes through the data called epochs, and the loss gradually falls.
The 1986 paper noted something important: as the weights adjust, hidden units inside the network come to represent useful features of the task that nobody programmed. In an image model, early layers end up detecting edges and later layers respond to shapes and objects. That is why the approach scales to problems, such as recognizing speech, where nobody could write the rules by hand.
There are several kinds of learning signal. In supervised learning every example comes with a correct label, like a photo tagged dog, and the model learns to predict the label. In unsupervised learning there are no labels, and the model looks for structure such as clusters of similar customers. Self-supervised learning creates labels from the data itself: Google's BERT, released in October 2018, hid some words in sentences and learned to fill them in, which let it learn from huge amounts of unlabeled text. Large language models learn the same way by predicting the next word. In reinforcement learning the model acts, receives a reward or penalty, and adjusts to earn more reward over time. DeepMind's AlphaGo Zero, published in Nature in 2017, learned Go entirely from games against itself and beat its human-trained predecessor 100 to 0. Reinforcement learning from human feedback (RLHF) combines these ideas: people rank model answers, and the model is tuned to produce answers people prefer, which is how OpenAI's 2022 InstructGPT was trained.
The central test of learning is generalization, which Google's Machine Learning Crash Course defines as making good predictions on new data. To measure it, developers split their data into three parts. The training set is what the model learns from. The validation set is held back to compare versions and tune settings. The test set is kept aside until the end as a stand-in for real-world data. The main failure is overfitting: the model fits the training examples so closely, noise and all, that it does badly on new data. A typical warning sign is training loss that keeps falling while validation loss starts rising. Another trap is leakage, where near-copies of test examples sneak into the training data and make results look better than they are.
A few more terms make descriptions of AI training easier to follow. Parameters are the numbers the model learns; hyperparameters are the settings people choose before training, such as the learning rate, the batch size and the number of epochs. Pretraining is the first, most expensive phase, in which a model learns general patterns from a very large dataset. Fine-tuning continues training on a smaller, task-specific dataset, which is far cheaper because the model starts from what it already knows; reusing a pretrained model this way is called transfer learning. BERT followed exactly this recipe in 2018: one pretraining run on unlabeled text, then a short fine-tuning step for each of eleven different language tasks. Fine-tuning can also matter more than size. In OpenAI's 2022 InstructGPT study, human evaluators preferred answers from a 1.3-billion-parameter model fine-tuned with human feedback over answers from the 175-billion-parameter GPT-3, a model more than 100 times larger. How a model is trained, and on what, often shapes its behavior as much as how big it is.
Machine learning differs from human learning in ways that matter. A child can learn a new word from a few uses; AlexNet needed about 1.3 million labeled images to learn 1,000 categories. Most deployed models also stop learning once training ends: the parameters are frozen, so a chatbot does not update itself from a single conversation unless its developers retrain it. Compared with hand-written rules, a learned model handles messy inputs far better but is harder to inspect, because its knowledge is spread across millions of numbers rather than written as readable rules.
Learning also fails in predictable ways. A model learns whatever pattern best reduces the loss, which may be a shortcut rather than the intended skill, such as recognizing a hospital's scanner instead of a disease. Biased or unrepresentative data produces biased predictions, and Google's course notes that training, validation and test data should come from similar distributions for results to hold. When the world changes after training, a problem called distribution shift, accuracy can drop without any warning. In reinforcement learning, a poorly chosen reward can be satisfied in unintended ways.
Learning is expensive in data and computing. AlexNet took five to six days to train on two consumer graphics cards in 2012. Modern models need far more: the Stanford AI Index 2025 reports that the training compute used for notable AI models has been doubling about every five months. More is not automatically better, though. DeepMind's 2022 Chinchilla study found that many large language models were undertrained for their size, and that model size and training data should grow together; its 70-billion-parameter Chinchilla, trained on four times more data than the 280-billion-parameter Gopher with the same compute, outperformed Gopher on a wide range of tasks.
04工作範例
手動訓練一號模型
選擇一個小任務:使用模型 y = w 乘以 x,根據尺寸 x 預測價格 y,其中 w 是單一參數,從 w = 1 開始。
舉一個訓練範例,其中 x = 2,真實答案為 y = 6;模型預測 1 乘以 2 = 2。
計算平方誤差損失:(2 減 6) 平方 = 16。
求損失相對於 w 的梯度:2 乘以(預測減目標)乘以 x = 2 乘以(減 4)乘以 2 = - 16。
應用學習率為 0.05 的梯度下降:w_new = 1 減 0.05 倍(減 16)= 1.8。
再預測:1.8 乘以 2 = 3.6,損失為 (3.6 - 6) 平方 = 5.76,低於 16。
重複更新幾次並檢查 w 是否接近 3,該值與範例完全吻合。
它顯示了什麼
經過幾步之後,w 穩定在 3 附近,損失接近零。這與訓練大型神經網路的循環相同,只是使用一個參數而不是數十億個參數。要檢查泛化性,請在模型從未見過的新對(例如 x = 5、y = 15)上測試最終的 w。
現實世界的實施
影像辨識:AlexNet 學會了將約 130 萬張標籤的照片分為 1,000 個類別,並以 15.3% 的 top-5 錯誤率贏得了 ILSVRC-2012 競賽,而次優的錯誤率為 26.2%。
透過自我對弈學習:2017 年《自然》雜誌中描述的 DeepMind 的 AlphaGo Zero 從沒有人類對局的隨機走法開始,僅使用強化學習,以 0 的比分擊敗了早期擊敗冠軍的 AlphaGo 100 場比賽。
從原始文本中學習:Google 的 BERT 於 2018 年 10 月發布,透過預測隱藏單字對未標記文字進行預訓練,然後在 11 項語言理解任務上創下新記錄。
向人類評分學習:OpenAI 2022 年的 InstructGPT 工作使用人們的答案排名來微調 GPT-3,貼標者更喜歡 13 億參數的調整模型,而不是原始的 1750 億參數的 GPT-3。
垃圾郵件過濾器和推薦:日常系統從標記的範例中學習,例如標記為垃圾郵件的電子郵件或觀看者看完的電影,然後將學習到的模式應用於新電子郵件或新觀眾。
人工智慧學習的未來:數據、損失、梯度下降和回饋
Several research directions aim to make learning cheaper, more reliable and more continuous. Data efficiency is one: the gap between a child learning from a few examples and AlexNet needing about 1.3 million labeled images remains large, and self-supervised methods such as BERT's 2018 masked-word training were a step toward using unlabeled data instead of costly labels. Scaling rules are another. DeepMind's 2022 Chinchilla work changed how labs balance model size against data, and the Stanford AI Index 2025 shows training compute still doubling about every five months, which raises questions about energy use and whether high-quality training data will run short. Continual learning, meaning models that update safely after deployment without forgetting earlier skills, is an open problem; most systems today are retrained in large batches instead. Learning from feedback is also evolving. RLHF, as used for InstructGPT in 2022, depends on human ratings that are slow and can be inconsistent, so researchers are testing feedback generated by other models and rewards checked automatically, such as whether code passes tests. Each approach brings its own risk of the model optimizing the measurement rather than the goal, so better evaluation sits alongside better training on the research agenda.
付諸實踐
為什麼這很重要
- 更明確的決策. 它可以幫助您將清晰的技術聲明與行銷語言分開。
- 成本與預算. 在花費金錢或時間之前,您可以提出更好的實施問題。
- 團隊與工作流程. 具有共同理解的團隊可以做出更好的產品、政策和學習決策。
留意
- 不同的團隊可能會以不同的方式使用相同術語,因此請儘早定義範圍。
- 基準測試可能看起來很強大,但實際效能卻參差不齊。
- 忽視數據品質和評估計劃通常會產生脆弱的結果。
後續步驟
- 從您需要的結果的簡單語言定義開始。
- 在測試之前選擇一種成功指標和一種失敗條件。
- 使用代表性資料運行小型試點,而不是完善的演示集。
- 記錄人工智慧如何學習:數據、損失、梯度下降和回饋在哪些方面有幫助,以及哪些更簡單的方法更好。
資料來源與延伸閱讀
- Nature (Rumelhart, Hinton and Williams, 1986)透過反向傳播誤差來學習表示
- NeurIPS 2012 (Krizhevsky, Sutskever and Hinton)使用深度卷積神經網路進行 ImageNet 分類
- Nature (Silver et al., 2017)在沒有人類知識的情況下掌握圍棋遊戲
- arXiv (Devlin et al., 2018)BERT:用於語言理解的深度雙向變壓器的預訓練
- arXiv (Ouyang et al., 2022)訓練語言模型遵循人類回饋的指令
- arXiv (Hoffmann et al., 2022)訓練計算最優的大型語言模型
- Google Machine Learning Crash Course過擬合
- Stanford Institute for Human-Centered AI2025年人工智慧指數報告
不斷探索
常見問題
人工智慧如何學習:數據、損失、梯度下降和回饋?
人工智慧透過調整數百萬或數十億個內部數字(稱為參數)來學習,以便其訓練範例的輸出更接近所需的答案(透過稱為損失的錯誤分數來衡量)。我們的目標不是記住這些例子,而是在新例子上表現良好。了解這個循環可以解釋為什麼人工智慧需要如此多的數據和運算能力,以及為什麼它會以令人驚訝的方式失敗。
在監督式學習中,訓練資料包括哪些內容?
監督學習使用與目標輸出或標籤(例如標記為「貓」或「狗」的照片)配對的範例來學習輸入到輸出的對應。
以相似的購物習慣將客戶分組,而不使用任何預先定義的類別,是哪種類型的學習的範例?
無監督學習在沒有標籤的資料中找到結構,例如聚類。沒有人告訴模型這些組別應該是什麼。
強化學習代理(例如學習玩電玩的代理)如何提升?
強化學習是由獎勵訊號引導的嘗試和錯誤。經過多次嘗試,智能體傾向於那些能夠獲得更多獎勵的行動。
模型在其訓練範例上表現良好,但在可比較的保留範例上表現不佳。這顯示了哪種模式?
過度擬合描述了一種與訓練資料緊密匹配但在新資料上表現不佳的模型。
為什麼要將測試集與擬合模型和選擇其設定分開?
保留的測試集在訓練和模型選擇後給出單獨的估計;重複使用來調整選擇會削弱這種分離。