AI の学習方法: データ、損失、勾配降下法、フィードバック

When people say an AI system learns, they mean something narrower and more mechanical than human learning. Most modern AI is built with machine learning: instead of a programmer writing rules such as if the email contains this word, mark it as spam, the programmer supplies examples and a procedure that finds the rules automatically. The thing that learns is a model, which is a mathematical function with adjustable settings called parameters or weights. A small model may have thousands of parameters; AlexNet, the image model that reshaped computer vision in 2012, had 60 million. Learning means finding values for those parameters that make the model's outputs useful.

Training follows a repeating loop. First, the model receives an example, such as a photo, and makes a prediction, such as cat with 70% confidence. Second, a loss function compares the prediction with the correct answer and turns the mistake into a single number: the bigger the mistake, the bigger the loss. Third, the system works out how each parameter contributed to that loss. For neural networks this is done with backpropagation, a procedure described by Rumelhart, Hinton and Williams in Nature in October 1986, which passes the error backward through the layers to compute a gradient, meaning the direction in which each parameter should move to reduce the error. Fourth, an optimizer called gradient descent nudges every parameter a small step in that direction. The size of the step is set by the learning rate. Repeat this over millions of examples, often in small batches and over several full passes through the data called epochs, and the loss gradually falls.

The 1986 paper noted something important: as the weights adjust, hidden units inside the network come to represent useful features of the task that nobody programmed. In an image model, early layers end up detecting edges and later layers respond to shapes and objects. That is why the approach scales to problems, such as recognizing speech, where nobody could write the rules by hand.

There are several kinds of learning signal. In supervised learning every example comes with a correct label, like a photo tagged dog, and the model learns to predict the label. In unsupervised learning there are no labels, and the model looks for structure such as clusters of similar customers. Self-supervised learning creates labels from the data itself: Google's BERT, released in October 2018, hid some words in sentences and learned to fill them in, which let it learn from huge amounts of unlabeled text. Large language models learn the same way by predicting the next word. In reinforcement learning the model acts, receives a reward or penalty, and adjusts to earn more reward over time. DeepMind's AlphaGo Zero, published in Nature in 2017, learned Go entirely from games against itself and beat its human-trained predecessor 100 to 0. Reinforcement learning from human feedback (RLHF) combines these ideas: people rank model answers, and the model is tuned to produce answers people prefer, which is how OpenAI's 2022 InstructGPT was trained.

The central test of learning is generalization, which Google's Machine Learning Crash Course defines as making good predictions on new data. To measure it, developers split their data into three parts. The training set is what the model learns from. The validation set is held back to compare versions and tune settings. The test set is kept aside until the end as a stand-in for real-world data. The main failure is overfitting: the model fits the training examples so closely, noise and all, that it does badly on new data. A typical warning sign is training loss that keeps falling while validation loss starts rising. Another trap is leakage, where near-copies of test examples sneak into the training data and make results look better than they are.

A few more terms make descriptions of AI training easier to follow. Parameters are the numbers the model learns; hyperparameters are the settings people choose before training, such as the learning rate, the batch size and the number of epochs. Pretraining is the first, most expensive phase, in which a model learns general patterns from a very large dataset. Fine-tuning continues training on a smaller, task-specific dataset, which is far cheaper because the model starts from what it already knows; reusing a pretrained model this way is called transfer learning. BERT followed exactly this recipe in 2018: one pretraining run on unlabeled text, then a short fine-tuning step for each of eleven different language tasks. Fine-tuning can also matter more than size. In OpenAI's 2022 InstructGPT study, human evaluators preferred answers from a 1.3-billion-parameter model fine-tuned with human feedback over answers from the 175-billion-parameter GPT-3, a model more than 100 times larger. How a model is trained, and on what, often shapes its behavior as much as how big it is.

Machine learning differs from human learning in ways that matter. A child can learn a new word from a few uses; AlexNet needed about 1.3 million labeled images to learn 1,000 categories. Most deployed models also stop learning once training ends: the parameters are frozen, so a chatbot does not update itself from a single conversation unless its developers retrain it. Compared with hand-written rules, a learned model handles messy inputs far better but is harder to inspect, because its knowledge is spread across millions of numbers rather than written as readable rules.

Learning also fails in predictable ways. A model learns whatever pattern best reduces the loss, which may be a shortcut rather than the intended skill, such as recognizing a hospital's scanner instead of a disease. Biased or unrepresentative data produces biased predictions, and Google's course notes that training, validation and test data should come from similar distributions for results to hold. When the world changes after training, a problem called distribution shift, accuracy can drop without any warning. In reinforcement learning, a poorly chosen reward can be satisfied in unintended ways.

Learning is expensive in data and computing. AlexNet took five to six days to train on two consumer graphics cards in 2012. Modern models need far more: the Stanford AI Index 2025 reports that the training compute used for notable AI models has been doubling about every five months. More is not automatically better, though. DeepMind's 2022 Chinchilla study found that many large language models were undertrained for their size, and that model size and training data should grow together; its 70-billion-parameter Chinchilla, trained on four times more data than the 280-billion-parameter Gopher with the same compute, outperformed Gopher on a wide range of tasks.