AI Glossary Term
What does “GSM8K” mean?
Definition
A benchmark of grade-school math word problems used to evaluate step-by-step reasoning in language models.
How GSM8K works
Example
A typical problem: a store sells pencils in packs, a student buys several packs and gives some away; how many remain? The model must work through two or three arithmetic steps.
Why it matters
GSM8K checks multi-step reasoning on grade-school math. Strong models now score very high, so harder tests are used too.
Common questions
Who made it?
OpenAI researchers in 2021.
How big is it?
About 8,500 problems, with roughly 7,500 for training and 1,300 for testing.
Does a high score mean math ability?
It shows skill on this style of problem, and possible exposure to similar questions in training data should be considered.