What does “HumanEval” mean?

A standardized test or dataset used to measure and compare model performance.

A benchmark of grade-school math word problems used to evaluate step-by-step reasoning in language models.

A held-out dataset used to measure model quality after training.

Additional inference computation used during response generation to improve quality or reasoning.

A code-evaluation metric measuring the chance that at least one of k generated samples passes the tests.

Policies, standards, and oversight mechanisms that guide how AI is developed and used in society.