āļ„āļđāđˆāļĄāļ·āļ­āļžāļ·āđ‰āļ™āļāļēāļ™

āļāļēāļĢāđ€āļĢāļĩāļĒāļ™āļĢāļđāđ‰āļŦāļĨāļąāļāļŠāļđāļ•āļĢ

Curriculum learning orders training examples according to a chosen schedule, often from easier to harder examples.

āļŸāļąāļ‡āļ„āļģāđāļ™āļ°āļ™āļģāļ™āļĩāđ‰āļšāļĢāļĢāļĒāļēāļĒāđ‚āļ”āļĒ AI · ~2 min

āļšāļ™āļŦāļ™āđ‰āļēāļ™āļĩāđ‰āļ­āđˆāļēāļ™ 2 āļ™āļēāļ—āļĩ
  1. āļ āļēāļžāļĢāļ§āļĄ
  2. āđ€āļˆāļēāļ°āļĨāļķāļ
  3. āļāļēāļĢāđƒāļŠāđ‰āļ‡āļēāļ™āļˆāļĢāļīāļ‡āđƒāļ™āđ‚āļĨāļāđāļŦāđˆāļ‡āļ„āļ§āļēāļĄāđ€āļ›āđ‡āļ™āļˆāļĢāļīāļ‡
  4. āļ­āļ™āļēāļ„āļ•āļ‚āļ­āļ‡āļāļēāļĢāđ€āļĢāļĩāļĒāļ™āļĢāļđāđ‰āļŦāļĨāļąāļāļŠāļđāļ•āļĢ
  5. āļāļēāļĢāļ™āļģāđ„āļ›āļ›āļāļīāļšāļąāļ•āļī
  6. āļŠāļģāļĢāļ§āļˆāļ•āđˆāļ­āđ„āļ›
  7. āļ„āļģāļ–āļēāļĄāļ—āļĩāđˆāļžāļšāļšāđˆāļ­āļĒ

āļ āļēāļžāļĢāļ§āļĄ

Its benefits depend on how difficulty is defined and on the task.

āđ€āļˆāļēāļ°āļĨāļķāļ

Bengio and colleagues’ 2009 paper discusses curriculum learning as a training strategy inspired by curricula in education. The paper studies tasks and curricula in specific experiments; it does not show that every task benefits from easy-to-hard training. Self-paced learning is a related approach in which the learner’s objective or current model influences which examples are selected. A common curriculum strategy orders examples from easier to harder, but schedules and difficulty measures vary. Anti-curriculum presents harder examples before easier ones under a defined schedule. Self-paced learning uses current model loss to select examples, often beginning with lower-loss cases; it is related to but distinct from a prescribed curriculum. The original 2010 self-paced paper reports experiments in object localization, noun-phrase coreference, motif finding, and handwritten-digit recognition. The continuation hypothesis proposes beginning with a smoother or easier objective and gradually approaching the target. Evaluate any schedule against a comparable baseline with the same data and compute because benefits vary by task and setup.

āļāļēāļĢāđƒāļŠāđ‰āļ‡āļēāļ™āļˆāļĢāļīāļ‡āđƒāļ™āđ‚āļĨāļāđāļŦāđˆāļ‡āļ„āļ§āļēāļĄāđ€āļ›āđ‡āļ™āļˆāļĢāļīāļ‡

Train with a declared easy-to-hard sequence.

Compare with a shuffled-data baseline using the same examples.

Record the curriculum schedule and difficulty score.

Test whether ordering changes convergence or generalization.

āļ­āļ™āļēāļ„āļ•āļ‚āļ­āļ‡āļāļēāļĢāđ€āļĢāļĩāļĒāļ™āļĢāļđāđ‰āļŦāļĨāļąāļāļŠāļđāļ•āļĢ

Curriculum methods remain task-dependent. Report the difficulty measure and schedule, and compare against appropriate baselines.

āļāļēāļĢāļ™āļģāđ„āļ›āļ›āļāļīāļšāļąāļ•āļī

āļ—āļģāđ„āļĄāļĄāļąāļ™āļ–āļķāļ‡āļŠāļģāļ„āļąāļ

  • āļāļēāļĢāļ•āļąāļ”āļŠāļīāļ™āđƒāļˆāļ—āļĩāđˆāļŠāļąāļ”āđ€āļˆāļ™āļĒāļīāđˆāļ‡āļ‚āļķāđ‰āļ™. āļŠāđˆāļ§āļĒāđƒāļŦāđ‰āļ„āļļāļ“āđāļĒāļāļ„āļģāļāļĨāđˆāļēāļ§āļ­āđ‰āļēāļ‡āļ—āļēāļ‡āđ€āļ—āļ„āļ™āļīāļ„āļ—āļĩāđˆāļŠāļąāļ”āđ€āļˆāļ™āļ­āļ­āļāļˆāļēāļāļ āļēāļĐāļēāļ—āļēāļ‡āļāļēāļĢāļ•āļĨāļēāļ”āđ„āļ”āđ‰
  • āļ•āđ‰āļ™āļ—āļļāļ™āđāļĨāļ°āļ‡āļšāļ›āļĢāļ°āļĄāļēāļ“. āļ„āļļāļ“āļŠāļēāļĄāļēāļĢāļ–āļ–āļēāļĄāļ„āļģāļ–āļēāļĄāļāļēāļĢāđƒāļŠāđ‰āļ‡āļēāļ™āļ—āļĩāđˆāļ”āļĩāļ‚āļķāđ‰āļ™āļāđˆāļ­āļ™āļ—āļĩāđˆāļˆāļ°āđƒāļŠāđ‰āđ€āļ‡āļīāļ™āļŦāļĢāļ·āļ­āđ€āļ§āļĨāļē
  • āļ—āļĩāļĄāļ‡āļēāļ™āđāļĨāļ°āļ‚āļąāđ‰āļ™āļ•āļ­āļ™āļāļēāļĢāļ—āļģāļ‡āļēāļ™. āļ—āļĩāļĄāļ—āļĩāđˆāļĄāļĩāļ„āļ§āļēāļĄāđ€āļ‚āđ‰āļēāđƒāļˆāļĢāđˆāļ§āļĄāļāļąāļ™āļˆāļ°āļ•āļąāļ”āļŠāļīāļ™āđƒāļˆāđ€āļāļĩāđˆāļĒāļ§āļāļąāļšāļœāļĨāļīāļ•āļ āļąāļ“āļ‘āđŒ āļ™āđ‚āļĒāļšāļēāļĒ āđāļĨāļ°āļāļēāļĢāđ€āļĢāļĩāļĒāļ™āļĢāļđāđ‰āđ„āļ”āđ‰āļ”āļĩāļ‚āļķāđ‰āļ™

āļĢāļ°āļ§āļąāļ‡

  • āđāļ•āđˆāļĨāļ°āļ—āļĩāļĄāļ­āļēāļˆāđƒāļŠāđ‰āļ„āļģāđ€āļ”āļĩāļĒāļ§āļāļąāļ™āļ•āđˆāļēāļ‡āļāļąāļ™ āļ”āļąāļ‡āļ™āļąāđ‰āļ™āļ„āļ§āļĢāļāļģāļŦāļ™āļ”āļ‚āļ­āļšāđ€āļ‚āļ•āļ•āļąāđ‰āļ‡āđāļ•āđˆāđ€āļ™āļīāđˆāļ™āđ†
  • āđ€āļāļ“āļ‘āđŒāļĄāļēāļ•āļĢāļāļēāļ™āļ­āļēāļˆāļ”āļđāđāļ‚āđ‡āļ‡āđāļāļĢāđˆāļ‡āđƒāļ™āļ‚āļ“āļ°āļ—āļĩāđˆāļ›āļĢāļ°āļŠāļīāļ—āļ˜āļīāļ āļēāļžāđƒāļ™āđ‚āļĨāļāđāļŦāđˆāļ‡āļ„āļ§āļēāļĄāđ€āļ›āđ‡āļ™āļˆāļĢāļīāļ‡āđ„āļĄāđˆāđ€āļ—āđˆāļēāļāļąāļ™
  • āļāļēāļĢāđ€āļžāļīāļāđ€āļ‰āļĒāļ•āđˆāļ­āļ„āļļāļ“āļ āļēāļžāļ‚āđ‰āļ­āļĄāļđāļĨāđāļĨāļ°āđāļœāļ™āļāļēāļĢāļ›āļĢāļ°āđ€āļĄāļīāļ™āļĄāļąāļāļŠāļĢāđ‰āļēāļ‡āļœāļĨāļĨāļąāļžāļ˜āđŒāļ—āļĩāđˆāđ€āļ›āļĢāļēāļ°āļšāļēāļ‡

āļ‚āļąāđ‰āļ™āļ•āļ­āļ™āļ•āđˆāļ­āđ„āļ›

  1. āđ€āļĢāļīāđˆāļĄāļ•āđ‰āļ™āļ”āđ‰āļ§āļĒāļ„āļģāļˆāļģāļāļąāļ”āļ„āļ§āļēāļĄāļ āļēāļĐāļēāļ˜āļĢāļĢāļĄāļ”āļēāļ‚āļ­āļ‡āļœāļĨāļĨāļąāļžāļ˜āđŒāļ—āļĩāđˆāļ„āļļāļ“āļ•āđ‰āļ­āļ‡āļāļēāļĢ
  2. āđ€āļĨāļ·āļ­āļāđ€āļĄāļ•āļĢāļīāļāļ§āļąāļ”āļ„āļ§āļēāļĄāļŠāļģāđ€āļĢāđ‡āļˆāļŦāļ™āļķāđˆāļ‡āļĢāļēāļĒāļāļēāļĢāđāļĨāļ°āđ€āļ‡āļ·āđˆāļ­āļ™āđ„āļ‚āļ„āļ§āļēāļĄāļĨāđ‰āļĄāđ€āļŦāļĨāļ§āļŦāļ™āļķāđˆāļ‡āļĢāļēāļĒāļāļēāļĢāļāđˆāļ­āļ™āļāļēāļĢāļ—āļ”āļŠāļ­āļš
  3. āļ”āļģāđ€āļ™āļīāļ™āļāļēāļĢāļ™āļģāļĢāđˆāļ­āļ‡āļ‚āļ™āļēāļ”āđ€āļĨāđ‡āļāļ”āđ‰āļ§āļĒāļ‚āđ‰āļ­āļĄāļđāļĨāļ•āļąāļ§āđāļ—āļ™ āđ„āļĄāđˆāđƒāļŠāđˆāļŠāļļāļ”āļŠāļēāļ˜āļīāļ•āļ—āļĩāđˆāļŠāļ§āļĒāļ‡āļēāļĄ
  4. āđ€āļ­āļāļŠāļēāļĢāļ—āļĩāđˆāļāļēāļĢāđ€āļĢāļĩāļĒāļ™āļĢāļđāđ‰āļ•āļēāļĄāļŦāļĨāļąāļāļŠāļđāļ•āļĢāļŠāđˆāļ§āļĒāđ„āļ”āđ‰ āđāļĨāļ°āļ§āļīāļ˜āļĩāļ—āļĩāđˆāļ‡āđˆāļēāļĒāļāļ§āđˆāļēāļˆāļ°āļ”āļĩāļāļ§āđˆāļē

āļŠāļģāļĢāļ§āļˆāļ•āđˆāļ­āđ„āļ›

āļˆāļ”āļŦāļĄāļēāļĒāļ‚āđˆāļēāļ§āļŸāļĢāļĩ

āļĢāļąāļšāļŠāļĢāļļāļ› AI āļĢāļēāļĒāļ§āļąāļ™

āđ€āļĢāļ·āđˆāļ­āļ‡āļĢāļēāļ§ AI āļ—āļĩāđˆāļ•āļĢāļ§āļˆāļŠāļ­āļšāđāļĨāđ‰āļ§ 3 āđ€āļĢāļ·āđˆāļ­āļ‡āļ—āļļāļāļ§āļąāļ™āļ—āļģāļ‡āļēāļ™ āđ€āļŠāđ‰āļē, āđ€āļ‚āļĩāļĒāļ™āđ€āļ›āđ‡āļ™āļ āļēāļĐāļēāļ­āļąāļ‡āļāļĪāļĐāļ˜āļĢāļĢāļĄāļ”āļē āļŸāļĢāļĩāļ•āļĨāļ­āļ”āđ„āļ› āđ„āļĄāđˆāļĄāļĩāđ‚āļ†āļĐāļ“āļē

āļ­āļĩāđ€āļĄāļĨāļŦāļ™āļķāđˆāļ‡āļ‰āļšāļąāļšāļ—āļļāļāļ§āļąāļ™āļ—āļģāļ‡āļēāļ™ āļĒāļāđ€āļĨāļīāļāļāļēāļĢāļŠāļĄāļąāļ„āļĢāđƒāļ™āļ„āļĨāļīāļāđ€āļ”āļĩāļĒāļ§ āđ€āļĢāļēāđ„āļĄāđˆāđ€āļ„āļĒāļ‚āļēāļĒāļŦāļĢāļ·āļ­āđāļŠāļĢāđŒāļ—āļĩāđˆāļ­āļĒāļđāđˆāļ­āļĩāđ€āļĄāļĨāļ‚āļ­āļ‡āļ„āļļāļ“

āļ—āļ”āļŠāļ­āļšāļ•āļąāļ§āđ€āļ­āļ‡

Take the Curriculum Learning quiz

āļ‚āđ‰āļ­āđ€āļŠāļ™āļ­āđāļ™āļ°āļ—āļąāļ™āļ—āļĩāļŠāļģāļŦāļĢāļąāļšāļ—āļļāļāļ„āļģāļ•āļ­āļš āđāļĨāļ°āđƒāļšāļĢāļąāļšāļĢāļ­āļ‡āļ—āļĩāđˆāļŠāļēāļĄāļēāļĢāļ–āđāļŠāļĢāđŒāđ„āļ”āđ‰āļžāļĢāđ‰āļ­āļĄ ID āļ—āļĩāđˆāļ•āļĢāļ§āļˆāļŠāļ­āļšāđ„āļ”āđ‰āđ€āļĄāļ·āđˆāļ­āļ„āļļāļ“āļœāđˆāļēāļ™āļŦāļĨāļąāļāļŠāļđāļ•āļĢ

āđ€āļĢāļīāđˆāļĄāđāļšāļšāļ—āļ”āļŠāļ­āļš

āļŠāļ™āļąāļšāļŠāļ™āļļāļ™āļāļēāļĢāļĻāļķāļāļĐāļē AI āļŸāļĢāļĩ AI Understanding āđ€āļ›āđ‡āļ™āļ­āļ‡āļ„āđŒāļāļĢāđ„āļĄāđˆāđāļŠāļ§āļ‡āļŦāļēāļœāļĨāļāļģāđ„āļĢ 501(c)(3) — āđ„āļĄāđˆāļĄāļĩāđ‚āļ†āļĐāļ“āļē āđ„āļĄāđˆāļĄāļĩāļāļģāđāļžāļ‡āđ€āļ‡āļīāļ™, āļ•āļĨāļ­āļ”āđ„āļ› āļšāļĢāļīāļˆāļēāļ„

āļ„āļģāļ–āļēāļĄāļ—āļĩāđˆāļžāļšāļšāđˆāļ­āļĒ

What is Curriculum Learning?

Curriculum learning orders training examples according to a chosen schedule, often from easier to harder examples. Its benefits depend on how difficulty is defined and on the task.

What ordering does a common curriculum-learning strategy use?

A common curriculum orders examples from easier to harder, but difficulty definitions and schedules vary by task; this is not a required or universally beneficial ordering.

āđƒāļ„āļĢāđ€āļ›āđ‡āļ™āļ„āļ™āđāļ™āļ°āļ™āļģāļ„āļģāļ§āđˆāļē 'āļāļēāļĢāđ€āļĢāļĩāļĒāļ™āļĢāļđāđ‰āļ•āļēāļĄāļŦāļĨāļąāļāļŠāļđāļ•āļĢ' āđƒāļ™āļĢāļēāļĒāļ‡āļēāļ™āļ›āļĢāļ°āļˆāļģāļ›āļĩ 2009 āļ—āļĩāđˆāļĄāļĩāļŠāļ·āđˆāļ­āđ€āļŠāļĩāļĒāļ‡

Bengio āđāļĨāļ°āļœāļđāđ‰āļĢāđˆāļ§āļĄāđ€āļ‚āļĩāļĒāļ™āđ„āļ”āđ‰āļˆāļąāļ”āļ—āļģāļŦāļĨāļąāļāļŠāļđāļ•āļĢāļāļēāļĢāđ€āļĢāļĩāļĒāļ™āļĢāļđāđ‰āļ”āđ‰āļēāļ™āļāļēāļĢāđ€āļĢāļĩāļĒāļ™āļĢāļđāđ‰āļ‚āļ­āļ‡āđ€āļ„āļĢāļ·āđˆāļ­āļ‡āļ­āļĒāđˆāļēāļ‡āđ€āļ›āđ‡āļ™āļ—āļēāļ‡āļāļēāļĢāđƒāļ™āļĢāļēāļĒāļ‡āļēāļ™āļ›āļĩ 2009

āđƒāļ™āļāļēāļĢāđ€āļĢāļĩāļĒāļ™āļĢāļđāđ‰āļ”āđ‰āļ§āļĒāļ•āļ™āđ€āļ­āļ‡ āđƒāļ„āļĢāđ€āļ›āđ‡āļ™āļ„āļ™āļ•āļąāļ”āļŠāļīāļ™āđƒāļˆāļ§āđˆāļēāļ•āļąāļ§āļ­āļĒāđˆāļēāļ‡āđƒāļ”āļ—āļĩāđˆ 'āļ‡āđˆāļēāļĒ' āļžāļ­āļ—āļĩāđˆāļˆāļ°āļāļķāļāļāļ™āđ„āļ”āđ‰āļ•āļąāđ‰āļ‡āđāļ•āđˆāđ€āļ™āļīāđˆāļ™āđ†

Self-paced learning is a related approach in which the model’s current losses help select examples, often beginning with lower-loss examples and expanding the set. It is not identical to every externally specified easy-to-hard curriculum.

āđ€āļŦāļ•āļļāđƒāļ”āļāļēāļĢāđ€āļĢāļĩāļĒāļ™āļĢāļđāđ‰āļ•āļēāļĄāļŦāļĨāļąāļāļŠāļđāļ•āļĢāļˆāļķāļ‡āļ­āļēāļˆāļŠāđˆāļ§āļĒāđ€āļžāļīāđˆāļĄāļ›āļĢāļ°āļŠāļīāļ—āļ˜āļīāļ āļēāļžāđ„āļ”āđ‰

The continuation hypothesis proposes beginning with a smoother or easier objective and gradually moving toward the target objective; it is one possible explanation for why some curricula help, not a universal guarantee.

How should a team test whether an easy-to-hard curriculum helps a model?

A curriculum’s benefit depends on task, schedule, data, and optimization. A controlled baseline with comparable data and compute is needed; no universal setting guarantees the largest gain.

āđ€āļĢāļĩāļĒāļ™āļĢāļđāđ‰āļ•āđˆāļ­āđ„āļ›

āļ„āļģāđāļ™āļ°āļ™āļģāļ—āļĩāđˆāđ€āļāļĩāđˆāļĒāļ§āļ‚āđ‰āļ­āļ‡

āļ„āļģāđāļ™āļ°āļ™āļģāđ€āļžāļīāđˆāļĄāđ€āļ•āļīāļĄāļ—āļĩāđˆāđ€āļĨāļ·āļ­āļāļŠāļģāļŦāļĢāļąāļšāļŦāļąāļ§āļ‚āđ‰āļ­āļ™āļĩāđ‰

āļ•āđˆāļ­āđ„āļ›āļ„āļģāđāļ™āļ°āļ™āļģāļ•āđˆāļ­āđ„āļ›

āļāļēāļĢāđ€āļĢāļĩāļĒāļ™āļĢāļđāđ‰āļ­āļĒāđˆāļēāļ‡āļ•āđˆāļ­āđ€āļ™āļ·āđˆāļ­āļ‡āđāļĨāļ°āļāļēāļĢāļĨāļ·āļĄāļ­āļĒāđˆāļēāļ‡āļŦāļēāļĒāļ™āļ°

āļ„āļ§āļēāļĄāļĢāļđāđ‰āļžāļ·āđ‰āļ™āļāļēāļ™

āļĢāļąāļšāđƒāļšāļĢāļąāļšāļĢāļ­āļ‡ AI āļŸāļĢāļĩāļŦāļĨāļąāļāļŠāļđāļ•āļĢāđāļĨāļ°āļāļēāļĢāļŠāļ­āļšāļŠāļąāđ‰āļ™ āđ† āļŸāļĢāļĩ āđ„āļĄāđˆāļĄāļĩāļ„āđˆāļēāđƒāļŠāđ‰āļˆāđˆāļēāļĒ āđ„āļĄāđˆāļĄāļĩāļšāļąāļ•āļĢāđ€āļ„āļĢāļ”āļīāļ• āļ›āļĢāļ°āļĄāļēāļ“ 2.5 āļŠāļąāđˆāļ§āđ‚āļĄāļ‡āđ€āļĢāļīāđˆāļĄāļ•āđ‰āļ™āļŸāļĢāļĩ →