Ururimi AI

Ikirangantego cya YaRN

YaRN extends RoPE-based context through frequency-aware scaling and an attention-temperature adjustment.

Umva iki gitaboAI-byavuzwe · ~2 min

Kuriyi page2 min soma
  1. Incamake
  2. Kwibira cyane
  3. Gushyira mu bikorwa Isi
  4. Kazoza ka YaRN Imirongo Idirishya
  5. Kubishyira mubikorwa
  6. Komeza Ubushakashatsi
  7. Ibibazo bikunze kubazwa

Incamake

Its experiments demonstrate model extensions with limited additional training, with useful performance and resource costs still requiring evaluation.

Kwibira cyane

Uniform Position Interpolation compresses all RoPE frequencies together. YaRN instead preserves high-frequency dimensions used for local positional detail, interpolates low-frequency dimensions for broader range, and blends treatment through a transition region. It also modifies attention temperature as part of the extension method. The paper evaluates long-context adaptations of particular models; its reported data and training-step comparisons should not be converted into a universal tiny-data guarantee. Increasing the supported length does not show that every task is solved reliably at every position. Evaluate the chosen model, adaptation, and intended workload rather than assuming that any rescaled checkpoint has the same capability.

Gushyira mu bikorwa Isi

Kwagura moderi ya LLaMA ifunguye kuva 4K kugeza 128K ibimenyetso kugirango ibashe kwinjiza codebase yose cyangwa amasezerano maremare muri pass imwe

Kureka chatbot ikagumana amateka maremare y'ibiganiro utagabanije guhinduka mbere

Incamake y'ibitabo birebire by'ibitabo cyangwa inyandiko-mvugo y'amasaha menshi arenga icyitegererezo cy'ibanze kavukire

Guhindura bihendutse moderi yabanje gutozwa kubikorwa birebire-byo kugarura ibintu ukoresheje akantu gato keza neza

Kazoza ka YaRN Imirongo Idirishya

Research compares positional methods, training schedules, and long-context tasks. Measure retrieval and generation quality across positions and lengths together with latency and memory. Nominal support, a good average score, and dependable use on a particular task are different forms of evidence, so a new maximum window should be evaluated in context.

Kubishyira mubikorwa

Impamvu ari ngombwa

  • Umuvuduko n'igipimo. Ururimi rwakazi rushobora kugenda byihuse nta gutamba guhuzagurika.
  • Kugera no kugera. Yagura uburyo bwindimi nuburyo bwo gutumanaho.
  • Ibyemezo bisobanutse. Amakipe arashobora kumara umwanya munini murubanza mugihe automatike ikora gusubiramo.

Witondere

  • Ibintu bifatika bishobora kwinjiza bucece raporo, gushyigikira imigendekere, cyangwa ibisubizo byubushakashatsi.
  • Kwihuta byihuse birashobora gukora ibisubizo bidahuye mubisabwa bisa.
  • Ibyanditswe byumvikana birashobora kugaragara niba kugenzura kugenzura ari ntege.

Intambwe ikurikira

  1. Sobanura imiterere isohoka, amajwi, hamwe nubuziranenge mbere yo gutangira.
  2. Ibisubizo byibanze hamwe nisoko yizewe igihe cyose ukuri kwingirakamaro.
  3. Komeza kugenzura abantu kugenzura ibisubizo byinshi.
  4. Kurikirana uburyo bwo kunanirwa no kongera imyitozo cyangwa akazi gahoraho.

Komeza Ubushakashatsi

Ibaruwa y'amakuru y'ubuntu

Ibuka amakuru ya buri munsi ya AI

Inkuru eshatu zizewe za AI buri gitondo cy'iminsi y'akazi, zanditse mu rurimi rworoshye. Ubuntu iteka ryose, nta kwamamaza.

Imeri imwe buri munsi y'akazi. Kwiyimura mu buryo bumwe. Ntitwigera tugurisha cyangwa ngo dusangire aderesi yawe.

Isuzume

Take the YaRN Context Window Scaling quiz

Ibisubizo ako kanya kuri buri kibazo, n'icyemezo gishobora gusangirwa gifite ID yagenzurwa umaze gutsinda isomo.

Tangira ikibazo

Fasha uburezi bwa AI bw'ubuntu. AI Understanding ni umuryango utegamiye kuri leta wa 501(c)(3) — nta kwamamaza, nta kwishyura, iteka ryose. Tanga impano

Ibibazo bikunze kubazwa

What is YaRN Context Window Scaling?

YaRN extends RoPE-based context through frequency-aware scaling and an attention-temperature adjustment. Its experiments demonstrate model extensions with limited additional training, with useful performance and resource costs still requiring evaluation.

Nuwuhe mwanya-kodegisi ya YaRN ihindura kugirango yongere uburebure?

YaRN isobanura 'Nyamara ubundi RoPE extensioN' kandi ikora mugukuraho imirongo yizunguruka ikoreshwa muri Rotary Position Embeddings.

Nigute YaRN ifata inshuro nyinshi nubunini buke bwa RoPE?

Uburyo bwa YaRN 'NTK-by-ibice' burinda ibipimo byinshi (byaho) mugihe bihuza imirongo mike (ndende-ndende) kugirango birinde kwangiza amakuru yaho.

How should YaRN's reported data and fine-tuning efficiency be interpreted?

The method's efficiency claims have an experimental scope and depend on model and adaptation choices.

Usibye gukuraho inshuro nyinshi, ni ubuhe buryo bwiyongereye YaRN ikoresha kugirango ibitekerezo birebire bihamye?

YaRN ihindura ubushyuhe bwo kwitondera kugirango yishyure entropy / logit ihinduka ibaho mugihe ikurikiranye rirerire cyane.

What motivates evaluating a RoPE model before using a context far beyond its original training length?

Extension quality is a model- and task-dependent question rather than an automatic consequence of accepting more tokens.