Èdè AI Itọsọna
YàRN Ojú Ferese Wiwọn
YaRN extends RoPE-based context through frequency-aware scaling and an attention-temperature adjustment.
Gbọ si itọsọna yiiAI-ṣe afihan · ~2 min
Lori iwe yi2 min ka
Akopọ
Its experiments demonstrate model extensions with limited additional training, with useful performance and resource costs still requiring evaluation.
Jin Dive
Uniform Position Interpolation compresses all RoPE frequencies together. YaRN instead preserves high-frequency dimensions used for local positional detail, interpolates low-frequency dimensions for broader range, and blends treatment through a transition region. It also modifies attention temperature as part of the extension method. The paper evaluates long-context adaptations of particular models; its reported data and training-step comparisons should not be converted into a universal tiny-data guarantee. Increasing the supported length does not show that every task is solved reliably at every position. Evaluate the chosen model, adaptation, and intended workload rather than assuming that any rescaled checkpoint has the same capability.
Real-World imuse
Mimu awoṣe LLAMA ti o ṣii lati 4K si awọn ami ami 128K ki o le mu gbogbo koodu koodu kan tabi adehun gigun ni iwe-iwọle kan
Gbigba chatbot kan duro awọn itan-akọọlẹ ibaraẹnisọrọ gigun pupọ laisi gige awọn iyipada iṣaaju
Akopọ awọn iwe aṣẹ gigun-iwe tabi awọn iwe afọwọkọ-wakati pupọ ti o kọja ferese abinibi ti awoṣe ipilẹ
Didiwọn ṣe adaṣe awoṣe ti ikẹkọ tẹlẹ fun awọn iṣẹ-ṣiṣe igbapada ọrọ-pipẹ ni lilo ṣiṣe atunṣe-itanran kekere nikan
Ọjọ iwaju ti Iwọn Window Tọrọ YaRN
Research compares positional methods, training schedules, and long-context tasks. Measure retrieval and generation quality across positions and lengths together with latency and memory. Nominal support, a good average score, and dependable use on a particular task are different forms of evidence, so a new maximum window should be evaluated in context.
Fifi o sinu iwa
Kini idi ti o ṣe pataki
- Iyara ati iwọn. Ṣiṣan iṣẹ ede le gbe ni iyara laisi irubọ aitasera.
- Wiwọle ati arọwọto. O faagun iraye si kọja awọn ede ati awọn aza ibaraẹnisọrọ.
- Awọn ipinnu diẹ sii. Awọn ẹgbẹ le lo akoko diẹ sii lori idajọ lakoko ti adaṣe n kapa atunwi.
Ṣọra fun
- Awọn otitọ ti a sọ di mimọ le tẹ awọn ijabọ sii ni idakẹjẹ, awọn ṣiṣan atilẹyin, tabi awọn abajade iwadii.
- Ifamọ kiakia le ṣẹda awọn abajade aisedede kọja awọn ibeere ti o jọra.
- Awọn data ọrọ ifarabalẹ le farahan ti awọn idari wiwọle ko lagbara.
Next awọn igbesẹ
- Ṣetumo ọna kika iṣẹjade, ohun orin, ati awọn iṣedede didara ṣaaju ṣiṣejade.
- Awọn idahun ilẹ pẹlu awọn orisun ti o gbẹkẹle nigbakugba ti deede ba ṣe pataki.
- Jeki aaye ayẹwo atunyẹwo eniyan fun awọn abajade ti o ga julọ.
- Tọpinpin awọn ilana ikuna ati tunṣe awọn itọsi tabi ṣiṣan iṣẹ nigbagbogbo.
Tesiwaju Ṣiṣawari
Ìwé ìròyìn ọfẹ
Gba ìkànìyàn AI ojoojúmọ́
Ìtàn AI mẹta tó jẹ́rìí ní gbogbo owurọ́ ọjọ́ iṣẹ́, kọ́wé ní Gẹ̀ẹ́sì tó rọrùn. Ọfẹ́ fún gbogbo àkókò, kò sí ìpolówó.
Ìmeèlì kan ní gbogbo ọjọ́ iṣẹ́. Yọkúrò ní ìkànìyàn ní ìkànìyàn kan. A kò ta tàbí pín àdírẹ́sì rẹ.
Ṣe idanwo fun ara rẹ
Take the YaRN Context Window Scaling quiz
Ìdáhùn lẹ́sẹ̀kẹsẹ̀ lórí gbogbo ìdáhùn, àti iwe-ẹ̀rí tó lè pin pẹ̀lú ID tó lè jẹ́rìí lẹ́yìn tí o bá kọ́ ẹ̀kọ́.
Ṣe àtìlẹ́yìn ẹ̀kọ́ AI ọfẹ́. AI Understanding jẹ́ àjọ àìní èrè 501(c)(3) — kò sí ìpolówó, kò sí àkúnya owó, ní gbogbo àkókò. Ṣe ẹbun kan
Awọn ibeere ti a beere nigbagbogbo
Kini YaRN Context Window Scaling?
YaRN extends RoPE-based context through frequency-aware scaling and an attention-temperature adjustment. Its experiments demonstrate model extensions with limited additional training, with useful performance and resource costs still requiring evaluation.
Eto fifi koodu ipo wo ni YaRN ṣe atunṣe lati fa gigun ọrọ-ọrọ pọ si?
YaRN duro fun 'Sibẹsi afikun RoPE miiran' ati pe o ṣiṣẹ nipa atunkọ awọn igbohunsafẹfẹ iyipo ti a lo ninu Awọn ifibọ ipo Rotari.
Bawo ni YaRN ṣe tọju igbohunsafẹfẹ giga dipo awọn iwọn RoPE igbohunsafẹfẹ-kekere?
Ọna 'NTK-nipasẹ-apakan' ti YaRN ṣe itọju awọn iwọn igbohunsafẹfẹ giga-giga (agbegbe) lakoko ti o dapọ awọn igbohunsafẹfẹ-kekere (ibiti o gun) lati yago fun ibajẹ alaye agbegbe.
How should YaRN's reported data and fine-tuning efficiency be interpreted?
The method's efficiency claims have an experimental scope and depend on model and adaptation choices.
Yato si awọn loorekoore iwọntunwọnsi, atunṣe afikun wo ni YaRN lo lati jẹ ki akiyesi gigun ni iduroṣinṣin?
YaRN ṣe atunṣe iwọn otutu ifarabalẹ lati sanpada fun entropy/logit iyipada ti o waye nigbati awọn ilana ba pẹ pupọ.
What motivates evaluating a RoPE model before using a context far beyond its original training length?
Extension quality is a model- and task-dependent question rather than an automatic consequence of accepting more tokens.
Tesiwaju kikọ
Jẹmọ awọn itọsọna
Awọn itọsọna diẹ sii ti a yan fun koko yii