Reba neza na Ntare
Lookahead na Ntare nibintu bibiri bigezweho kuri neural-net optimizme.
Incamake
Lookahead wraps any base optimizer with 'slow' and 'fast' weights for more stable progress, while Lion (EvoLved Sign Momentum) was discovered by an AI program search and updates weights using only the sign of a momentum term — making it memory-light and often faster than Adam.
Kwibira cyane
Lookahead, yatanzwe na Zhang, Hinton na bagenzi be muri 2019, ikora optimizer isanzwe (nka Adam cyangwa SGD) kugirango k intambwe, hanyuma ihindure ibice bitandukanye by '' buhoro 'bipima igice cyinzira igana aho uburemere bwihuse bwarangirira. Ibi bigabanya ihungabana kandi bigabanya sensibilité kuri hyperparameter. Intare, yasohowe na Google mu 2023, yavuye mubushakashatsi bwa porogaramu yikigereranyo hejuru ya optimizer algorithms. Ikurikirana imbaraga ariko ikoresha ikimenyetso cyibikorwa mukuvugurura, buri kintu cyose rero kigenda kijyana nintambwe ihamye mu cyerekezo cyibimenyetso byegeranijwe. Intare ibika gusa buffer (igice cya leta ya Adamu, ikomeza ibiri), ikoresha kwangirika kwinshi nigipimo gito cyo kwiga, kandi yahujije cyangwa ikubita Adamu mubyerekezo binini nicyitegererezo cyururimi mugihe imyitozo yihuse kandi ihendutse.
Ubushishozi
Ivugurura rya Lookahead: nyuma k intambwe yihuse itanga uburemere θ_ifunguro, uburemere buhoro bugenda nka φ ← φ + α (θ_ifunguro - φ), hanyuma optimizer yihuta igasubira φ. Kuvugurura intare: m ← β1 · m + (1 - β1) · g kuri interpolation, ariko intambwe yuburemere ni θ ← θ - η · (ikimenyetso (β2 · m + (1 - β2) · g) + λθ). Igikorwa cyibimenyetso gikora buri guhuza ibikorwa byo kuvugurura ubunini buringaniye, bukora nkibisanzwe bitagaragara kandi bigasobanura impamvu Intare ikeneye igipimo gito cyo kwiga kurusha Adamu.
Ingaruka z'Ingamba
Igiciro na bije
Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
Ibyemezo bisobanutse
Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
Kugenzura ubuziranenge
Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.
Kazoza ka Lookahead na Optimizers
Intare yakiriwe mumahugurwa menshi manini manini kuko igabanya ububiko bwa optimizer kandi irashobora kwihutisha guhuza, kandi kuvumbura kwayo kwerekana ubushakashatsi bwakozwe na 'AI-gushushanya-AI' gushakisha algorithm nkisoko nyayo yinyungu zifatika. Witege byinshi gushakisha-gukurura optimizers, gahunda ya Hybrid ihuza Lookahead-yuburyo butinda uburemere hamwe nibimenyetso bishya, hamwe no kwiyongera gushishikajwe no gukoresha neza optimizers nkuko ingano yicyitegererezo ikomeza gushimangira ingengo yimari ya GPU.
Gushyira mu bikorwa Isi
Gupfunyika Adam hamwe na Lookahead kugirango uhagarike imyitozo ya transformateur no kugabanya imbaraga za hyperparameter.
Gukoresha Intare kugirango uhugure icyerekezo kinini (urugero, ViT) hamwe na optimizer yo hasi kuruta Adamu.
Gutegura imvugo yindimi hamwe nintare kugirango ugere kubwukuri kugereranije kugiciro cya compte.
Guhuza Lookahead na SGD mubikorwa byo kongera imbaraga-biga kugirango politiki igende neza.
Ingaruka & Kurinda
Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.
Igishushanyo mbonera
Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
Tegura inzira yo gusubiza ibyabaye mbere yo gupima.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Lookahead and Lion Optimizers quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Gukwirakwiza Leta Kurekura kuri CPU na NVMe
Ibibazo bikunze kubazwa
What is Lookahead and Lion Optimizers?
Lookahead na Ntare nibintu bibiri bigezweho kuri neural-net optimizme. Lookahead ipfunyika ibyingenzi byose hamwe nuburemere bwa 'buhoro' na 'byihuse' kugirango iterambere rihamye, mugihe Intare (EvoLved Sign Momentum) yavumbuwe nubushakashatsi bwa porogaramu ya AI kandi ikavugurura uburemere ukoresheje gusa ikimenyetso cyigihe gito - bigatuma yibuka-urumuri kandi akenshi byihuta kuruta Adamu.
Niyihe ntambwe yo kubara intambwe yo kuvugurura ibiro byintare?
Intare ikoresha ikimenyetso cyibikorwa kuri interpolated momentum term, bityo buri kintu cyose kigenda kuntambwe imwe mubyerekezo byerekana ibimenyetso.
Nigute Lookahead itunganya uburyo bwiza?
Lookahead ikoresha byihuse imbere yimbere ya k intambwe, hanyuma yimuka buhoro buhoro igice cyinzira igana uburemere bwihuse hanyuma igasubiramo.
Nigute optimizer ya Ntare yavumbuwe?
Intare (EvoLved Sign Momentum) yavuye mubikorwa-byo gushakisha gahunda byahindutse kandi bisuzuma gahunda yo guhitamo abakandida.
Ugereranije na Adamu, niyihe nyungu yibanze yo kwibuka Intare?
Adam akurikirana ibihe byambere nubwa kabiri; Intare ibika gusa imbaraga imwe gusa, igabanya hafi ya optimizer ya leta yibuka.
Kuki Intare isaba igipimo gito cyo kwiga kurusha Adamu?
Kuva imikorere yibimenyetso ikosora ubunini bwa buri guhuza ibikorwa, intambwe ifatika ni nini, bityo igipimo gito cyo kwiga (no kugabanuka kwinshi).