Amajwi yanditseho
Amajwi yanditseho amajwi atanga interuro-mvugo-karemano isobanura ibiri muri clip yafashwe amajwi, nk '' ihembe rya gari ya moshi rivuga iyo ryambutse urwego. ' Ihuza amajwi nururimi rwo gushakisha, kugerwaho, no gusobanukirwa.
Kwibira cyane
Amajwi yanditseho amajwi (bakunze kwita amajwi yerekana amajwi) atandukanye no kumenyekanisha imvugo: aho kwandukura amagambo yavuzwe, asobanura ibyerekanwe muri acoustic muri rusange, harimo amajwi atavuga, inkomoko yabo, n'imibanire yabo. Icyitegererezo gishobora gusohora 'inyoni zivuga mugihe amazi atemba inyuma.' Ibi bisaba gusobanukirwa ibintu byinshi byumvikana, gahunda zabo, hamwe nibisobanuro byabo, hanyuma ugahimba interuro nziza, imeze nkabantu. Ibipimo ngenderwaho bisanzwe birimo Clotho na AudioCaps, hamwe n'ibipimo nka CIDEr, SPICE, hamwe n'amajwi yihariye ya SPIDEr na FENSE. Igikorwa gishyigikira uburyo bwo kubona abatumva kandi bigoye-kumva, gushakisha amajwi ashingiye ku majwi, hamwe na AI ikungahaye cyane. Ingorane zayo nyamukuru ni ugutanga ibisobanuro byombi mubyukuri kandi bisanzwe byavuzwe.
Ubushishozi
Sisitemu nyinshi zikoresha encoder-decoder igishushanyo: kodegisi y'amajwi, akenshi CNN yateguwe nka PANNs cyangwa transformateur nka transfert ya majwi yerekana amajwi, ihindura clip muburyo bwo gushyiramo ibintu, hamwe na decoder y'ururimi, akenshi ihinduranya cyangwa imiterere y'ururimi neza, itanga ibisobanuro byanditse kumagambo ukoresheje ijambo hamwe nibitekerezo kuri ibyo biranga. Gutandukanya imvugo-amajwi yerekana imvugo (CLAP) hamwe namakuru manini yazamuye neza kuvuga neza kandi neza, bituma hafi ya zeru-shusho yanditse.
Ingaruka z'Ingamba
Kugera no kugera
Itezimbere kugerwaho binyuze mu kwandukura, kuvuga, no guhuza amajwi.
Igiciro na bije
Amatsinda yibitangazamakuru arashobora kohereza amajwi yihuse hamwe na bije nto.
Umuvuduko n'igipimo
Sisitemu ireba abakiriya irashobora gutunganya imikoranire ivugwa murwego runini.
Igihe kizaza cyanditseho amajwi
Ibisobanuro biri hamwe hamwe nini nini yerekana amajwi-yerekana amajwi ashobora gusobanura, gusubiza ibibazo bijyanye, nimpamvu hejuru yijwi muri sisitemu imwe. Tegereza ibisobanuro bikize, birebire, kandi birashobora kugenzurwa, harimo ibisobanuro byigihe gito nuwuvuga cyangwa amarangamutima. Moderi ihuriweho yerekana amajwi, inyandiko, niyerekwa bizemerera abakoresha kubaza amajwi mubiganiro. Kugabanya amakuru arambuye no kunoza ibipimo byo gusuzuma bihuye nubushishozi bwabantu bikomeza kuba ibyingenzi mubikorwa byo kwizerwa.
Gushyira mu bikorwa Isi
Gukora ibisobanuro byerekana amajwi adasanzwe kubatumva kandi bigoye-kumva-birenze ibirenze kuvuga.
Guha imbaraga inyandiko-shakisha hejuru yububiko bunini bwamajwi kugirango abanditsi bashobore kubona clips mubisobanura
Auto-taging hamwe nincamake ya videwo yoherejwe nabakoresha na podcasts kugirango bagusabe kandi berekane
Gufasha abafite ubumuga bwo kutabona gusobanukirwa ibibakikije binyuze mubisobanuro byavuzwe byamajwi ari hafi
Ingaruka & Kurinda
Gukoresha nabi amajwi no kwigira ibyago byiyongera mugihe uruhushya rubuze.
Ukuri kurashobora kugabanuka hejuru yimvugo, imvugo, cyangwa urusaku rwibidukikije.
Amajwi yubukorikori arashobora kwibeshya kumvugo yukuri nta kirango gisobanutse.
Igishushanyo mbonera
Shaka uruhushya rusobanutse rwo gufata amajwi, gukoroniza, no gukoresha.
Ikizamini cyiza mubiganiro bitandukanye hamwe nuburyo bwimbere.
Sobanura igihe umuntu agomba gusuzuma cyangwa kwemeza ibisubizo.
Andika amajwi yubukorikori kandi ugumane inyandiko zerekana kubazwa.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Audio Captioning quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Kode ya Audio
Ibibazo bikunze kubazwa
What is Audio Captioning?
Amajwi yanditseho amajwi atanga interuro-mvugo-karemano isobanura ibiri muri clip yafashwe amajwi, nk '' ihembe rya gari ya moshi rivuga iyo ryambutse urwego. ' Ihuza amajwi nururimi rwo gushakisha, kugerwaho, no gusobanukirwa.
Nigute amajwi yanditseho amajwi atandukanye no kumenya imvugo yikora?
Kumenyekanisha imvugo bisobanura amagambo, mugihe amajwi yanditsemo atanga interuro isobanura amajwi nibibaho, harimo n'amajwi atavuga.
Nibihe bibiri bya datasets nibipimo bisanzwe byerekana amajwi?
Clotho na AudioCaps nibipimo bikoreshwa cyane kubikorwa byanditse byikora byikora.
Ni ubuhe bwubatsi sisitemu nyinshi yanditseho amajwi ikoresha?
Kode ya majwi ihindura clip mu gushiramo kandi decoder y'ururimi itanga ibisobanuro byanditse ku ijambo, mubisanzwe ubyitayeho.
Ni ukubera iki kwandika amajwi bifite agaciro kubigeraho?
Ibisobanuro byerekana amajwi y'ingenzi atavuga, nk'impuruza cyangwa amashyi, biha abatumva kandi bigoye kumva-abakoresha imvugo ikungahaye kuruta insanganyamatsiko yonyine.
Niki muri ibi ni igipimo cyo gusuzuma gikoreshwa mu gufata amajwi?
CIDEr (hamwe na SPICE, SPIDEr, na FENSE) ipima ubuziranenge bwanditse; ibindi ni ibara, inshuro, cyangwa amajwi aranguruye.