HejuruUbuyobozi bukurikira
Amazi atagaragara kumashusho ya AI na Audio
AI igaragara
Amabwiriza ya AI
Gukoresha amajwi ya AI umunwa sync ihindura amashusho yisura kugirango umunwa wacyo uhure numurongo mushya wo kuvuga, uteganya imiterere yiminwa iva mumajwi ubwayo.
Umva iki gitaboAI-byavuzwe · ~3 min
Nibyingenzi byo gushidikanya amasomo na firime mu zindi ndimi no gukemura ibiganiro byongeye kwandikwa. Tekinike imwe irashobora kandi gutuma abantu nyabo bagaragara bavuga amagambo batigeze bavuga.
Wav2Lip, yasohowe mu 2020 n'abashakashatsi bo muri IIIT Hyderabad (ACM Multimedia), niyo moderi izwi cyane ifunguye iminwa hamwe nicyitegererezo cyingirakamaro kumurima. Kuri buri kintu cyagenewe intego, generator yakira inyongeramusaruro eshatu: isura nigice cyayo cyo hepfo yipfundikiriye, ikadiri yerekana umuntu umwe utanga isura, hamwe nidirishya rigufi rya mel-spekrogramu yamajwi hafi yicyo gihe. Umuyoboro wa encoder-decoder noneho ushushanya mukarere ka munwa, kandi ibisubizo byahujwe gusubira muri videwo yumwimerere. Umusanzu wingenzi ni "impuguke" yabanje gutozwa ivangura rishingiye kuri SyncNet. Ikomeza gukonjeshwa mugihe cyamahugurwa kandi igacira urubanza niba gukora akanya gato kumunwa uhuye nijwi. Uburyo bwambere bwahuguye ivangura rikorana hamwe na generator, kandi abo bavangura bagaragaye nkabacamanza babi ba sync. Wav2Lip yakoresheje kandi ivangura-ryiza ritandukanye. Intege nke zayo zari ibihingwa byo mu kanwa bidakemutse, amenyo atagaragara kandi adasanzwe cyangwa guhindagurika kumupaka wa mask. Nyuma sisitemu nka VideoReTalking, MuseTalk hamwe na diffusion ishingiye kuri LatentSync igamije ibisubizo bikaze, bihamye. Ubwiza bugenzurwa muburyo butandukanye. LSE-D ipima intera iri hagati y amajwi na videwo ya SyncNet, aho hepfo ari nziza. LSE-C ipima ikizere cya SyncNet, aho hejuru ari nziza. Ibipimo-byiza-bipimo nka FID hamwe nu manota yabantu bikubiyemo realism. Igitekerezo gikocamye ni uko amanota meza ya sync bisobanura videwo ifatika. Gukabya umunwa gukabije birashobora gutsinda amanota neza kandi bikagaragara nabi, bityo isuzuma ryabantu rikomeza kuba ngombwa. Imwe mu mbogamizi nuko fonema nyinshi zisangiye imiterere yumunwa igaragara, cyangwa viseme, bityo ikarita yerekana amajwi kumunwa ni myinshi-imwe. Gukoresha nabi bikurikira biturutse ku ikoranabuhanga: guhuza ijwi rya clon hamwe no guhuza iminwa birashobora gutanga amagambo yemeza ibinyoma. Abashakashatsi berekanye ko kudahuza fonema na visem, cyane cyane iminwa idashobora gufunga kuri 'm', 'b' na 'p', bishobora kwerekana ibinyoma.
Abatanga amasomo kumurongo batanga inyigisho mucyesipanyoli kandi bagakoresha iminwa kugirango umunwa wumwigisha uhure nijwi ryahinduwe, ukareba ibyahinduwe mubisobanuro bya videwo.
Umwanditsi wa videwo ahuza umunwa wumukinnyi kumurongo wibiganiro byongeye kwandikwa nyuma yo kuraswa, byemejwe nabakinnyi, aho kugirango bisubireho.
Umugenzuzi w'ukuri asuzuma clip ya virusi y'umunyapolitiki maze abona iminwa itigera ifunga byimazeyo amajwi ya 'b', 'p' na 'm', ikimenyetso kizwi cy'impimbano.
Umushakashatsi akoresha Wav2Lip kumurongo wamashusho yerekana umutwe hanyuma agatsinda ibisubizo hamwe na LSE-D na LSE-C kugirango abigereranye nuburyo bushya bwo gukwirakwiza bushingiye kumunwa.
Guhuza iminwa bigenda bihinduka ikintu gisanzwe cyo gukinisha no gukoresha amashusho yerekana amashusho, kandi ubushakashatsi bugenda bugana kumurongo wo hejuru, amenyo atajegajega hamwe nururimi, hamwe no guhuza isura yose yo mumaso hamwe no kugenda mumutwe aho kuba umunwa gusa. Kumenya ni isiganwa ryintwaro: ibimenyetso byubucamanza nka phoneme-viseme bidahuye bigenda bigabanuka uko generator itera imbere. Inkiko zimwe zatangiye kugenga itangazamakuru ryibeshya, cyane cyane amatora. Kwemererwa numuntu werekanwe, kumenyekanisha neza, no kwerekana ibimenyetso byerekana ko bizakomeza kuba intandaro yo gukoresha inshingano.
Ibaruwa y'amakuru y'ubuntu
Inkuru eshatu zizewe za AI buri gitondo cy'iminsi y'akazi, zanditse mu rurimi rworoshye. Ubuntu iteka ryose, nta kwamamaza.
Imeri imwe buri munsi y'akazi. Kwiyimura mu buryo bumwe. Ntitwigera tugurisha cyangwa ngo dusangire aderesi yawe.
Isuzume
Ibisubizo ako kanya kuri buri kibazo, n'icyemezo gishobora gusangirwa gifite ID yagenzurwa umaze gutsinda isomo.
Fasha uburezi bwa AI bw'ubuntu. AI Understanding ni umuryango utegamiye kuri leta wa 501(c)(3) — nta kwamamaza, nta kwishyura, iteka ryose. Tanga impano
Gukoresha amajwi ya AI umunwa sync ihindura amashusho yisura kugirango umunwa wacyo uhure numurongo mushya wo kuvuga, uteganya imiterere yiminwa iva mumajwi ubwayo. Nibyingenzi byo gushidikanya amasomo na firime mu zindi ndimi no gukemura ibiganiro byongeye kwandikwa. Tekinike imwe irashobora kandi gutuma abantu nyabo bagaragara bavuga amagambo batigeze bavuga.
Numunwa nyawo uhishe, generator igomba kwishingikiriza kumajwi kugirango ihitemo imiterere yumunwa.
Abavangura bahuguwe hamwe na generator bari abacamanza bafite intege nke za sync. Impuguke yakonje itanga ikimenyetso cyizewe.
LSE-D ni intera, bityo indangagaciro ntoya bivuze gushyiramo amajwi numunwa byegereye, bivuze guhuza neza.
Ikadiri yerekana indangamuntu hamwe nimiterere. Amajwi atanga imiterere yumunwa.
Wav2Lip yakoraga ku bihingwa bito byo mu maso, ku buryo umunwa n'amenyo byakunze kugaragara byoroshye kuruhande rukarishye.
Komeza wige
Abandi bayobozi batoranijwe kuriyi ngingo
HejuruUbuyobozi bukurikira
Amazi atagaragara kumashusho ya AI na Audio
AI igaragara