Amabwiriza ya AI

Gukoresha amajwi ya AI

Gukoresha amajwi ya AI umunwa sync ihindura amashusho yisura kugirango umunwa wacyo uhure numurongo mushya wo kuvuga, uteganya imiterere yiminwa iva mumajwi ubwayo.

Umva iki gitaboAI-byavuzwe · ~3 min

Kuriyi page4 min soma
  1. Incamake
  2. Kwibira cyane
  3. Gushyira mu bikorwa Isi
  4. Kazoza ka Audio-Iyobowe na AI Umunwa
  5. Kubishyira mubikorwa
  6. Komeza Ubushakashatsi
  7. Ibibazo bikunze kubazwa

Incamake

Nibyingenzi byo gushidikanya amasomo na firime mu zindi ndimi no gukemura ibiganiro byongeye kwandikwa. Tekinike imwe irashobora kandi gutuma abantu nyabo bagaragara bavuga amagambo batigeze bavuga.

Kwibira cyane

Wav2Lip, yasohowe mu 2020 n'abashakashatsi bo muri IIIT Hyderabad (ACM Multimedia), niyo moderi izwi cyane ifunguye iminwa hamwe nicyitegererezo cyingirakamaro kumurima. Kuri buri kintu cyagenewe intego, generator yakira inyongeramusaruro eshatu: isura nigice cyayo cyo hepfo yipfundikiriye, ikadiri yerekana umuntu umwe utanga isura, hamwe nidirishya rigufi rya mel-spekrogramu yamajwi hafi yicyo gihe. Umuyoboro wa encoder-decoder noneho ushushanya mukarere ka munwa, kandi ibisubizo byahujwe gusubira muri videwo yumwimerere. Umusanzu wingenzi ni "impuguke" yabanje gutozwa ivangura rishingiye kuri SyncNet. Ikomeza gukonjeshwa mugihe cyamahugurwa kandi igacira urubanza niba gukora akanya gato kumunwa uhuye nijwi. Uburyo bwambere bwahuguye ivangura rikorana hamwe na generator, kandi abo bavangura bagaragaye nkabacamanza babi ba sync. Wav2Lip yakoresheje kandi ivangura-ryiza ritandukanye. Intege nke zayo zari ibihingwa byo mu kanwa bidakemutse, amenyo atagaragara kandi adasanzwe cyangwa guhindagurika kumupaka wa mask. Nyuma sisitemu nka VideoReTalking, MuseTalk hamwe na diffusion ishingiye kuri LatentSync igamije ibisubizo bikaze, bihamye. Ubwiza bugenzurwa muburyo butandukanye. LSE-D ipima intera iri hagati y amajwi na videwo ya SyncNet, aho hepfo ari nziza. LSE-C ipima ikizere cya SyncNet, aho hejuru ari nziza. Ibipimo-byiza-bipimo nka FID hamwe nu manota yabantu bikubiyemo realism. Igitekerezo gikocamye ni uko amanota meza ya sync bisobanura videwo ifatika. Gukabya umunwa gukabije birashobora gutsinda amanota neza kandi bikagaragara nabi, bityo isuzuma ryabantu rikomeza kuba ngombwa. Imwe mu mbogamizi nuko fonema nyinshi zisangiye imiterere yumunwa igaragara, cyangwa viseme, bityo ikarita yerekana amajwi kumunwa ni myinshi-imwe. Gukoresha nabi bikurikira biturutse ku ikoranabuhanga: guhuza ijwi rya clon hamwe no guhuza iminwa birashobora gutanga amagambo yemeza ibinyoma. Abashakashatsi berekanye ko kudahuza fonema na visem, cyane cyane iminwa idashobora gufunga kuri 'm', 'b' na 'p', bishobora kwerekana ibinyoma.

Gushyira mu bikorwa Isi

Abatanga amasomo kumurongo batanga inyigisho mucyesipanyoli kandi bagakoresha iminwa kugirango umunwa wumwigisha uhure nijwi ryahinduwe, ukareba ibyahinduwe mubisobanuro bya videwo.

Umwanditsi wa videwo ahuza umunwa wumukinnyi kumurongo wibiganiro byongeye kwandikwa nyuma yo kuraswa, byemejwe nabakinnyi, aho kugirango bisubireho.

Umugenzuzi w'ukuri asuzuma clip ya virusi y'umunyapolitiki maze abona iminwa itigera ifunga byimazeyo amajwi ya 'b', 'p' na 'm', ikimenyetso kizwi cy'impimbano.

Umushakashatsi akoresha Wav2Lip kumurongo wamashusho yerekana umutwe hanyuma agatsinda ibisubizo hamwe na LSE-D na LSE-C kugirango abigereranye nuburyo bushya bwo gukwirakwiza bushingiye kumunwa.

Kazoza ka Audio-Iyobowe na AI Umunwa

Guhuza iminwa bigenda bihinduka ikintu gisanzwe cyo gukinisha no gukoresha amashusho yerekana amashusho, kandi ubushakashatsi bugenda bugana kumurongo wo hejuru, amenyo atajegajega hamwe nururimi, hamwe no guhuza isura yose yo mumaso hamwe no kugenda mumutwe aho kuba umunwa gusa. Kumenya ni isiganwa ryintwaro: ibimenyetso byubucamanza nka phoneme-viseme bidahuye bigenda bigabanuka uko generator itera imbere. Inkiko zimwe zatangiye kugenga itangazamakuru ryibeshya, cyane cyane amatora. Kwemererwa numuntu werekanwe, kumenyekanisha neza, no kwerekana ibimenyetso byerekana ko bizakomeza kuba intandaro yo gukoresha inshingano.

Kubishyira mubikorwa

Impamvu ari ngombwa

  • Umuvuduko n'igipimo. AI igaragara irashobora gukora igenzura, gutahura, no gutondekanya imirimo kurwego.
  • Kubaka amahitamo. Amakipe arema arashobora prototype ibitekerezo byihuse hamwe nintoki nkeya.
  • Itsinda hamwe nakazi. Ibikorwa birashobora gukoresha amashusho nibimenyetso bya videwo byari bigoye gutunganya.

Witondere

  • Uburenganzira bwishusho hamwe no kwemererwa birashobora guhinduka ibyago byemewe n'amategeko niba ibimenyetso bidasobanutse.
  • Imikorere yicyitegererezo irashobora gutandukana kumurika, demografiya, nibidukikije.
  • Ibyiza byibinyoma birashobora kutamenyekana keretse niba ibyiringiro byateganijwe bikurikiranwa.

Intambwe ikurikira

  1. Sobanura ibipimo byo kwemererwa kugiciro, kwibutsa, nibiciro byamakosa.
  2. Gerageza hamwe namakuru ajyanye nuburyo nyabwo bwo gukora.
  3. Ongeraho isubiramo ryabantu kubwizere buke cyangwa guhanura cyane.
  4. Kurikirana icyitegererezo cya drift hanyuma uhindurwe nyuma ya kamera cyangwa dataset ihinduka.

Komeza Ubushakashatsi

Ibaruwa y'amakuru y'ubuntu

Ibuka amakuru ya buri munsi ya AI

Inkuru eshatu zizewe za AI buri gitondo cy'iminsi y'akazi, zanditse mu rurimi rworoshye. Ubuntu iteka ryose, nta kwamamaza.

Imeri imwe buri munsi y'akazi. Kwiyimura mu buryo bumwe. Ntitwigera tugurisha cyangwa ngo dusangire aderesi yawe.

Isuzume

Take the Audio-Driven AI Lip Sync quiz

Ibisubizo ako kanya kuri buri kibazo, n'icyemezo gishobora gusangirwa gifite ID yagenzurwa umaze gutsinda isomo.

Tangira ikibazo

Fasha uburezi bwa AI bw'ubuntu. AI Understanding ni umuryango utegamiye kuri leta wa 501(c)(3) — nta kwamamaza, nta kwishyura, iteka ryose. Tanga impano

Ibibazo bikunze kubazwa

Ni ubuhe buryo bwo gukoresha amajwi ya AI ikoreshwa na AI?

Gukoresha amajwi ya AI umunwa sync ihindura amashusho yisura kugirango umunwa wacyo uhure numurongo mushya wo kuvuga, uteganya imiterere yiminwa iva mumajwi ubwayo. Nibyingenzi byo gushidikanya amasomo na firime mu zindi ndimi no gukemura ibiganiro byongeye kwandikwa. Tekinike imwe irashobora kandi gutuma abantu nyabo bagaragara bavuga amagambo batigeze bavuga.

Kuki Wav2Lip ihisha igice cyo hepfo yintego?

Numunwa nyawo uhishe, generator igomba kwishingikiriza kumajwi kugirango ihitemo imiterere yumunwa.

Niki cyatumye ivangura rya sync ya Wav2Lip ritandukana nuburyo bwabanje?

Abavangura bahuguwe hamwe na generator bari abacamanza bafite intege nke za sync. Impuguke yakonje itanga ikimenyetso cyizewe.

Nigute LSE-D igomba gusomwa?

LSE-D ni intera, bityo indangagaciro ntoya bivuze gushyiramo amajwi numunwa byegereye, bivuze guhuza neza.

Ni uruhe ruhare rw'ikadiri yerekana muri Wav2Lip?

Ikadiri yerekana indangamuntu hamwe nimiterere. Amajwi atanga imiterere yumunwa.

Ninde wari ufite intege nke zigaragara za Wav2Lip yumwimerere?

Wav2Lip yakoraga ku bihingwa bito byo mu maso, ku buryo umunwa n'amenyo byakunze kugaragara byoroshye kuruhande rukarishye.