Overview
Muse imhando yemavara-kune-mufananidzo kubva Google inogadzira mapikicha nekuzadza masiki emifananidzo tokeni zvese kamwechete, zvichiita kuti ikurumidze kupfuura nhanho-ne-nhanho kupararira. Izvo zvine basa nekuti zvakaratidza kuti iwe unogona kuwana yepamusoro-mhando, yakanyatsoenderana mifananidzo pasina inononoka iterative denoising iyo majenareta mazhinji anovimba nayo.
Muse Masked Generative Imaging ndeyekombuta-kuona mafambiro anodudzira kana kugadzira midhiya yekuona yekuongorora, mashandiro, uye kugadzira.
Deep Dive
Muse inoshanda munzvimbo yakasarudzika yemufananidzo. VQGAN yakadzidziswa inoshandura pikicha kuita gidhi yematokeni akazara, semazwi ezvivharo zvekuvaka. Panguva yekudzidziswa, chikamu chikuru chezviratidzo izvi chinovharwa kunze, uye Transformer inodzidza kufanotaura kuti idzoke, inomisikidzwa pazvinyorwa zvakadzikiswa kubva kumhando yemutauro wakaoma nechando (T5-XXL). Panguva yechizvarwa Muse anotanga kubva kune-akafukidzwa gidhi uye anodhirodha mumatenderedzwa akafanana, achifanotaura akawanda ma tokeni padanho uye nekumisazve avo vasina chivimbo. Dhizaini-matanho maviri inotanga kuburitsa yakaderera-resolution token grid, ipapo super-resolution modhi inozadza yakakwira-resolution grid. Nekuti akawanda ematokeni anogadzirisa panguva imwe chete, iyo 900M uye 3B parameter modhi inoburitsa 256 kana 512 pixel mufananidzo mune mashoma mashoma ekupfuura.
Technical Insight
Iyo yakakosha trick inofanana decoding nekuvimba-kwakavakirwa remasking, inowanzonzi MaskGIT-maitiro sampling. Panzvimbo pekufanotaura chiratidzo chimwe chete panguva (autoregressive) kana kubudisa mazana enguva (kupararira), Muse anofanotaura zviratidzo zvose zvakavharwa, anochengeta avo vane chivimbo zvikuru, uye anovhara zvakare mamwe ose kutenderera kunotevera. Kushandisa yakaomeswa T5-XXL mavara encoder inopa kunzwisisa kwakasimba kwemutauro mahara, uye kushanda pane discrete tokeni kunoita kuti modhi ikonzere nezvemifananidzo senge mazwi.
Mastering Muse Masked Generative Imaging
Muse imhando yemavara-kune-mufananidzo kubva Google inogadzira mapikicha nekuzadza masiki emifananidzo tokeni zvese kamwechete, zvichiita kuti ikurumidze kupfuura nhanho-ne-nhanho kupararira. Izvo zvine basa nekuti zvakaratidza kuti iwe unogona kuwana yepamusoro-mhando, yakanyatsoenderana mifananidzo pasina inononoka iterative denoising iyo majenareta mazhinji anovimba nayo. Muse Masked Generative Imaging ndeyekombuta-kuona mafambiro anodudzira kana kugadzira midhiya yekuona yekuongorora, mashandiro, uye kugadzira. Kuti uvake kunzwisisa kwakadzama, bata Muse Masked Generative Imaging semuenzaniso wekushandisa, kwete chinhu chimwe chete: tsanangura zvinodikanwa, kujekesa fungidziro, uye patsanura izvo zvingaitwe nehurongwa hwakavimbika kubva kune zvichiri kuda kutonga kwenyanzvi.
Mukuita, zvikwata zvakasimba zvinoshandisa Muse Masked Generative Imaging chiyero chechokwadi nemashandiro anoita semhando yedata, kusiyana kwemwenje, uye kuenderana kwemazita. Ivo vanonyora zvakajeka maitiro ebudiriro, bvunzo vachipokana ne data rechokwadi uye mafambiro ebasa, uye iterate zvichibva pane zvakacherechedzwa maitiro ekutadza kwete kuhwina-nguva imwe chete yebhenji. Apa ndipo apo kunzwisisa kwe theoretical kunoshanduka kuve kugona kwakasimba pane chigadzirwa, mutemo, uye mashandiro.
Visual AI inogona kuita otomatiki yekuongorora, yekuona, uye yekumaka mabasa pachiyero. Panguva imwecheteyo, kodzero dzeMufananidzo uye kubvumirwa kunogona kuve njodzi dzepamutemo kana hunhu husina kujeka. Nzira yakatsiga ndeyekubatanidza kukurumidza kuyedza nekutonga: mhanyisa vatyairi vendege, tora humbowo, buritsa matanda esarudzo, uye urambe uchivandudza chengetedzo semaitiro emuenzaniso, zvinotarisirwa nemushandisi, uye zvinodikanwa zvekutonga.
Strategic Impact
Visual AI inogona kuita otomatiki yekuongorora, yekuona, uye yekumaka mabasa pachiyero.
Visual AI inogona kuita otomatiki yekuongorora, yekuona, uye yekumaka mabasa pachiyero. Mukutumirwa kwemhando yepamusoro, izvi zvinoshandurirwa kuita mitemo inoyerwa yekushanda, miganhu yevaridzi, uye tsika dzekudzokorora dzinodzokororwa kuitira kuti zvikwata zvikwire kuvimba pane kukwidza kusajeka.
Zvikwata zvekugadzira zvinogona prototype pfungwa nekukurumidza nekudzokororwa kwemaoko mashoma.
Zvikwata zvekugadzira zvinogona prototype pfungwa nekukurumidza nekudzokororwa kwemaoko mashoma. Mukutumirwa kwemhando yepamusoro, izvi zvinoshandurirwa kuita mitemo inoyerwa yekushanda, miganhu yevaridzi, uye tsika dzekudzokorora dzinodzokororwa kuitira kuti zvikwata zvikwire kuvimba pane kukwidza kusajeka.
Mashandisirwo anogona kushandisa masaini emifananidzo nemavhidhiyo ayo aimbove akaoma kugadzirisa.
Mashandisirwo anogona kushandisa masaini emifananidzo nemavhidhiyo ayo aimbove akaoma kugadzirisa. Mukutumirwa kwemhando yepamusoro, izvi zvinoshandurirwa kuita mitemo inoyerwa yekushanda, miganhu yevaridzi, uye tsika dzekudzokorora dzinodzokororwa kuitira kuti zvikwata zvikwire kuvimba pane kukwidza kusajeka.
Real-World Implementation
Rapid concept art uye mood board uko muimbi anoda misiyano mizhinji mumasekonzi kwete maminetsi.
Zero-kupfura pendi, sekubvisa chinhu uye kuita kuti modhi izadze nzvimbo yakafukidzwa zvinoenderana nenzvimbo dzakatenderedza.
Outpainting yekuwedzera foto kupfuura miganhu yayo yepakutanga yemabhana kana akasiyana mareshiyo.
Kugadzirisa pasina masiki, sekuchinja ruvara rwembwa kana denga kuti rinyure nekugadzirisa zvinyorwa nekumisikidza zvakare ma tokeni akanganisa.
Maitiro Ekuita
Muse Masked Generative Imaging mukuita
Rapid concept art uye mood board uko muimbi anoda misiyano mizhinji mumasekonzi kwete maminetsi.
Yekukurumidza pfungwa yehunyanzvi uye mabhodhi emafungiro apo muimbi anoda misiyano mizhinji mumasekonzi kwete maminetsi Zvikwata zvinowanzowana zvibodzwa zviri nani kana zvichitsanangudza zvikumbaridzo zvemhando yepamusoro kumberi, chengetedza nzira yekukwira kwevanhu yemakesi ekumucheto, uye kuteedzera zvese zvakawanikwa zvechigadzirwa nemitengo yekukanganisa nekufamba kwenguva.
Muse Masked Generative Imaging mukuita
Zero-kupfura pendi, sekubvisa chinhu uye kuita kuti modhi izadze nzvimbo yakafukidzwa zvinoenderana nenzvimbo dzakatenderedza.
Zero-kupfurwa pendi, sekubvisa chinhu uye kuita kuti modhi izadze dunhu rakafukidzwa richienderana nenzvimbo dzakatenderedza Matimu anowanzo kuwana mibairo iri nani kana achinge atsanangura zvikumbaridzo zvemhando yepamusoro kumberi, chengetedza nzira yekukwira kwevanhu yemakesi ekumucheto, uye kuteedzera zvese zvakawanikwa zvechigadzirwa uye mutengo wekukanganisa nekufamba kwenguva.
Muse Masked Generative Imaging mukuita
Outpainting yekuwedzera foto kupfuura miganhu yayo yepakutanga yemabhana kana akasiyana mareshiyo.
Outpainting yekuwedzera foto kupfuura miganhu yayo yepakutanga yemabhana kana akasiyana mareshiyo Matimu anowanzo kuwana mhedzisiro iri nani kana achinge atsanangura emhando yepamusoro kumberi, chengetedza nzira yekukwira kwevanhu yemakesi ekumucheto, uye kuteedzera zvese zvakawanikwa zvechigadzirwa uye mutengo wekukanganisa nekufamba kwenguva.
Muse Masked Generative Imaging mukuita
Kugadzirisa pasina masiki, sekuchinja ruvara rwembwa kana denga kuti rinyure nekugadzirisa zvinyorwa nekumisikidza zvakare ma tokeni akanganisa.
Kugadzirisa-isina masiki, sekuchinja ruvara rwembwa kana denga kuti rinyure nekugadzirisa zvinyorwa nekukurumidza uye kugadzirisa zvakare ma tokens akakanganisika Matimu anowanzo kuwana mibairo iri nani kana achinge atsanangura emhando yepamusoro kumberi, chengetedza nzira yekukwira kwevanhu yemakesi emupendero, uye kuteedzera zvese zvakawanikwa zvechigadzirwa nemitengo yekukanganisa nekufamba kwenguva.
Njodzi & Guardrails
Kodzero dzemifananidzo uye kubvumirwa kunogona kuve njodzi dzepamutemo kana provenance isina kujeka.
Kuita kwemuenzaniso kunogona kusiyanisa kupenya, huwandu hwevanhu, uye nharaunda.
Manyepo enhema anogona kusacherechedzwa kunze kwekunge zvikumbaridzo zvekuvimba zvikatariswa.
Implementation Roadmap
Tsanangura maitiro ekugamuchirwa echokwadi, kurangarira, uye mutengo wekukanganisa.
Tsanangura maitiro ekugamuchirwa echokwadi, kurangarira, uye mutengo wekukanganisa. Bata nhanho yega yega segedhi rehumbowo: kana maitiro asina kusangana, imbomira kuburitsa, vhara gaka, uye wobva wawedzera kushandiswa.
Edzai nedata rinoenderana nemamiriro chaiwo ekugadzira.
Edzai nedata rinoenderana nemamiriro chaiwo ekugadzira. Bata nhanho yega yega segedhi rehumbowo: kana maitiro asina kusangana, imbomira kuburitsa, vhara gaka, uye wobva wawedzera kushandiswa.
Wedzera ongororo yemunhu kune yakaderera-kusavimbika kana yakakwirira-inokanganisa kufanotaura.
Wedzera ongororo yemunhu kune yakaderera-kusavimbika kana yakakwirira-inokanganisa kufanotaura. Bata nhanho yega yega segedhi rehumbowo: kana maitiro asina kusangana, imbomira kuburitsa, vhara gaka, uye wobva wawedzera kushandiswa.
Tevera modhi kudonha uye simbisa mushure mekuchinja kwekamera kana dataset.
Tevera modhi kudonha uye simbisa mushure mekuchinja kwekamera kana dataset. Bata nhanho yega yega segedhi rehumbowo: kana maitiro asina kusangana, imbomira kuburitsa, vhara gaka, uye wobva wawedzera kushandiswa.