Whisper
Liggéeyukaay baatu whisper dafay tënk baat bu nekk buñ bind ci waxtu ndoorte ak waxtu jeexal ci audio bi.
Résumé
This turns a flat transcript into a clickable, searchable timeline used for captions, dubbing, and editing.
Plongeur bu xóot
OpenAI's Whisper ab enkodeer-dekodeer la buy soppi kàddu yi, waaye li muy génne ci segment bu nekk dafay joxe timestamp yu gàtt, du jox benn baat bu nekk. Liggéeyukaay ci wàllu baat mooy feexal bërëb boobu. Kaf gi gëna bari luñu koy jëfandikoo (whisper-timestamped ak WhisperX) dafay jàng poids cross-attention model bi: decoder bi dafay bàyyi xel ci kadre audio yi muy génne token bu nekk, ba noppi barabu bàyyi xel bi gëna mag dafay màndargaal kañ lañu wax baat boobu. Dynamic Time Warping dafay forse ab mapping bu monotone, budul jaxasoo ci palanteer audio bu 30 seconde. WhisperX lu moy loolu dafay doxal xeetu fonem bu wuute buñ forse (lu melni wav2vec 2.0) ci kaw bind bu Whisper ngir am ay pegg yu gëna ñaw. Lépp soo ko boolee mu am baat bu nekk ñu tampoon ko ci fukki milisegond.
Gis-gis xarala
Whisper dafay jëfandikoo audio ci 30 seconde, ñu soppi ko ci spectrogram log-Mel, ñu kode ko ci 50 kadre ci seconde (benn kadre 20 ms yu nekk). Fàttaliku croix dafay lëkkale token buñ dekode bu nekk ak kaadar yooyu; argmax bi dafay nekk waxtu baat bi. Dynamic Time Warping dafay fexe nu ñu delloo ci anam wu monotone suko defee timbre yi duñu musa dellu ginaaw. Alternatif yiñ forse ngir méngale transkripsioŋ bi ak audio bi ci wàllu fonem, loolu dafay joxe boor yu gëna set moo gën toppu attention bu ñor bi.
njeextalu pexe
Dugg ak yegg
Dafay gëna yombal jëfandikoo gi jaaraleko ci transkripsioŋ, nettali ak interfaasu baat.
Njëgg ak budget
Ekipu mejaa yi mën nañu yónnee audio bu leer ci anam wu gëna gaaw te seen xaalis gëna néew.
Gaawaay ak yaatuwaay
Sistem yiy jàkkarloo ak kiliyaan bi mën nañu def waxtaan ci anam wu gëna yaatu.
Ëlëgu Whisper
Xaarandil alignment buñ lakk ci decoder bi moo gën ñu koy boole ci ganaw, boole ci poñ yu wóor ci baat bu nekk suko defee editër yi xam ban timestamp lañu wara wóolu. Liggéeyu streaming ngir sous-titre yi ci saasi mingi gëna yomba, niko robuste bi ci oparlër yuy jaxasoo, music, ak weccoo kode. Kom xeetu lakk yu bari yi ñungi magg, kalite bi ci lakk yu nééw yi defa wara tëj bërëb bi ak Anngele, suko defee dubbing otomatik ak kapsioŋ yu mel ni karaoke gëna wóór.
Doxal ci àdduna dëgg
Defar ay mbindu YouTube ak TikTok yu baat yi di feeñ ci ekraŋ bi ni ñu ko waxee
Editëru sous-titre yu am doole yi lay may nga klike ci benn baat nga dem ci waxtu audio bi
Tekki script yi ak audio orginaal bi ngir mëna doublage otomatik ak lip-sync
Tabax ay archive podcast yuñ mëna seetee, fu ab laajtu bind di wàcci ci simili biñu ko wax
Risk yi ak balustrade yi
Jëfandikoo baat ci anam wu jaarul yoon ak niru ak nit dafay gëna yokk sudee nanguwul.
Jaar-jaar mën na wàññeeku ci aksan yi, dialect yi wala barab yu bari xumbaay.
Audio synthetik mën nañu ko jaawale ak wax ju dëggu sudee amul etiket bu leer.
Roadmap ngir samp gi
Wutal ndigal bu leer ngir jàpp baat bi, klone ko ak jëfandikoowaat ko.
Saytu kalite ci kàddukat yu bari ak anam yu bari ci ginaaw.
Mandargal kañ la nit wara xoolaat wala nangu ay génne.
Etiketu audio synthetik te nga denc dokimaa ci fimu bawoo ngir mëna lim.
Weyal di banneexu
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Whisper Timestamped Word Alignment quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gis bi ci topp
Xamme kàddu yu am tuuti
Laaj yi ñuy faral di laaj
What is Whisper Timestamped Word Alignment?
Liggéeyukaay baatu whisper dafay tënk baat bu nekk buñ bind ci waxtu ndoorte ak waxtu jeexal ci audio bi. Loolu dafay soppi transkripsioŋ bu plat nekk kalendriye buñ mëna klike, seetee, ñu jëfandikoo ko ci sous-titre yi, dubbing yi ak soppali yi.
Lan la alignment niveau mots yokk ci bi Whisper amul?
Whisper dafay joxe timestamp yu xawa jege ci segment bu nekk; alignment dafay yokk waxtu ndoorte ak jeexal baat bu nekk.
Ban siñaal bi ci biir la whisper-timestamped di jëfandikoo ngir gis kàddu yi ci waxtu wi?
Cross-attention dafay wane ban kadre audio la dekodeer bi bàyyi xelam ci bi muy génne token bu nekk, di wane waxtu wi.
Lu tax ñuy jëfandikoo Deformaasioŋ Dynamik ci waxtu liggéey bi?
DTW dafay fexe timestamps yi dem ci kanam ci yoon wu baax te baat yi duñu jaxasoo, defar kalendriye bu jaar yoon.
naka lay def ba WhisperX gëna ñaw ay fësal kàddu suñu ko méngale ak bàyyi xel bu ñor?
WhisperX dafay méngale mbindu Whisper ci jëfandikoo xeetu fonem bu melni wav2vec 2.0 ngir am boor yu gëna set te baña bàyyi xel ci.
Ban vitees la encodeur Whisper di defaree kadre audio yi?
Whisper dafay kode spectrogram log-Mel ci lu tollu ci 50 kaadar ci segond bu nekk, wala benn kaadar ci 20 milisegond yu nekk.