Ci bi mu melGis bi ci topp
Modèlu eskalaasioŋ boppam ci kaw Kubernetes
Xarala
GUIDE teknik
KServe ak xeetu liggéey ci Kubernetes
KServe platform buñ yamale la, juddoo ci Kubernetes ngir joxe xeetu jàngu masin ci anam wu yaatu.
Déglul tegtal biiIA-nettali · ~2 min
Ci xët wii2 simili jàng
Résumé
Dafay jox ekip yi benn anam bu leer ngir mëna jëfandikoo ay model yu am autoscaling, génne kanari, ak escale-to-zero, di dindi li ëpp ci plomberie Kubernetes.
Plongeur bu xóot
Bu njëkk ñu xamee ko ci KFServing te mingi juddoo ci projet Kubeflow, KServe dafay màndargaal ab jumtukaay buñ jagleel InferenceService. Danga bind benn fichier YAML bu gàtt buy joxoñ benn model buñ denc ci dencukaay mbir (S3, GCS, Azure Blob), KServe mooy liggéey ci leneen. Dafay jàppale inference yiy wax luy waaja am ak, di gëna yokk, LLM biy defar. KServe dafay yónnee ay 'waxtu liggéey' yuñ defaree lu jiitu ngir kaadar yiñ gëna xam (TensorFlow, TorchServe, Triton, scikit-learn, XGBoost, Hugging Face) te dafay jàppale konteneer yiñ personaaliseer. Ñu tabax ko ci kaw Knative Serving ak benn layer reso (Istio wala lu mel noonu), dafay joxe autoscaling bu lalu ci laaj boole ci dëgg-dëgg scale-to-zero, kon model yu idle yi duñu lekk benn ordinatër. Dafay yamale itam API biy wax luy waaja am ci Open Inference Protocol, suko defee kiliyaan yi di waxtaan ak model bu nekk ci anam wu wuute, kaadar bi du ci dara.
Doxal ci àdduna dëgg
Benn bànk dafay dugal ab xeetu poñ leble ci bind ab InferenceService YAML bu am 10 ligne di joxoñ xeetu S3, ak KServe di yoriinu eskalaasioŋ otomatik ak dugg.
Benn ekipu e-commerce dafay jëfandikoo KServe canary rollouts ngir yónnee 10 pursaa ci dem bi ak dikk bi ci xeetu xalaat bu bees, ba noppi dem ba 100 pursaa su metrics yi xoolee bu baax.
Laboratoire buy gëstu dafay liggéey ci fukki-fukki model yu ñu bariwul luñu koy jëfandikoo ak scale-to-zero, kon model bu nekk dafay wëréelu su amee laaj bu yegsi te du lekk benn GPU bimu nekkee ci idle.
Benn ekipu MLOps dafay jëfandikoo benn komponent transformatër KServe ngir doxal nataal buy soppi yaatuwaayam ak normalisasioŋ balaa prediktër bi di doxal benn xeetu gis-gis bu Triton di liggéey.
Ëlëgu KServe ak xeetu liggéey ci Kubernetes
KServe mingi gaaw jëm kanam ci IA buy defar, yokk benn piste bu lalu ci LLM ak ay man-man yu melni KV-cache-aware routing, model cache, ak prefill / decode buñ xaaj ngir model làkk yu mag. Xaarandi lëkkaloo bu gëna xóot ak motëri inference yu melni vLLM, gëna baax ci liggéeyum node yu bari ngir model yu rëy lool ci benn GPU, ak yoon ci niveau gateway ngir balance charge bu sukkandiko ci token. Bi mu nekkee projet buy dundal CNCF, mingi nekk de facto standard bu ubbeeku ngir teg ay model ci ginaaw Kubernetes, di wàññi bërëb bi am ci digganté ay mbiri gëstu ak ay poñ yu mujj yu am doole.
Teg ko ci jëf
Lu tax mu am solo
- Njëgg ak budget. Dogal yi architecture di jël dañuy indi njariñ ak njëgu liggéey bi ay at ci ginaaw.
- dogal yu gëna leer. Njàngalem xarala yi dafay jàppale ekip yi ñu tànn li gën, te baña yam ci li gëna bees daal.
- Xool kalite. Tanneef yu gëna baax ci wàllu ingeñër dina wàññi jafe-jafe yi ci wàllu wóor ci liggéey bi.
Moytul
- Optimize benn benchmark mën na nëbb ñakk kattan yu gëna yaatu ci sistem bi.
- Njëg li ñuy fay ci infrastructure yi ak ci toppatoo dañuy faral di suufeel.
- Bu sistem yi di gëna xawa jafee xam, jafe-jafe yi am ci wàllu kaaraange ak seetlu mën nañu gëna bari.
Jéego yi ci topp
- Mandargal latency, kalite, ak njëg yi laata ngay jëfandikoo.
- Benchmark ci biir sargal ak done yu dëggu.
- Jumtukaay bi di saytu njuumte yi, derive bi ak njeextalu jëfandikukat bi.
- Waajal rollback ak yooni tontu ci jafe-jafe yi laata ngay eskale.
Weyal di banneexu
Bulletin bu amul fayda
Jàngal xibaar AI bu bes
Ñatti jaar-jaari IA yuñ firndeel bis bu nekk ci ayu-bis bi, ñu bind leen ci Àngle bu leer. Gratis ba fàww, amul yëgle.
Benn email bis bu nekk ci ayu-bis bi. Benn klik nga dindi abonemaa bi. Dunu musa jaay wala séddoo sa adres.
Nattal sa bopp
Take the KServe and Model Serving on Kubernetes quiz
Feedback ci saasi ci bépp tontu, ak sertifikaa buñ mëna séddoo ak ID buñ mëna saytu soo jàllee ci njàng mi.
Jàppale njàngum IA te doo fay. AI Understanding mbootaay la buy def te Yàlla tax — amul yëgle, amul paywall, musul am. Defal ab sarax
Laaj yi ñuy faral di laaj
Luy KServe ak xeetu liggéey ci Kubernetes?
KServe platform buñ yamale la, juddoo ci Kubernetes ngir joxe xeetu jàngu masin ci anam wu yaatu. Dafay jox ekip yi benn anam bu leer ngir mëna jëfandikoo ay model ak autoscaling, génne canary, ak scale-to-zero, di dindi li ëpp ci plomberie Kubernetes.
Lan moo nekkoon tuuru KServe balaa muy nekk projet bu nekk boppam?
KServe mingi juddoo ci KFServing ci biir projet Kubeflow balaa muy nekk platform bu moom boppam.
Lan mooy jumtukaayu Kubernetes bi njëkk bi nga tànn ngir dugal ab model ak KServe?
Yaa ngi fësal ab jumtukaay buñ jagleel InferenceService, lu gëna bari ci YAML, ngir dugal ak tabb ab xeetu liggéey.
Ban mënin mooy tax model KServe yi duñu def benn ordinatër ba keroog laaj bi yegsi?
Tabax ci kaw Knative, KServe dafay jàppale scale-to-zero, wàcce ay replika ci zero sudee amul trafik ak tàmbali sedd ci laaj.
Ban projet mooy joxe autoscaling bu lalu ci laaj bu KServe?
KServe dafa tabax ci kaw Knative Serving, mooy eskale ay replika yu sukkandiko ci concurrence wala tolluwaayu laajte ba noppi may eskale-ba-zero.
naka la ko KServe di def ba mëna am ay model ci pod biy tàmbali?
Dencukaay yiy tàmbali dencukaay yi dañuy yebbi modelu artefact yi ci dencukaay yi (lu melni S3 wala GCS) ci biir pod bi, di dindi model yi ci nataal bi.
Weyal di jàng
Gid yu jëm ci loolu
Tann nañu yeneen njiit ngir topic bii