Lu xew
llama.cpp genne na 0.4.0 ci GitHub ci 4 septembre. DeepSeek-V4-Flash-Vision-Exp. Dafay yokk itam ay parametru dugal wideo, ay yamaleg ëmbiitu serwër bu nekk, jàngu tensor bu taye, coppite ci yoonu kàngam, yokkute ci cache KV, ak gëna xéewale backend yu bari. ggml bi dafay yeesal 0.22.0 ba 0.23.0. Projet bi dafa wax ni versioŋ bi dafay yokk ay flash yu néew, jëfandikoo asynchrone ak API yu aju ci joxe, xew-xew RPC ak API async, ak ndimbalu dem ak dikk Apple RDMA. Xët wi dafay lim tabax guddi gu nekk buñu xamme ni b10809. Du bind disponibilite binaire buñ defar, li ñuy laaj ci instalaasioŋ bi, séddaleb poid model bi, wala njëg yi.
Xëtu génne bu GitHub dafay wane v0.4.0 ni mooy génne bu mujj ba noppi bind waxtu génne bu 4 septembre ci 19:56, te leeralul zone zone ci bind biñ joxe. Li ñuy génne dafa amaale coppite ci API yu melni llama_lazy_mode, seytu dayo tampon kwantizer, sesioŋ yu bees ak xeetu etaa yi, ak ndimbalu tokenization multimodal yu bees.
Coppite yiñ amal ci model bi bokkuna ci ndimbalu architecture Qwen3.8-Flash-Next, ndimbalu Nemotron-3-75B-A9B, ndimbalu DSpark ngir Nemotron 3.5, ndimmbalu nanbeige4.2-3B, jàngat taarixu tensor bu taye, xool bu nekk yokkute, ak kaaraange ci RAM peaks ci diiru model loading.
Coppite yi am ci anam yu bari ak ci serwër yi bokkuna ci jàppale DeepSeek-V4-Flash-Vision-Exp, tànneefi liiñ komand wideo, àppu ëmbiit bu nekk, URL done ngir media yi, sàmm bu njëkk ci génnug xalaat, ak bañ woote jumtukaayi ndimbal yu njëkk. UI bi itam dafay soppi jumtukaay-politigu ak jekkal jekkal, waaye source bi du joxe benn done jëfandikukat wala performance.
Ay leeral ci cosaan: github.com ↗
Lu tax mu am solo
Lii ab yeesal bu am solo la ci runtime open-source bi developpeur yi di jëfandikoo ngir tabax IA inference ak aplikaasioŋu multimodal. Modèle bimu yaatal mën na tax model yu bees yi ñu mëna natt ci sistem yu llama.cpp, ci noonu la jàppale wideo-input di yaatal xeetu media yi sistem yooyu mëna jëfandikoo. Source bi dafay leeral performance ak liggéey bi jëm ci mémoire, waaye du joxe benn benchmark bu moom boppam, kon njariñ yi ci jëfandikoo dañuy aju ci hardware bi, formaa model yi, ak configuration deployment bi.
Li ñuy génne dafay boole yeneen xeeti tabax yu bees ak benn kodu jëfandikoo bu bari, lu ci melni Qwen3.8-Flash-Next ak ndimbalu Nemotron-3-Puzzle. Loolu mën na wàññi liggéeyu boole bi ci developpeur yiy jàngat model yooyu, ndigam source bi du taxawal deggoo ci bépp platform wala configuration.
Coppite ggml bi dafa yokk jumtukaay ngir bàyyi xel ci lu néew, backend yu asynchrone, xew-xewu wooteb doxalin bu sori, ak Apple RDMA. Coppite yooyu mën nañu am solo ci jëfandikoo yi mëna jëfandikoo backend yooyu, waaye xëtu génne gi du xayma latency, throughput, jëfandikoo mémoire, wala yokkute ci wóor.
Parametru wideo, ndimbalu URL done ngir media yi, ak coppite yi am ci preprocessing multimodal dañuy joxe yoon wu gëna fëgër ngir aplikaasioŋ yiy jëfandikoo wideo ak yeneen media. Source bi waxul ndax mën-mën yooyu am nañu ci tabax yuñ defar wala ci ban model-specific limitations.
Mekanism buy weccoo xalaat: naka lay doxee
Saytu xarala yu bees yi ci ginaaw yokkute bii ci anam wu weccoo xalaat.
Which component of an AI application is the machine-learning model itself?
Li nga wara seetaan ci topp
Li ñuy topp ci génne yi, test yiñ jagleel backend bi, ak këyitu liggéey yi dañu wara leeral ni model bu bees bi ak man-mani wideo yi di doxee ci hardware biñ jàppale. Li gëna wóorul ci saasi mooy ndax jëfandikoo gi njëkk ci Qwen3.8-Flash-Next jotna liggéeyu optimisation biñ digoon, ak ni sparse-attention ak coppite ci mémoire di gëna baaxal liggéey bi.
Seetal yeesali gëna xéewale ci Qwen3.8-Flash-Next ak yeneen defar ngir yooni model multimodal yu bees yiñ yokk.
Seetal njariñu référence yiy tàqale li ñuy wax ci jëfandikoo xëtu génne gi ak njariñ yiñ natt ci gaawaay, jëfandikoo mémoire, ak concurrence.
Saytu dokimaa ci wàllu jëfandikoo gi, fichier model yiñ jàppale, li ñuy laaj ci hardware bi, ak waajal liggéey bi. Xëtu génne gi joxeewul leeral yooyu.