Dellu ci xibaar yi
YeesalAI Understanding

Benchmark gisna ni ndawu juntuwaay yu bees yi mënnañu defar ay woote jumtukaayi jëmmal hardware, waaye wóor gi defay wuute

Benn référence bu bees bu arXiv dafa wane ni agent IA yu ubbeeku yi mën nañu def ay liggéey yu bari ci jumtukaayi MCP, fekk tegtali jumtukaay yi, guddaayu context bi ak configuration agent yi dañu am njeexital yu am solo ci wóor gi.

6 min readRead the primary source
Primary-source image accompanying Benchmark finds local AI agents can handle many hardware-design tool calls, but reliability varies
Këyitu xët bu njëkkSource biñ enregistre
Siiwalkat
arxiv.org
Lëkkalekaayu cosaan
arxiv.orghttps://arxiv.org/abs/2608.26199
Xeetu balluwaay
Këyitu njëkk - ab yëgle ofisel, këyit, dosiye, wala xëtu pàrti bu njëkk bi ñuy jàng ci saasi.
KontekstXam lii ci 60 seconde

Tambalil fii

Term yu am solo

Référence
Test buñ yamale wala ensemble done yuñ jëfandikoo ngir natt ak méngale liggéeyu model bi.
API (interfaasu prograam aplikaasioŋ)
Benn anam buñ tëral ngir benn sistem losisel mëna yónnee ay laaj ak jot tontu ci beneen sistem.
MCP (Protokolu Kontekst Modèlu)
Protocole bu ubbeeku buy may aplikaasioŋu IA yi ñu mëna déngoo ak jumtukaayi biti, balluwaayi done, ak joxekati kontekst ci anam wuñ miin.
Nattal sa boppQuiz Agent IA

Lu xew

Gëstukat yi dañu natt juróom ñaari xeeti làkk yu ubbeeku, yuñ jëfandikoo ci gox bi, ci wàllu jëmmal liggéey buñ simulee ci serwëru Model Context Protocol. Referanse bi dafay muur coppite yu benn-benn, chaîne dependence yu bari, laaj yu baaxul, laaj yuñ bind bu baaxul ak liggéey yu lalu ci serwëru jumtukaay yu bari. Këyit dafa wax ni model yu dëgër yi def nañu lu jege-matt ci yenn workflows, waaye performance soppeeku bu baax ak prompting ak design agent.

Këyit bi ñu joxe arXiv ci 25 Uut, laaj na ndax ay ndawu juntuwaay yu bees yu ñuy doxalee ci ay xeetu lakk yu yaatu yu ñu dugal ci gox bi munnañu otomatise ci anam wu wóór ay ligeeyukaay yu ñu defar ci ay juntuwaay yu ñuy woowe ay jumtukaay. Source bi dafay leeral workflows yooyu ni ay operation yu ñuy baamtu ak yu lalu ci dependence, lu ci melni sos ay composant, yokk ay port ak ay conexioŋ ci kablaas. Ndax sujet bi ab ndawu liggéey la buy jaxasoo ak ab environmaa bu jëmmal, jàngat bi dafay jàngat lu ëpp ndax ab model mën na defar mbind mu gëm: dafay xool ndax ab ndawu liggeykat bi dafay def toppalante bi ñuy seentu ci woote jumtukaay yi, boole ci sargal nekkinu environmaa bi ak dependency yi.

Ngir defar environmaa test bi, gëstukat yi dañu tabax benn serwër MCP buy wane tolluwaayu logique ak dependence bu jumtukaayu jëmmal hardware buñ jëfandikoo ci developpement system embedded. bi dafa amaale ay anam yu bari yu mëna indi njuumte: soppali benn jëfandikoo, chaine dependence yu bari, laaj yu baaxul, laaj yuñ bind bu baaxul ak jumtukaayi serwër yu bari. Jariñ bii dafa am solo ndax wootu jumtukaay mën na nekk lu wóor ci wàllu sintaks te ba leegi mënu ñu ko jëfandikoo sudee dañu ko def ci yoon wu jaarul yoon, dafay jëm ci mbir mu baaxul wala sudee mënul joxe leeral ci coppite yiñ njëkka def ci def liggéey bi. Source bi xamul jumtukaayu moomeel bi wala joxe lim liggéeyu benchmark ci bind biñ joxe.

Gëstukat yi jàngat nañu juróom ñaari xeeti open-source ba noppi ñu méngale tànneefi agent-pipeline yu bari. Lu ci melni kàddu yi ak matt gi sistem bi di sàkku, bariwaayu leeral yi ci jumtukaay yi, yaatuwaayu muy wax ci model bi ak ndax benn ndawu liggéey moo yor liggéey yi wala ñu xaaj ko ay ndawu liggéey yu bari. Sunu sukkandikoo ci abstract bu këyit bi, dañu defar référence bi ngir xoolaat kàttanu model bi ak configuration bi. Taxawaay boobu amna solo ndax liggéeyu benn model ci environmaa bi jumtukaay bu leer mën na soppiku su tegtal yi ko wër, jaar-jaar bi am wala xaaj liggéey bi soppeeku.

Këyit dafay wax ci njariñ yu bari yu aju ci configuration bi. Modèle yu dëgër yi def nañu coverage bu mat sëkk ci woote yi ñuy seentu ci workflow yiñ def ci benchmarked, waaye wóor gi mingi aju bu bari ci structure liggéey bi ak ci configuration agent bi. Tegtal jumtukaay yu gëna mat sëkk dañuy wàññi gaañ-gaañu yi. Few-shot prompting moo waral yenn model yi ñàkka def dara, fekk contexte cumulatif bi dafa gaañ model yu tënk yi. Decomposition multi-agent jàppale liggéeykat yu néew doole yi wala sesioŋ yu gëna gudd, ndigam dafay laaj yeneen woote. Li ñu gis mooy li preprint bi wax; source biñ joxe joxeewul pursàntaas yi ci suuf, rang model-ci-model, ñàkka wóor ci lim yi wala misaali njuumte yi.

Ay leeral ci cosaan: arxiv.org ↗

Lu tax mu am solo

Gëstu bi dafay wax ci jafe-jafe yi ci jëfandikoo IA biñ dalal ci defar ay aparey: ay leeral ci komponent yu nëbbu yi ak sàrti joxe tur yi mën nañu soxla jëfandikoo ci gox bi. Li ñu gis mooy jëfandikoo jumtukaay bu wóor a ngi aju ci kàttan model bi kese, waaye itam ci ni ñuy leeralee jumtukaay yi, ba ñaata la agent yi di am ci contexte bi, ak ndax dañu xaaj liggéey bi ci agent yu bari. Loolu dafay may ekipu ingénieur yi tànneef yu fëgër ci jëmmal ngir ñu natt ko balaa ñuy wóolu ndawu liggéey yi.

Jàngat bi dafa am solo ci mbootaay yi mënu yónnee ay leeral ci jëmmal hardware ci API buñ yamale. Këyit dafa wax ni ay jafe-jafe ci wàllu kumpa ci wàllu màndarga yi ak joxe tur yi dañuy faral di waral ñu koy dugal ci gox bi. Su demee noonu, laaj bi du rek ndax sistem IA bi mën na digle kode wala leeral benn sircuit. Sistem bi dafa wara dox ci biir environmaa bu jumtukaay buñ saytu, baña yàq ay dependence ak def coppite yu yeneen jéego yi ci jëmmal mëna jëfandikoo. Benn buñ jagleel jafe-jafe yooyu moo gëna am solo ci jëfandikoo gi, moo gën benn poñ buñ jagleel xeetu làkk wi, doonte source bi wanewul ni benchmark bi dafay wax luy waaja am ci liggéey bi.

Li ñu gis itam dafa soppi bàyyi xel ci tànneef model bi kese, dem ci jëmmal sistemu ndawu liggéey bi. Tegtal yu leer ci jumtukaay yi dañuy wàññi jafe-jafe yi, loolu dafay tekki ni interface bi am ci digganté model bi ak jumtukaayi jëmmal bokk na ci jafe-jafe wóor yi. Gaañ-gaañu yiñ xamle ci contexte cumulatif bu ëpp dafay wane ni jox ab agent taarix bu gëna bari amul benn njariñ ci saasi, rawatina ci model yu tënk yi. Resultaa bi jaxasoo ci few-shot prompting beneen àrtu bu am njariñ la: misaal yiy jàppale benn sistem mën na tax beneen sistem bàyyi liggéey. Kon ekip yiy jàngat ndawu liggéey yi dañu wara natt ay laaj, ay sàrti muy tekki ak ay jumtukaayi jumtukaay, duñu jàpp ni xeetu mbir mi kese mooy soppiku.

Resultaa bi dafay gëna am solo ci tegtali jëfandikoo gi, du firnde ni IA defar na aparey bi boppam. Jawriñ biñ rapoorte mooy coverage bi ñuy seentu ci woote yi, te abstract bi waxul ndax konseptioŋ yi ñu defar méngoo nañu ak sàrti kuuraŋ, waxtu, defar, kaaraange wala saytu. Du wax itam ni dañu koy méngale ak ingenieur nit, otomatisation buñ miin, model yuñ yamale wala script deterministik. Këyit bi itam arXiv preprint la, kon waxtaanam nekkul fii muy gis-gis yuñ xoolaat ci seeni moroom. Valeur publique bi gëna am solo mooy ràññee ay kompromis yu wóor yu ekipu hardware yi mëna amal lànket, boole ci bàyyi kalite ak kaaraange ci génnug ingenieur bu mujj bi te kenn xamul lumuy tekki.

Interactive Mechanism

Mekanism buy weccoo xalaat: naka lay doxee

Saytu xarala yu bees yi ci ginaaw yokkute bii ci anam wu weccoo xalaat.

Agent Lifecycle Stage:
1
User Intent & Planning: "Audit customer refund request #4092 and settle payment."
2
Tool Calling: Emits structured JSON call crm_get_transaction(id='4092').
3
Guardrail & Verification:🛡️ Paused: High-value action requires human operator sign-off.
4
Final Settlement: Refund recorded, email receipt dispatched, and audit log stored.
Core takeaway: An AI agent is not just a language model—it is a closed loop of planning, tool invocation, and environment feedback. Production systems require self-healing retries and strict human approval guardrails.
Saytu konsept buy weccoo xalaat+10 Points
AI Agents Quiz

An agent must create a draft calendar event for Tuesday at 2 p.m. Which evidence would establish the requested result?

Li nga wara seetaan ci topp

Resultaa yi dañu bawoo ci benn preprint ak benn serwër buñ simulee buy wane tolluwaayu logique ak dependence bu jumtukaayu jëmmal hardware. Woytu bi firndewul ni sistem yi defar nañu ay jëmmal yuñ mëna defar, wàññi nañu diiru ingenieur yi wala ñu liggéey ci anam wu wóor ci liggéey bi. Yokkuteg xool dafa wara sëgg ci liggéeyu , dàntite model, njuumti, njuumti, njubte jumtukaay dëgg ak njëgu jëfandikoo agent yu bari.

Këyit dafa wara leer ni bi di màndargaale njariñ, ñaata liggéey ak chaine dependence yi mu am, juróom ñaari model yi ñu natt ak ni resultaa yi di wuutee ci laaj yu baax, yu baaxul ak yu baaxul. Xëtu arXiv biñ joxe dafay firndeel tuuru këyit bi, bindkat yi, bis bi ñu ko joxe ak abstract, waaye du tablo yu leer wala protokol jàngat. Detay yooyu dina ñu wane ndax "jege-matt" coverage-call coverage dafay wane robustness bu yaatu wala performance bu dëgër ci benn set bu gàtt bu simulated workflows.

Jéego bu am solo bi ci topp mooy saytu losisel biñ defaree hardware dëgg ak yeneen liggéey yu bari ci wàllu ingeñër. Serveur MCP dañu ko fësal ni dafay génne staat ak logique dependence bu jumtukaay buñ moom, waaye source bi taxul ñu méngoo ak doxalinu jumtukaay bi yépp wala ak jafe-jafe yi ci projet yu dëggu yi. Firnde yu am njariñ yi ci topp ñooy test yu amaale jëmmal yu gëna mag, coppite yiñ laaj, tontu jumtukaay yu baaxul, defaraat ginaaw woote yu lajj ak xool bu moom boppam ci jëmmal jëmmal. Resultaa yi dañu wara joxe itam latency, token ak njëgu woote jumtukaay, ndax këyit bi dafa wax ni decomposition multi-agent dafay gëna baaxal yenn mbir ci njëgu woote yu bees.

Mbootaay yiy xalaat ci sistem yu mel noonu war nañu xool ndax agent yi dañuy tënk ci coppite yuñ mëna delloo, ndax bépp jëf buy soppi réew mi dañu ko gëm, ak ndax ingenieur nit ñi dañuy xoolaat li génn balaa ñuy jëfandikoo ko ci suuf. Source bi leeralul benn production bu ñuy jëfandikoo, politiku kaaraange wala modelu jëfandikoo-kontrol, kon mënu ñu ni kaaraange yooyu dañu am. Dafay bàyyi itam ubbeeku ci ni ñuy aaree kumpa ci deployment yu dëkk bi ak ndax palanteer yu gëna mag wala model yu gëna am doole di dindi ñakk kattan yiñ xamle. Laaj bi gëna am solo ci liggéey bi ëlëg bokkul ak ndax ab ndawu liggéey mën na def woote bi ñuy seentu, waaye ndax mën na ko def ci anam wu dëppoo, ci wàllu koom-koom ak ci anam wu wóor ci wàllu jëmmal hardware bi yépp.

Gid ak quiz yu ci méngoo

Agent IAModel IA leeral nañu koTaggat ci IATransformatërNatt li nga xam — natt quiz IA bu amul faydaSeetal benn baat IA ci sunu glossaireToppal toppukaayu génne xeetu IA
Gis nga lii am njariñ?